It almost goes without saying that there should be a strong relationship between evidence and policy. Government interventions that are not based on research and a solid understanding of the context in which they are implemented are less likely to succeed. Similarly, it is important for government to substantiate the claims they make about service delivery using evidence so that genuine accountability can take place.

Having a strong evidence base enables better planning. By understanding what is needed and what has been implemented so far, governments are able to adjust and target their interventions so that they have the greatest positive impact possible. Government (at all levels) will not be able to improve the lives of people living in informal settlements and backyard homes if they do not adequately understand the scale and nature of the issues facing these groups and their broader neighbourhoods. It is also not possible to appropriately evaluate and adjust current strategies if solid evidence is not collected about how and where they have been rolled out. In short, evidence gathering and analysis should play a role in both assessing problems and improving the implementation of solutions.

Evidence and access to data also play a crucial role in improving accountability. Holding governments accountable for their delivery (or nondelivery) of informal settlement upgrading is not possible if we do not know what has been delivered. And while there are statutory requirements that compel municipalities to report on their housing delivery and informal settlement upgrading to National Treasury, this data is often not accessible to the broader public in a simple and understandable format. Instead, people who want to track implementation have to sift through various different documents like Integrated Development Plans (IDPs), Built Environment Performance Plans (BEPPs), and Municipal Spatial Development Frameworks (MSDFs). To make matters more complicated, the formats of IDPs, BEPPs, and MSDFs are not standardised, meaning that it is difficult to make direct comparisons. A lack of data and easily accessible evidence are hindering efforts to track government progress, which has negative implications for informal settlement upgrading and accountability in general. The collection and dissemination of evidence are crucial aspects of service delivery.

A final issue to note relates to the heavy reliance on quantitative evidence. While quantitative evidence is clearly important – it is crucial to know how many sites have been upgraded, how many homes have been delivered, and what this cost – it cannot provide a holistic understanding of policy implementation and its wider effects. Quantitative indicators tell us nothing about the quality of homes or services that are delivered. Similarly, they reveal very little about how this delivery has impacted quality of life for beneficiaries. As a result, much of the housing-related evidence that is produced is relatively limited in what it tells us about broader spatial transformation and the way that government interventions are experienced.

There are reasons for the dominance of quantitative indicators. They are perceived as the easiest way to measure progress, they more easily allow comparisons, and they are attractive to politicians. However, they obscure important information, and relying on them alone does not produce ‘deeper’ knowledge.

Another example of the inadequacy of only using quantitative indicators is what they reveal, or rather do not reveal, about the process of policy implementation itself. By focusing narrowly on inputs (the money spent) and outputs (the number of sites serviced or homes delivered), quantitative indicators do not reveal any information about how the process of implementation went. Barriers and blockages are not examined, and the institutional and interpersonal aspects of service delivery are ignored. As a result, governments that rely exclusively on quantitative evidence are unlikely to be able to identify things that went well, things that went badly, and things that can be improved.