By Krithika Rajagopalan and Shagun Sabarwal
CLEAR South Asia
The COVID-19 pandemic has exacerbated existing development problems such as unemployment, poverty, and inequality, underscoring the need for carefully framing policies using data and evidence that is generated through reliable and strong monitoring & evaluation (M&E) systems. Many governments in low- and middle- income countries collect data but may not use it well. There is an opportunity to use emerging technologies and systems based on Artificial Intelligence (AI) and machine learning to improve real-time decision-making. For technological innovations to be truly useful in driving effective decisions which affect millions of people, we need to improve the data quality, reliability of information generated and system readiness to absorb these technologies.
The importance of decisions based on scientific evidence and facts cannot be understated amid the rising spread of misinformation which has also complicated the war against the pandemic. At the level of policymaking, one of the ways to ensure this is through M&E data that is generated in-house or by third-party evaluators. However, low quality data or the absence of the value of M&E data can lead to decisions taken which may be ineffective or in some cases, counter-productive. Studies suggest that even well-drafted programmes that appear to work well based on data collected on administrative indicators, can fail to work in practice.
For technological innovations to be truly useful in driving effective decisions which affect millions of people, we need to improve the data quality, reliability of information generated and system readiness to absorb these technologies.
Some of the common barriers to good quality data include system-level challenges such as inconsistency in the mode of data collection across various data generating units, and man-made errors at the data collection and data entry stages. Lack of proper integration of these databases might create duplicates, and might also lead to the publication of a combined version of data collected by different units at different points in time. This is problematic because such data may not be entirely representative.
The underlying economic, political and social institutions also need to be conducive to collect good quality data. It is important for government departments to lay down comprehensive data collection protocols to collect quality data on policy processes and outcomes, secure the data that is collected, and institute an effective monitoring process to ensure adherence to protocol. Such protocols can enable consistency and resilience in the face of changes in the political system or staff within government departments. In our work at the CLEAR South Asia Center, which is hosted at the Abdul Latif Jameel Poverty Action Lab (J-PAL) South Asia, we engage in many such capacity building and advisory services with government partners to help them use quality evidence to inform decisions. We also conduct workshops on helping stakeholders improve data quality through content such as conducting data quality checks and the design of robust data collection systems.
It is important for government departments to lay down comprehensive data collection protocols to collect quality data on policy processes and outcomes, secure the data that is collected, and institute an effective monitoring process to ensure adherence to protocol.
What did this work look like during the pandemic? There has been an increased prevalence of remote data collection over the last year. Researcher interest in methods such as Computer Assisted Telephonic Interviews (CATI), Interactive Voice Response Systems (IVRs), and self-reported surveys sent to respondents through SMS or the internet have seen a substantial increase globally. Some prerequisites for weaving them at a large scale into existing systems include digital literacy of respondents and survey staff, increased mobile-use penetration in a region, and internet and signal prevalence. Investing in these basics can help continue the propagation of low-cost remote data collection while ensuring quality data.
Predictive analysis using AI is another recent innovation, which seeks to use large datasets from previous economic events to predict future events, and incorporates various interlinked factors affecting the outcome of interest. Poor quality data will constrain the ability to observe patterns in such a relationship. The foundation for this must be built on a deep understanding of the geographical region and context on which the data collection and analysis is centred, and requires personnel trained in machine learning[1]. New methods also include geospatial analysis in policy areas such as urban planning and transportation planning, which require the availability of internet facilities and accurate GIS mapping. Climate change is an emerging policy area that can benefit from the geospatial analysis[2] for new policy recommendations.
The true potential for technological advancement in M&E can be harnessed only with a conducive infrastructure and enabling environment.
Maintaining security to avoid misuse of the published data and documentation is important as governments, donors, and researchers use technology for activities of varying levels of complexity. The many threats to the security of administrative and government data can range from intentional breaches into government platforms or authorised persons/departments unintentionally exposing information to the public due to lack of adequate skills or awareness such as intentional unauthorised access by hackers, unintentional breach of security (lax security protocols by the authorised user) and authorised users illegally using data for unauthorised purposes. To avoid such breach of security, existing systems can be empowered using reliable data storage mechanisms such as encrypted hard disks, encrypted data transfer processes, and restrictions on access networks.
The true potential for technological advancement in M&E can be harnessed only with a conducive infrastructure and enabling environment. Building capacity on the use of such technology is one vital component that we address at CLEAR South Asia through training courses on digital modes of data collection and measurement.
The current crisis has highlighted the need to revisit these systems to ensure that innovations actually translate into actionable policies.
We have a long way before M&E is used as effectively as possible to inform important decisions and improve development outcomes. The current crisis has highlighted the need to revisit these systems to ensure that innovations actually translate into actionable policies. As technological innovations abound, the focus should be on supporting currently fragmented M&E systems to move from data generation towards data use, and work towards improving their readiness to absorb technology.
[1] A type of computer programme that performs specific tasks such as data cleaning and analysis based on information inputted by users.
[2] Geospatial Information is a digital replication of the physical world in which all human, economic and environmental activity takes place. Analysing this information to guide decisions in termed as geospatial analysis.
Krithika Rajagopalan is a Senior Training Associate at CLEAR South Asia and J-PAL South Asia. She supports the team in partnership development with governments, civil society organisations, donors and multilateral organizations. Follow Krithika on Twitter.
Shagun Sabarwal is the Director of CLEAR South Asia and the Director of Policy, Training, and Communications at J-PAL South Asia. She promotes the Center’s mission to strengthen the monitoring, evaluation, learning systems, and data use of decision-makers in the region through capacity building and advisory services. Follow Shagun on Twitter.
