From online voices to evidence: Using social media conversations for AI-based evaluation of HIV testing promotion
- Jun 8
- 6 min read
Updated: 2 days ago
By Irasha Jayasekara
EvalYouth Sri Lanka
What if evaluation was not only about measuring outcomes, but about listening deeply, continuously and at scale? Traditionally evaluation has often focused on numbers: indicators, targets, and measurable results. While these are essential, they often overlook something equally important: the human experiences behind those numbers. Today social media platforms offer a new space where people openly share their concerns, questions and lived realities. This creates an opportunity for evaluators to move beyond measurement and toward meaningful listening. This blog explores how AI-assisted qualitative analysis of social media conversations can strengthen the evaluation of HIV testing promotion in Sri Lanka by bringing real voices into evidence. The blog argues that listening-centered evaluation is essential for designing more inclusive, effective and people-centered health interventions.
Limitations of traditional evaluation in HIV testing promotion
Evaluation practices in HIV-related programmes have often relied on structured tools designed to capture measurable change. However, in sensitive public health areas such as HIV, this approach has clear limitations. In Sri Lanka, HIV testing uptake among key populations continues to be influenced by stigma, fear and concerns about confidentiality. These are not easily measurable factors, yet they play a decisive role in shaping health-seeking behavior. Numbers can tell us how many people accessed a service, but they cannot fully explain why others hesitate or avoid it altogether. This highlights a critical gap in evaluation within HIV programmes; one that requires a deeper understanding of human emotions, perceptions and social contexts. Evaluation risks producing incomplete or misleading conclusions without this understanding.
The rise of social media as a source of evaluation evidence
The rapid expansion of digital platforms has transformed how people access and share information. Social media is no longer just a communication tool; it has become a space where individuals seek guidance, express uncertainty and interact in real time. For many people, especially those from marginalized communities, these platforms offer a sense of anonymity and safety that encourages openness. This shift presents a powerful opportunity for evaluation. Instead of relying solely on designed data collection tools, evaluators can now learn from naturally occurring conversations. These interactions provide immediate and authentic insights into user needs and behaviors, opening the door to a more responsive and adaptive form of evaluation that evolves alongside the intervention itself.
Case example: The Know4Sure digital platform in Sri Lanka
The Know4Sure digital platform in Sri Lanka illustrates how this opportunity can be translated into practice. As an online initiative promoting HIV testing among key populations, it generates a continuous stream of user interactions, including comments, messages and inquiries. These interactions reflect real concerns and experiences, often expressed in users’ own words and shaped by their social realities. By applying AI-assisted qualitative analysis, large volumes of such data can be systematically examined to identify patterns and emerging themes. However, the true value of this approach lies not only in its analytical capacity but in its ability to transform evaluation into an ongoing process of listening. Instead of producing static reports, evaluators can engage with real-time feedback, enabling timely adjustments, improved communication strategies and more responsive interventions.
Key user insights: Information needs and decision-making
Listening to these conversations reveals important insights into how individuals think about HIV testing. A dominant pattern is the strong demand for clear and trustworthy information. Many users ask questions about testing procedures, timelines and outcomes, reflecting uncertainty and hesitation. (Example: “I want to get an HIV test. Can you tell me how and where I can do it?”) These information needs are not simply gaps in knowledge; they are part of a broader decision-making process where individuals weigh risks, fears and expectations. Understanding this process is critical for designing interventions that respond to user needs and support informed decision-making.
Barriers to HIV testing: Fear and trust in digital spaces
Fear remains a persistent and powerful barrier. Even within the relative anonymity of digital spaces, users’ express concerns about being identified, judged or stigmatized. (Example: “Is there a safe and anonymous way to get tested for HIV?”) This highlights that increasing service availability alone is not sufficient. Emotional and social dimensions must also be addressed if interventions are to be effective. Closely linked to this is the issue of trust. Users continuously assess the credibility of the platform through its tone, responsiveness and clarity of communication. While quantitative metrics such as reach and engagement rates are useful, they fail to capture this qualitative dimension of user experience. Meaningful engagement depends not just on how many people interact, but on how they perceive and trust the platform, which ultimately influences their willingness to act.
From engagement to action: Understanding user transitions
Importantly, some interactions signal moments of transition, where users move from curiosity toward readiness to act. Questions about where and how to get tested suggest a shift from passive interest to active intention. These moments are particularly valuable from an evaluation perspective, as they highlight opportunities for timely intervention and tailored support. However, the conversations also reveal a gap in continuity. While initial engagement is often strong, there is limited evidence of sustained interaction beyond first contact. This raises important questions about follow-up, long-term engagement and the overall effectiveness of digital interventions in supporting users throughout their journey from awareness to action and beyond. Protecting user privacy, ensuring responsible data use and maintaining transparency are critical for building trust and safeguarding participants in digital environments.
AI-enabled continuous evaluation and human interpretation
These insights point to a broader shift in how evaluation should be understood and practiced. Rather than being a one-time activity conducted at the end of a project, evaluation can become a continuous learning process embedded within programme implementation. AI plays an important role in enabling this shift by making it possible to process large volumes of qualitative data efficiently and consistently. However, human interpretation remains essential to ensure that findings are meaningful, contextually grounded and ethically sound. In sensitive areas such as HIV, where behaviors are shaped by complex social and emotional factors, qualitative evidence provides a depth of understanding that quantitative data alone cannot achieve. At the same time, ethical considerations must remain central.
Challenges and ethical considerations in AI-based evaluation
Despite its potential, this approach is not without challenges. Digital platforms may not equally represent all segments of the population, particularly those with limited access to technology or digital literacy. Privacy concerns are especially significant when dealing with sensitive health-related information, requiring careful and responsible data management practices. In addition, social media environments are often saturated with large volumes of information, including misinformation and contradictory content. In some cases, individuals may rely on inaccurate online information for self-diagnosis or decision-making, which can influence both behaviour and the interpretation of online discussions. The use of AI tools also demands technical capacity and ongoing oversight to ensure accuracy, fairness, and the minimization of bias.
To address these challenges, evaluators need to apply critical and context-sensitive approaches, including triangulating social media data with other data sources, validating findings with stakeholders and maintaining human oversight in AI-assisted analysis. Clear ethical guidelines, digital literacy considerations, and transparent analytical processes are essential to ensure that insights generated are both credible and responsible. These considerations highlight the importance of combining technological innovation with human judgment and ethical awareness in AI-based evaluation.
Conclusion: Evaluation as an act of listening
Ultimately, evaluating social media interventions in Sri Lanka; such as the Know4Sure platform, which uses social media channels like Facebook, WhatsApp and YouTube to promote HIV testing demonstrates that evaluation can be more than measurement; it can be an act of listening.
By engaging with the voices expressed through digital platforms, evaluators can better understand the complex realities that shape health behaviors. AI makes it possible to listen at scale, but it is the responsibility of evaluators to interpret these voices thoughtfully and use them to inform meaningful action. As the field of evaluation continues to evolve in support of the Sustainable Development Goals, approaches that center on listening, learning and responsiveness will become increasingly important. Meaningful evaluation is not just about generating evidence; it is about understanding people and using that understanding to create more inclusive, responsive and effective systems that meet their needs. As HIV-related stigma remains a global challenge, how can we make evaluation systems more inclusive of the voices and lived experiences of those most affected?
Irasha Jayasekara is a Monitoring and Evaluation professional at the Family Planning Association of Sri Lanka and Co-Leader of EvalYouth Sri Lanka. She is passionate about digital evaluation, AI-assisted research and youth-led evaluation approaches. Her work focuses on HIV prevention, social media analytics and strengthening evidence-based public health interventions.
AI Disclaimer : AI tools were used solely to bring the blog to the required length and to correct grammatical issues. The blog's content, ideas, and narrative were authored by the human writer, not generated by AI.
Disclaimer: The content of the blog is the responsibility of the author(s) and does not necessarily reflect the views of Eval4Action co-leaders and partners.
Comments