Nonnecke, B., Krishnan, S., Akula, A., Cliff, A., Huang, A., Lin, A., Nehama, S., Byaruhanga, L., Masinde, D., Crittenden, C., & Goldberg, K. (May 2015). A mobile platform for participatory assessment of development programs and a case study in Uganda. Paper presented at the Humanitarian Technology Conference: Science, Systems and Global Impact. Boston, MA.
Development organizations need timely and reliable feedback on the efficacy of program interventions. Traditional assessment methodologies tend to be costly, time-consuming, and lack interaction: participants provide responses but seldom have access to the data or a role in its interpretation. We developed the Collaborative Assessment and Feedback Engine (CAFE) to engage communities from developing regions in collective assessment of local conditions, needs, and outcomes of development programs (see Figure 1).
The CAFE platform collects data in three phases: (1) Quantitative assessment, where participants submit responses on a scale from 0 to 10, “Strongly Disagree” to “Strongly Agree”, establishing baseline participant demographics and opinions. (2) Ideation, where participants provide a textual response to an open-ended question. (3) Collaborative evaluation, where participants evaluate the responses of others. CAFE provides researchers and participants with rapid, preliminary insights while in the field and can be used without Internet connectivity by taking advantage of ad-hoc networks (e.g., mobile devices networked to a laptop).
CAFE uses peer review to filter the most insightful ideas and Principal Component Analysis (PCA) to group participants based on their answers in the quantitative assessment phase. Insights from the statistical analysis are displayed to facilitate the discussion and incentivize participation. For example, median scores from the quantitative assessment and anonymous textual responses from the ideation phase are displayed immediately.
We present a case study on use of CAFE to assess family planning and reproductive health (FPRH) trainings at three health centers in Uganda in June 2014. Data were collected through CAFE from 137 women, including demographics, quantitative assessments of FPRH knowledge, and qualitative feedback on how to improve FPRH education strategies. Our data suggest that PCA can extract insights for further cross-sectional demographic analysis. For example, CAFE allowed us to identify that more tailored and locally relevant trainings are necessary to increase efficacy of FPRH outreach at each center. Our preliminary results suggest CAFE can collect reliable, direct and timely feedback on the efficacy of development programs.