Fostering the Use of Quasi-Experimental Designs for Evaluating Public Health Interventions: Insights from an MNCH mHealth Project in Malawi

Jean Christophe Fotso, Concern Worldwide U.S., Inc.
Jessica Crawford, VillageReach
A. Camielle Noordam, United Nations Children's Fund (UNICEF)
Zachariah Jezman, VillageReach
Ariel Higgins-Steele, Concern Worldwide U.S., Inc.
Amanda Robinson, Ohio State University
Hastings Honde, Invest in Knowledge Initiative (IKI)
Mila Rosenthal, Concern Worldwide U.S., Inc.

As the evidence base to support the growing mHealth sector is relatively nascent, the need to generate evidence through rigorous evaluation methods should be prioritized. This paper describes the design and key findings of an evaluation of a maternal, newborn and child health (MNCH) mHealth project in Malawi, and draws implications for future evaluations. Services offered included a toll-free case management hotline and mobile messaging service for women and children. The evaluation methods included a quasi-experimental pre-test post-test design, consisting of cross-sectional household surveys. Data was collected on about 2840 women and 3585 children at baseline, and 3855 and 3260, respectively at endline. While preliminary results from the intention-to-treat models show no effect on home-based and facility-based care, those from the treatment-on-the-treated models show statistically significant positive effects. Sound analytical methodologies are needed to understand the effects of exposure on outcomes. Implications for future evaluations will be discussed.