An Experimental Framework for Continual Improvement in Survey Research

Dennis Feehan, Princeton University
Matthew J. Salganik, Princeton University

Surveys are an essential measurement tool for many of the most important theoretical and policy questions in the social sciences. Unfortunately, in order to measure the things we care about with surveys, we often have to make difficult decisions about exactly how we should collect information from our respondents. Our paper begins by describing such a situation that we encountered in a study of populations most at risk for HIV in Rwanda. We describe how we conducted a survey experiment and exploited known quantities to gather evidence about how to best measure unknown quantities. We then generalize our experience into a framework that would allow researchers in a wide variety of contexts to steadily accumulate evidence about best practices by embedding experiments in their surveys. Each new survey can be an opportunity to add more to the body of knowledge available, continually improving the quality of everyone's estimates.