No.57 Using List Experiments to Study Vote Buying: Do List Experiments Run as Expected?

Kristen Kao and Ellen Lust 

Abstract 

A fundamental premise of list experiments is that they allow respondents to hide sensitive attitudes and behaviors among a list of other items. List experiments accomplish this by asking the respondent to count both sensitive and innocuous items, rather than answering questions directly about each item. Social scientists widely employ list experiments to overcome sensitivity bias but have not yet systematically studied whether the complicated nature of these experiments leads to implementation errors. Analysis of a list experiment across three countries suggests that respondents reveal their direct responses to list items more than half of the time. This problem is particularly prevalent among less educated and older respondents, and the complicated nature of the question mediates the relationships between age and education and reveals answers. Kao and Lust encourage scholars to include questions about implementation problems as standard follow-ups in list experiments to understand if they worked properly in the field.

Keywords: List experiment, survey methodology, Sub-Saharan Africa, sensitivity bias, social desirability bias