Oxford Research Encyclopedia of Politics reached a major milestone by publishing our 1000th article! For more information visit our News page.

Show Summary Details

Page of

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, POLITICS (oxfordre.com/politics). (c) Oxford University Press USA, 2020. All Rights Reserved. Personal use only; commercial use is strictly prohibited (for details see Privacy Policy and Legal Notice).

date: 14 July 2020

Summary and Keywords

Survey experiments are now quite common in political science. A recent analysis of the number of mentions of this term in political science journal articles demonstrates a dramatic increase from 2000 to 2013. In addition, the term survey experiment has been picked up by many other disciplines, by researchers in a variety of different countries. Given the large number of survey experiments already published, the goal here is not to review the numerous excellent studies using this methodology, because there are far too many, spanning too many different topics. Instead, this juncture—marked by both progress and the proliferation of this method—is used to highlight some of the issues that have arisen as this methodological approach has come of age. How might research using this methodology improve in political science? What are the greatest weaknesses of survey experimental studies in this discipline to date?

The explosive growth of survey experiments in political science speaks to their popularity as a means of establishing causal inference. In his reflection on the origins of survey experiments, Paul Sniderman has suggested that their quick rise in popularity was due to two factors: a) their ability to meet expected standards of external validity within the discipline without sacrificing internal validity, and b) the lower marginal cost per study relative to studies that were representative national surveys. Collaborative data collection efforts such as the Multi-Investigator Project and Time-sharing Experiments for the Social Sciences (TESS) made it possible for more scholars to execute population-based survey experiments at a lower cost per study than traditional surveys. Using shared platforms, researchers can execute many experiments for the price of one representative survey.

These explanations make perfect sense in the context of a field such as political science, where external validity traditionally has been valued more highly than internal validity. It may be surprising to younger colleagues to learn that, not all that long ago, experiments were deemed completely inappropriate within the discipline of political science, unless they were field experiments executed in the real world. Experiments involving interventions in naturally occurring political environments were deemed tolerable, but only political psychologists were likely to find experimentation more broadly acceptable due to their strong ties to psychology. In political science, survey experiments were a means of promoting experimental methods in an external-validity-oriented discipline. Survey experiments freed political scientists from college sophomores as subjects and promised that external validity need not be sacrificed for strong causal inference.

Times have obviously changed, and political scientists now embrace a much broader array of methodologies including both observational and experimental methods. This occasion provides an opportunity to re-evaluate the strengths and weaknesses of this innovative method, in theory and in practice.

Keywords: survey experiment, manipulation check, confounding, balance test, crowd-sourced samples, generalizability, internal validity, external validity, political decision making

Access to the complete content on Oxford Research Encyclopedia of Politics requires a subscription or purchase. Public users are able to search the site and view the abstracts and keywords for each book and chapter without a subscription. If you are a student or academic complete our librarian recommendation form to recommend the Oxford Research Encyclopedias to your librarians for an institutional free trial.

Please subscribe or login to access full text content.

If you have purchased a print title that contains an access token, please see the token for information about how to register your code.

For questions on access or troubleshooting, please check our FAQs, and if you can't find the answer there, please contact us.