Summary

Since the 2010s, online surveys have become a popular method among criminologists. Often these surveys are conducted with the assistance of private survey research companies, which gather large groups of people (i.e., respondents) who have indicated a willingness to share their opinions on a variety of issues. These panels of potential respondents vary in size and quality. Researchers planning to collect survey data via these online panels must also consider probability versus non-probability sampling methods. Probability samples provide stronger assurances that sample statistics—particularly, univariate point estimates—are generalizable to broader populations (e.g., adult Americans). They are also often very expensive, although this is somewhat dependent on the size and complexity of the proposed project. Two popular providers of probability samples of the American public are the Ipsos Knowledge Panel and the AmeriSpeak Omnibus panel. In criminology and criminal justice, researchers have used online probability panels to study a variety of topics, including behaviors regarding firearms, attitudes toward policing, and experiences of violence.

Non-probability samples present a budget-friendly alternative but may be less generalizable to populations of interest. Since 2010, these samples have become especially popular in the criminological literature and are much more commonly used than online probability samples. Findings from non-probability online surveys often yield remarkably similar relational inferences (e.g., correlations) to those obtained from probability samples. However, non-probability samples are generally unsuitable for providing generalizable univariate point estimates. Some of the leading providers of non-probability samples from panels are YouGov, Qualtrics, and Lucid. As of 2024, YouGov uses a matched opt-in sample with a more sophisticated sampling design, while Qualtrics and Lucid provide quota samples. Researchers may also directly recruit non-probability samples of respondents via crowdsourcing platforms, such as Amazon Mechanical Turk, or services that incorporate those platforms into their own business model, such as CloudResearch. Research suggests that platforms with more sophisticated sampling procedures tend to yield more accurate results. Consequently, matched opt-in samples such as YouGov are approximately twice as expensive as Qualtrics samples and are many times more expensive than crowdsourcing platforms. Finally, it should be noted that the demographic composition of online samples, even those that have been simply crowdsourced, tend to be more diverse than typical in-person non-probability samples used in criminology and criminal justice research (e.g., college students).