1-14 of 14 Results

  • Keywords: experiments x
Clear all

Article

The field of political science is experiencing a new proliferation of experimental work, thanks to a growth in online experiments. Administering traditional experimental methods over the Internet allows for larger and more accessible samples, quick response times, and new methods for treating subjects and measuring outcomes. As we show in this chapter, a rapidly growing proportion of published experiments in political science take advantage of an array of sophisticated online tools. Indeed, during a relatively short period of time, political scientists have already made huge gains in the sophistication of what can be done with just a simple online survey experiment, particularly in realms of inquiry that have traditionally been logistically difficult to study. One such area is the important topic of social interaction. Whereas experimentalists once relied on resource- and labor-intensive face-to-face designs for manipulating social settings, creative online efforts and accessible platforms are making it increasingly easy for political scientists to study the influence of social settings and social interactions on political decision-making. In this chapter, we review the onset of online tools for carrying out experiments and we turn our focus toward cost-effective and user-friendly strategies that online experiments offer to scholars who wish to not only understand political decision-making in isolated settings but also in the company of others. We review existing work and provide guidance on how scholars with even limited resources and technical skills can exploit online settings to better understand how social factors change the way individuals think about politicians, politics, and policies.

Article

Empirical media effects research involves associating two things: measures of media content or experience and measures of audience outcomes. Any quantitative evidence of correlation between media supply and audience response—combined with assumptions about temporal ordering and an absence of spuriousness—is taken as evidence of media effects. This seemingly straightforward exercise is burdened by three challenges: the measurement of the outcomes, the measurement of the media and individuals’ exposure to it, and the tools and techniques for associating the two. While measuring the outcomes potentially affected by media is in many ways trivial (surveys, election outcomes, and online behavior provide numerous measurement devices), the other two aspects of studying the effects of media present nearly insurmountable difficulties short of ambitious experimentation. Rather than find solutions to these challenges, much of collective body of media effects research has focused on the effort to develop and apply survey-based measures of individual media exposure to use as the empirical basis for studying media effects. This effort to use survey-based media exposure measures to generate causal insight has ultimately distracted from the design of both causally credible methods and thicker descriptive research on the content and experience of media. Outside the laboratory, we understand media effects too little despite this considerable effort to measure exposure through survey questionnaires. The canonical approach for assessing such effects: namely, using survey questions about individual media experiences to measure the putatively causal variable and correlating those measures with other measured outcomes suffers from substantial limitations. Experimental—and sometimes quasi-experimental—methods provide definitely superior causal inference about media effects and a uniquely fruitful path forward for insight into media and their effects. Simultaneous to this, however, thicker forms of description than what is available from close-ended survey questions holds promise to give richer understanding of changing media landscape and changing audience experiences. Better causal inference and better description are co-equal paths forward in the search for real-world media effects.

Article

Political scientists are increasingly using experiments to study African politics. Experimental methods help scholars to overcome two central research challenges: potential bias in responses to survey questions (social desirability bias), and establishing the effect of X on Y (causality). Regarding survey response bias, experimental methods have been used to study sensitive topics such as ethnic favoritism, clientelism, corruption, and vote buying. In terms of causality, experiments have helped to estimate the effects of programs aimed at enhancing the quality of democracy or public service delivery. Identifying the causes of the political behavior is critical to understanding the “nuts and bolts” of African politics. For policymakers, knowledge of what works to promote democratic accountability ensures the efficient allocation of scarce resources.

Article

Paul A. Djupe and Amy Erica Smith

Experiments in religion and politics model a communication system with three elements: who (the sample) is exposed to what (the treatment) and with what potential effect (the outcome). Most experiments in religion and politics focus on one of three types of samples: clergy, the faithful within certain religious groups, or all citizens within a polity. At the core of the experiment is the randomized treatment: an independent variable that the researcher manipulates and randomly assigns to treatment groups that are supposed to be equivalent in all other respects. Certain kinds of treatments tend to be associated with certain kinds of hypothesized outcomes. That is, most experiments in religion and politics involve investigating either (a) how a randomized treatment related to religion affects a political outcome or (b) how a randomized treatment related to politics affects a religious outcome. There are several types of religious treatments that closely mirror the actual insertion of religion into public life: manipulating candidates’ religious affiliations, behavior, and rhetoric; manipulating appeals attributed to religious elites and institutions; priming subjects’ own religious or political beliefs or manipulating other religious attributes of subjects; manipulating the characteristics of other citizens; and manipulating religious institutional cues received by clergy. Experimental methods are everywhere now in the study of religion and politics and provide clear benefits for understanding how religion and politics interact. Perhaps most importantly, the method imposes intellectual rigor, helping scholars pin down theoretically and empirically the precise mechanisms involved in the mutual impact between religion and politics. In addition, experimental control enables scholars to assert more confidently the direction of influence among variables that in the real world plausibly influence each other.

Article

Students of public opinion tend to focus on how exposure to political media, such as news coverage and political advertisements, influences the political choices that people make. However, the expansion of news and entertainment choices on television and via the Internet makes the decisions that people make about what to consume from various media outlets a political choice in its own right. While the current day hyperchoice media landscape opens new avenues of research, it also complicates how we should approach, conduct, and interpret this research. More choices means greater ability to choose media content based on one’s political preferences, exacerbating the severity of selection bias and endogeneity inherent in observational studies. Traditional randomized experiments offer compelling ways to obviate these challenges to making valid causal inferences, but at the cost of minimizing the role that agency plays in how people make media choices. Resent research modifies the traditional experimental design for studying media effects in ways that incorporate agency over media content. These modifications require researchers to consider different trade-offs when choosing among different design features, creating both advantages and disadvantages. Nonetheless, this emerging line of research offers a fresh perspective on how people’s media choices shapes their reaction to media content and political decisions.

Article

Aubrey Westfall and Özge Çelik Russell

Religion is a central and comprehensive identity for billions of people all over the world. Politicians and other political actors recognize the vitality of religion and use it for political purposes, deliberately signaling religion, religiosity, or religious values and connecting them to political outcomes or behaviors in an effort to influence the political preferences of religious practitioners. The most efficient way to make the connection between religion and politics is through religious cues. Religious cues create information shortcuts linking religious identity or values with a political candidate or issue. Religious cues are used by political and religious actors in secular and religious contexts and are typically one of two general types: identity cues, which engage an individual’s religious identity and activate an in-group/out-group effect, and linkage cues, which link religious values or beliefs with an issue or candidate. Identity cues are particularly tricky to use in secular contexts because they have been shown to have strong alienating effects on nonreligious people, thereby defeating the intended purpose of the cue sender. For this reason, coded religious language called “implicit cues” is used with greater frequency in political discourse where only the religious cue receiver recognizes the religious cue for what it is. This strategy allows a political candidate to reap the benefits of the cue without risking alienation. While scholars have made substantial progress in using experimental methods to disentangle the ways religious cues influence political behavior, there is ample opportunity for more research exploring different types of religious cues and the way they interact with other forms of cues and identities. Furthermore, most of the research on religious cues has focused on Christian cues in the United States, and a more diverse range of religions and contexts should be explored to understand the way religious cues influence political behavior. Researchers should also expand the definition of “religious practitioners” to explore how religious cues influence the growing number of people who do not affiliate with a religion or engage in practices traditionally associated with religiosity but do identify as religious. This would help to expand conceptualization of political behavior to more accurately reflect lived political experiences. Embracing these opportunities will allow the scholarly community to gain a better understanding of the varied political dynamics of religious cueing, which offers insights into how fundamental identities and attitudes are linked, thereby shedding more light on the complex dynamics of political behavior.

Article

It can be difficult for political scientists and economists to know when to use laboratory experiments in their research programs. There are longstanding concerns in economics and political science about the external invalidity of laboratory results. Making matters worse, a number of prominent academics recommend using field experiments instead of laboratory experiments to learn about human behavior because field experiments do not have the same external invalidity problems that plague laboratory experiments. The criticisms of laboratory experiments as externally invalid, however, overlook the many advantages of laboratory experiments that derive from their external invalidity. Laboratory experiments are preferable to field experiments at examining hypothetical scenarios (e.g., When automated vehicles dominate the roadways, what principles do people want their automobiles to rely on?), at minimizing erroneous causal inferences (e.g., Did a treatment produce the reaction researchers are studying?), and at replicating and extending previous studies. Rather than being a technique that should be abandoned in favor of field experiments, political scientists and economists should embrace laboratory experiments when testing theoretically important but empirically unusual scenarios, tracing experimental processes, and reproducing and building on prior experiments.

Article

Micah Dillard and Jon C.W. Pevehouse

Scholarship in international relations has taken a more quantitative turn in the past four decades. The field of foreign policy analysis was arguably the forerunner in the development and application of quantitative methodologies in international relations. From public opinion surveys to events data to experimental methods, many of the earliest uses of quantitative methodologies can be found in foreign policy analysis. On substantive questions ranging from the causes of war to the dynamics of public opinion, the analysis of data quantitatively has informed numerous debates in foreign policy analysis and international relations. Emerging quantitative methods will be useful in future efforts to analyze foreign policy.

Article

Jiawei Liu and Dietram A. Scheufele

There is a dichotomy in framing research that can be traced back to its multidisciplinary origins in psychology and sociology. Definitions of framing rooted in psychology are concerned with the differential presentation of the otherwise identical information and are often referred to as equivalence framing. Definitions rooted in more sociological traditions investigate how a message can be constructed with different sets of information to highlight contrasting perspectives on the same issue. The latter is typically referred to as emphasis framing. Although often subsumed under the same label, equivalence framing and emphasis framing are systematically different, both conceptually and operationally. Therefore, the two traditions need to be carefully distinguished in terms of their origins, conceptualization and operationalization of frames, underlying mechanisms, cognitive outcomes, and their relationships with other media effects theories. Categorizing existing studies revealed two major pitfalls in framing effects literatures. First, many political communication studies to date have adopted the emphasis framing approach. However, as substantial manipulation of information introduces confounding variables making it difficult for researchers to attribute the effect on the audience to the change of frames, this approach has relatively low internal validity in experiments and can hardly be distinguished from other cognitive media effects models, such as agenda setting and priming. Thus, the bias toward emphasis framing needs to be addressed by conducting research with equivalence frames so that a more concrete causal relationship between message framing and its effects can be established. In addition, little attention has been given to visuals in framing effects research so far. Considering that people consume information in a multimedia environment online, visual frames and verbal-visual interactions need to be further investigated.

Article

Public diplomacy has become an essential subject for both practitioners of foreign policy and scholars of international relations/world politics. The more the term achieves popularity and is used in policy papers, magazines, academic books, and articles, the greater the number of different definitions of the concept. Unfortunately, no universally agreed-upon definition exists. With regard to the international relations debate on the “-isms,” some researchers claim that public diplomacy is part of constructivism. Yet, while it may be appropriate to categorize public diplomacy as constructivist for norm-oriented reputation politics such as “naming and shaming,” many realists working from the rationalist paradigm have recognized the importance of public diplomacy in international relations. Recently, beyond discussions on definitions and scope of public diplomacy, many data-oriented, empirical studies have been published on the subject. For instance, moves have been made to rank which state can achieve the greatest level of soft power through the effective practice of public diplomacy. Moreover, quantitative text analysis (QTA) or content analysis frameworks have frequently been utilized to study how international media focus on controversial diplomatic issues between states. Even tweets and social networks are being studied to reveal what types of international diplomatic communications are supported and opposed by third-party domestic audiences. Rapid developments continue to be made in the methodological sophistication of public diplomacy studies.

Article

Field experiments allow researchers on political behavior to test causal relationships between mobilization and a range of outcomes, in particular, voter turnout. These studies have rapidly increased in number since 2000, many assessing the impact of nonpartisan Get-Out-the-Vote (GOTV) campaigns. A more recent wave of experiments assesses ways of persuading voters to change their choice of party or alter their social and political attitudes. Many studies reveal positive impacts for these interventions, especially for GOTV. However, there are far fewer trials carried out outside the United States, which means it is hard to confirm external validity beyond the U.S. context, even though many comparative experiments reproduce U.S. findings. Current studies, both in the United States and elsewhere, are growing in methodological sophistication and are leveraging new ways of measuring political behavior and attitudes.

Article

Alessandro Del Ponte, Reuben Kline, and John Ryan

Behavioral economics is an interdisciplinary field of inquiry that incorporates insights from psychology to enrich standard economic models that assume perfectly rational individuals. Empirical research in behavioral economics typically employs incentivized experiments that use economic games with real money on the line. In these experiments, subjects are awarded financial payoffs based on the decisions they make (either individually or as part of a group) in an institutional context designed by the researcher. Behavioral economics is well suited for political science because behavioral economics is interdisciplinary by nature and political science is not bound by any particular research paradigm. At the same time, the method is still novel to many political scientists despite many years of its use to study political topics in a variety of research areas. What unites the application of the method to these areas is the explicit consideration of conflict. For instance, scholars have uncovered social conflict between groups (e.g., voter polarization in the United States) using behavioral games as measures, or they have designed experiments around elections to test theories of candidate and voter behavior. Because of the clear financial incentives, economic experiments are especially useful for studying people’s actual preferences in areas such as redistribution as opposed to their stated preferences. Finally, the method can be used to design institutions that will help overcome conflict over scarce resources. In sum, the strengths of behavioral economics include: (a) the ability to vary institutional contexts; (b) clear incentives that ensure valid measures of preferences; (c) direct measures of behaviors instead of stated intentions which could be confounded by outside pressures such as social desirability.

Article

Recognizing its causal power, contemporary scholars of media effects commonly leverage experimental methodology. For most of the 20th century, however, political scientists and communication scholars relied on observational data, particularly after the development of scientific survey methodology around the mid-point of the century. As the millennium approached, Iyengar and Kinder’s seminal News That Matters experiments ushered in an era of renewed interest in experimental methods. Political communication scholars have been particularly reliant on experiments, due to their advantages over observational studies in identifying media effects. Although what is meant by “media effects” has not always been clear or undisputed, scholars generally agree that the news media influences mass opinion and behavior through its agenda-setting, framing, and priming powers. Scholars have adopted techniques and practices for gauging the particular effects these powers have, including measuring the mediating role of affect (or emotion). Although experiments provide researchers with causal leverage, political communication scholars must consider challenges endemic to media-effects studies, including problems related to selective exposure. Various efforts to determine if selective exposure occurs and if it has consequences have come to different conclusions. The origin of conflicting conclusions can be traced back to the different methodological choices scholars have made. Achieving experimental realism has been a particularly difficult challenge for selective exposure experiments. Nonetheless, there are steps media-effects scholars can take to bolster causal arguments in an era of high media choice. While the advent of social media has brought new challenges for media-effects experimentalists, there are new opportunities in the form of objective measures of media exposure and effects.

Article

Survey experiments are now quite common in political science. A recent analysis of the number of mentions of this term in political science journal articles demonstrates a dramatic increase from 2000 to 2013. In addition, the term survey experiment has been picked up by many other disciplines, by researchers in a variety of different countries. Given the large number of survey experiments already published, the goal here is not to review the numerous excellent studies using this methodology, because there are far too many, spanning too many different topics. Instead, this juncture—marked by both progress and the proliferation of this method—is used to highlight some of the issues that have arisen as this methodological approach has come of age. How might research using this methodology improve in political science? What are the greatest weaknesses of survey experimental studies in this discipline to date? The explosive growth of survey experiments in political science speaks to their popularity as a means of establishing causal inference. In his reflection on the origins of survey experiments, Paul Sniderman has suggested that their quick rise in popularity was due to two factors: a) their ability to meet expected standards of external validity within the discipline without sacrificing internal validity, and b) the lower marginal cost per study relative to studies that were representative national surveys. Collaborative data collection efforts such as the Multi-Investigator Project and Time-sharing Experiments for the Social Sciences (TESS) made it possible for more scholars to execute population-based survey experiments at a lower cost per study than traditional surveys. Using shared platforms, researchers can execute many experiments for the price of one representative survey. These explanations make perfect sense in the context of a field such as political science, where external validity traditionally has been valued more highly than internal validity. It may be surprising to younger colleagues to learn that, not all that long ago, experiments were deemed completely inappropriate within the discipline of political science, unless they were field experiments executed in the real world. Experiments involving interventions in naturally occurring political environments were deemed tolerable, but only political psychologists were likely to find experimentation more broadly acceptable due to their strong ties to psychology. In political science, survey experiments were a means of promoting experimental methods in an external-validity-oriented discipline. Survey experiments freed political scientists from college sophomores as subjects and promised that external validity need not be sacrificed for strong causal inference. Times have obviously changed, and political scientists now embrace a much broader array of methodologies including both observational and experimental methods. This occasion provides an opportunity to re-evaluate the strengths and weaknesses of this innovative method, in theory and in practice.