1-3 of 3 Results

  • Keywords: experiments x
Clear all


Experimental Designs in Business Research  

Heiko Breitsohl

Conducting credible and trustworthy research to inform managerial decisions is arguably the primary goal of business and management research. Research design, particularly the various types of experimental designs available, are important building blocks for advancing toward this goal. Key criteria for evaluating research studies are internal validity (the ability to demonstrate causality), statistical conclusion validity (drawing correct conclusions from data), construct validity (the extent to which a study captures the phenomenon of interest), and external validity (the generalizability of results to other contexts). Perhaps most important, internal validity depends on the research design’s ability to establish that the hypothesized cause and outcome are correlated, that variation in them occurs in the correct temporal order, and that alternative explanations of that relationship can be ruled out. Research designs vary greatly, especially in their internal validity. Generally, experiments offer the strongest causal inference, because the causal variables of interest are manipulated by the researchers, and because random assignment makes subjects comparable, such that the sources of variation in the variables of interest can be well identified. Natural experiments can exhibit similar internal validity to the extent that researchers are able to exploit exogenous events creating (quasi-)randomized interventions. When randomization is not available, quasi-experiments aim at approximating experiments by making subjects as comparable as possible based on the best available information. Finally, non-experiments, which are often the only option in business and management research, can still offer useful insights, particularly when changes in the variables of interest can be modeled by adopting longitudinal designs.


Natural Experiments in Business Research Methods  

Michael C. Withers and Chi Hon Li

Causal identification is an important consideration for organizational researchers as they attempt to develop a theoretical understanding of the causes and effects of organizational phenomena. Without valid causal identification, insights regarding organizational phenomena are challenging given their inherent complexity. In other words, organizational research will be limited in its scientific progression. Randomized controlled experiments are often suggested to provide the ideal study design necessary to address potential confounding effects and isolate true causal relationships. Nevertheless, only a few research questions lend themselves to this study design. In particular, the full randomization of subjects in the treatment and control group may not be possible due to the empirical constraints. Within the strategic management area, for example, scholars often use secondary data to examine research questions related to competitive advantage and firm performance. Natural experiments are increasingly recognized as a viable approach to identify causal relationships without true random assignment. Natural experiments leverage external sources of variation to isolate causal effects and avoid potentially confounding influences that often arise in observational data. Natural experiments require two key assumptions—the as-if random assignment assumption and the stable unit treatment value assumption. When these assumptions are met, natural experiments can be an important methodological approach for advancing causal understanding of organizational phenomena.


Experiments in Organization and Management Research  

Alex Bitektine, Jeff Lucas, Oliver Schilke, and Brad Aeon

Experiments randomly assign actors (e.g., people, groups, and organizations) to different conditions and assess the effects on a dependent variable. Random assignment allows for the control of extraneous factors and the isolation of causal effects, making experiments especially valuable for testing theorized processes. Although experiments have long remained underused in organizational theory and management research, the popularity of experimental methods has seen rapid growth in the 21st century. Gatekeepers sometimes criticize experiments for lacking generalizability, citing their artificial settings or non-representative samples. To address this criticism, a distinction is drawn between an applied research logic and a fundamental research logic. In an applied research logic, experimentalists design a study with the goal of generalizing findings to specific settings or populations. In a fundamental research logic, by contrast, experimentalists seek to design studies relevant to a theory or a fundamental mechanism rather than to specific contexts. Accordingly, the issue of generalizability does not so much boil down to whether an experiment is generalizable, but rather whether the research design matches the research logic of the study. If the goal is to test theory (i.e., a fundamental research logic), then asking the question of whether the experiment generalizes to certain settings and populations is largely irrelevant.