The recent “replication crisis” in the social sciences has led to increased attention on what statistically significant results entail. There are many reasons for why false positive results may be published in the scientific literature, such as low statistical power and “researcher degrees of freedom” in the analysis (where researchers when testing a hypothesis more or less actively seek to get results with p < .05). The results from three large replication projects in psychology, experimental economics, and the social sciences are discussed, with most of the focus on the last project where the statistical power in the replications was substantially higher than in the other projects. The results suggest that there is a substantial share of published results in top journals that do not replicate. While several replication indicators have been proposed, the main indicator for whether a results replicates or not is whether the replication study using the same statistical test finds a statistically significant effect (p < .05 in a two-sided test). For the project with very high statistical power the various replication indicators agree to a larger extent than for the other replication projects, and this is most likely due to the higher statistical power. While the replications discussed mainly are experiments, there are no reasons to believe that the replicability would be higher in other parts of economics and finance, if anything the opposite due to more researcher degrees of freedom. There is also a discussion of solutions to the often-observed low replicability, including lowering the p value threshold to .005 for statistical significance and increasing the use of preanalysis plans and registered reports for new studies as well as replications, followed by a discussion of measures of peer beliefs. Recent attempts to understand to what extent the academic community is aware of the limited reproducibility and can predict replication outcomes using prediction markets and surveys suggest that peer beliefs may be viewed as an additional reproducibility indicator.
Anna Dreber and Magnus Johannesson
The convergence between Basque and Romance is now largely unidirectional, with Basque becoming more like Romance, but shared features suggest that Basque had historically a considerable influence on the emerging Romance varieties in southern France and northern Iberia. Similar phonemic distinctions and phonetic realizations are found in adjacent Basque and Romance varieties, and sometimes beyond. The phoneme inventories of Basque and Castilian Spanish are largely identical. The Romance influence on Basque is most visible in the lexicon, as over half of the words used in everyday speech are of Latin or Romance origin. While the Basque contribution to the Romance lexicon of common nouns has been much more modest, some Basque anthroponyms have become very popular beyond the Basque Country. The integration of Latin verbs into the Basque lexicon triggered and then accelerated the switch to a tense-aspect system modeled on that of Romance. Like Spanish, the Basque varieties in Spain distinguish between two ‘be’-copulas, and two ‘have’-verbs. Certain types of relative clauses and passive constructions replicate Romance models, and a Basque mediopassive can be systematically translated into a Spanish clause with the pronoun se. The default constituent order of Basque is verb-final, but dependent clauses are often found in post-predicate position, matching the order found in Romance. While sharing many features with Romance varieties across southwestern Europe, Basque is closest to Castilian and Gascon, the two languages with which it has a long history of bilingualism and localized language shift.
Kelly M. Kadera and Dina Zinnes
The International Studies Association (ISA) was founded in 1959, but the Scientific Study of International Politics (SSIP) only became one of its sections in 1993. SSIP researchers initially believed that subject oriented committees were unparalleled. They argued that combining traditional and quantitative research on a single committee with a common subject focus would encourage cross-examination and provide a greater understanding of the problem—but this was not the case. Nowadays, the SSIP is dedicated to the pursuit of international studies using formal or empirical data. Following the methods of scientific inquiry, the section aims to support and promote replicable research in terms of the clarity of a theory and/or the testing of hypotheses. The SSIP is committed to bringing together researchers who, at all levels of analysis and with respect to the entire range of international political questions, pursue these issues using formally stated arguments and/or systematically collected and analyzed empirical data. Furthermore, the SSIP fully cooperates with any existing organization that has similar goals, both within and outside the ISA. The significance of international studies manifests through degrees and courses that engage students and researchers with the increasing number of issues and phenomena which have arisen in the current globalized world. As such, most education providers justify the need for the degrees by relating the increasing importance of the discipline with real-world situations and employment opportunities.
Karyna Pryiomka and Joshua Clegg
Like science in general, psychological research has never had a method. Rather, psychologists have deployed many methods under quite variable justifications. The history of these methods is thus a history of contestation. Psychology’s method debates are many and varied, but they mostly constellate around two interconnected concerns: psychology’s status as a science, and psychology’s proper subject matter. On the first question, the majority position has been an attempt to establish psychology as scientific, and thus committed to quantification and to objective, particularly experimental, methods. Challenging this position, many have argued that psychology cannot be a science, or at least not a natural one. Others have questioned the epistemic privilege of operationalization, quantification, experimentation, and even science itself. Connecting epistemic concerns with those of ethics and morality, some have pointed to the dehumanizing and oppressive consequences of objectification. In contrast to the debates over psychology’s status as a science, the question of its proper subject matter has produced no permanent majority position, but perennial methodological debates. Perhaps the oldest of these is the conflict over whether and how self, mind, or consciousness can be observed. This conflict produced famous disagreements like the imageless thought controversy and the behaviorist assault on “introspection.” Other recurrent debates include those over whether psychologists study wholes or aggregates, structures or functions, and states or dynamic systems.
Jonathan Grossman and Ami Pedahzur
Since 2001, unprecedented resources have been invested in research into global terrorism, resulting in a dramatic rise in the number of academic publications on the topic. Works by scholars from predominantly quantitative disciplines predominate in this literature, and the unfolding development of data science and big data research has accentuated the trend. Many researchers in global terrorism created event databases, in which every row represents a distinct terrorist attack and every column a variable (e.g., the date and location of the attack, the number of casualties, etc.). Such event data are usually extracted from news sources and undergo a process of coding—the translation of unstructured text into numerical or categorical values. Some researchers collect and code their data manually; others use an automated script, or combine the efforts of humans and software. Other researchers who use event data do not collect and process their data at all; rather, they analyze other scholars’ databases. Academics and practitioners have relied on such databases for the cross-regional study of terrorism, analyzing their data statistically in an attempt to identify trends, build theories, predict future incidents, and formulate policies. Unfortunately, event data on terrorism often suffer from substantial issues of accuracy and reproducibility. A comparison between the data on suicide terrorism in Israel and the occupied Palestinian territories in two of the most prominent databases in the field and an independent database of confirmed events reveals the magnitude of these problems. Among the most common pitfalls for event data are replication problems (the sources that the databases cite, if there are any at all, cannot be retrieved), selection bias (events that should have been included in the database are not in it), description bias (the details of events in the database are incorrect), and coding problems (for example, duplicate events). Some of these problems originate in the press sources that are used to create the databases, usually English-language newspaper articles, and others are attributable to deficient data-gathering and/or coding practices on the part of database creators and coders. In many cases, these researchers do not understand the local contexts, languages, histories, and cultures of the regions they study. Further, many coders are not trained in qualitative methods and are thus incapable of critically reading and accurately coding their unstructured sources. Overcoming these challenges will require a change of attitude: truly accurate and impactful cross-regional data on terrorism can only be achieved through collaboration across projects, disciplines, and fields of expertise. The creators of event databases are encouraged to adopt the high standards of transparency, replicability, data-sharing, and version control that are prevalent in the STEM sciences and among software developers. More than anything, they need to acknowledge that without good and rigorous qualitative work during the stage of data collection, there can be no good quantitative work during the stage of data analysis.
Boele De Raad and Boris Mlačić
The field of dispositional traits of personality is best summarized in terms of five fundamental dimensions: the Big Five personality trait factors, namely Extraversion, Agreeableness, Conscientiousness, Emotional Stability, and Intellect. The Big Five find their origin in psycho-lexical work in which the lexicon of a language is scanned for all words that can inform about personality traits. The Big Five factors have emerged most articulately in Indo-European languages in Europe and the United States, and weaker versions have appeared in non-Indo-European languages. The model is most functional and detailed in a format that integrates simple structure and circular representations. Such a format gives the Big Five system great accommodative potential, meaning that many or most of the concepts developed in approaches other than the Big Five can be located in that system, thus enhancing the communication about personality traits in the field. The Big Five model has been applied in virtually all disciplines of psychology, including clinical, social, organizational, and developmental psychology. In particular, the Big Five have been found useful in the field of learning and education where the factor Conscientiousness has been identified as a strong predictor of academic performance, but where other factors of the Big Five also have been demonstrated to play important roles, often in moderating or mediating sense. The Big Five model has faced a number of critical issues, one of which concerns the criteria of inclusion of trait-descriptive words from the lexicon. With relaxed criteria, allowing more than just dispositional trait words (e.g., trait words that are predominantly evaluative in nature), additional dimensions may emerge beyond the Big Five, mostly conveying features of morality. An important issue regards the cross-cultural applicability of trait-descriptive dimensions. With a cross-cultural emphasis, possibly no more than three factors, expressive of traits of Extraversion, Agreeableness, and Conscientiousness, make the best chance for claims of universality. For a good understanding of traits representing the remaining Big Five dimensions, and also dimensions that have sometimes been identified beyond the Big Five, it is not only important to specify their regional applicability, but also to articulate differences in research methodology.