Abstract
Using publicly available data from 299 pre-registered replications from the social sciences, we find that the language used to describe a study can predict its replicability above and beyond a large set of controls related to the paper characteristics, study design and results, author information, and replication effort. To understand why, we analyze the textual differences between replicable and nonreplicable studies. Our findings suggest that the language in replicable studies is transparent and confident, written in a detailed and complex manner, and generally exhibits markers of truthful communication, possibly demonstrating the researchers’ confidence in the study. Nonreplicable studies, however, are vaguely written and have markers of persuasion techniques such as the use of positivity and clout. Thus, our findings allude to the possibility that authors of nonreplicable studies are more likely to make an effort, through their writing, to persuade readers of their (possibly weaker) results.