ArticleRoyal Society open science2025
The reliability of replications: a study in computational reproductions.
Abstract read
In one paragraph
Article in Royal Society open science, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 3 papers.
0numbers the graph read from it
0cells of the map it votes in
3citing papers in PubMed
–field-weighted citation impact
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
3 citing papers in PubMed.
- Article
- The Social-Psychological Consequences of COVID-19: An Integrative Review and Research Agenda.International journal of environmental research and public health · 2026Article
- The reliability of replications: a study in computational reproductions.Royal Society open science · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
185 authors.
Nate BreznauOrganization and Program Planning, German Institute for Adult Education-Leibniz Center of Lifelong Learning, Bonn 53175, Germany.ORCID https://orcid.org/0000-0003-4983-3137
Eike Mark RinkeSchool of Politics and International Studies, University of Leeds, Leeds LS2 9JT, UK.ORCID https://orcid.org/0000-0002-5330-7634
Alexander WuttkeGeschwister Scholl Institute, LMU Munich, Munich 81541, Germany.ORCID https://orcid.org/0000-0002-9579-5357
Muna AdemDepartment of Sociology, University of Maryland, College Park 47405, USA.ORCID https://orcid.org/0000-0002-3560-9858
Jule AdriaansFaculty of Sociology, University of Bielefeld, Bielefeld 33615, Germany.ORCID https://orcid.org/0000-0001-7782-505X
Esra AkdenizSchool of Medicine, Marmara University, Istanbul 34722, Turkey.ORCID https://orcid.org/0000-0001-5022-2216
Amalia Alvarez-BenjumeaThe Institute of Public Goods and Policies (IPP), Centro de Ciencias Humanas y Sociales-Consejo Superior de Investigaciones Científicas, Madrid 28003, Spain.ORCID https://orcid.org/0000-0002-5829-2099
Henrik K AndersenInstitute of Sociology, Chemnitz University of Technology, Chemnitz 09126, Germany.ORCID https://orcid.org/0000-0001-6842-5337
Daniel AuerSocial and Political Science, Collegio Carlo Alberto, Turin 10122, Italy.ORCID https://orcid.org/0000-0003-4454-2365
Flavio AzevedoDepartment of Interdisciplinary Social Science, Utrecht University, Utrecht 3584CH, Netherlands.ORCID https://orcid.org/0000-0001-9000-8513
Oke BahnsenSchool of Social Sciences, University of Mannheim, Mannheim 68159, Germany.ORCID https://orcid.org/0000-0003-3198-2804
Ling BaiOrganization and Program Planning, German Institute for Adult Education-Leibniz Center of Lifelong Learning, Bonn 53175, Germany.
Dave BalzerInstitute of Sociology, Johannes Gutenberg University Mainz, Mainz 55128, Germany.ORCID https://orcid.org/0000-0001-8345-7169
Paul C BauerInstitute of Political Science, University of Freiburg, Freiburg im Breisgau 79085, Germany.ORCID https://orcid.org/0000-0002-8382-9724
Gerrit BauerDepartment of Sociology, LMU Munich, Munich 80801, Germany.ORCID https://orcid.org/0000-0002-3682-8323
Markus BaumannOffice of the Executive Board, GESIS-Leibniz Institute for the Social Sciences, Mannheim 68159, Germany.ORCID https://orcid.org/0000-0003-4783-868X
Sharon BauteDepartment of Politics and Public Administration, University of Konstanz, Konstanz 78457, Germany.ORCID https://orcid.org/0000-0003-2931-935X
Verena BenoitFaculty of Social Sciences, Economics, and Business Administration, University of Bamberg, Bamberg 96052, Germany.ORCID https://orcid.org/0000-0002-8596-9202
Julian BernauerMannheim Centre for European Social Research (MZES), University of Mannheim, Mannheim 68131, Germany.ORCID https://orcid.org/0000-0001-5699-5543
Carl BerningInstitute for Political Science, Johannes Gutenberg University Mainz, Mainz 55099, Germany.
Anna BertholdFaculty of Social Sciences, Economics, and Business Administration, University of Bamberg, Bamberg 96052, Germany.ORCID https://orcid.org/0000-0002-1017-5731
Felix S BethkeResearch Depart-ment IV: Intrastate Conflict, Peace Research Institute Frankfurt (PRIF), Frankfurt 60329, Germany.ORCID https://orcid.org/0000-0002-4259-6071
Thomas BiegertDepartment of Social Policy, London School of Economics and Political Science, London WC2A 2AE, UK.ORCID https://orcid.org/0000-0001-5437-2561
Katharina BlinzlerSurvey Data Curation, GESIS-Leibniz Institute for the Social Sciences, Mannheim 50667, Germany.
Johannes N BlumenbergKnowledge Exchange and Outreach, GESIS-Leibniz-Institute for the Social Sciences, Mannheim 67067, Germany.ORCID https://orcid.org/0000-0003-0943-2283
Licia BobzienFaculty of Economics and Social Sciences, University of Potsdam, Potsdam 10117, Germany.ORCID https://orcid.org/0000-0003-2274-509X
Andrea BohmanDepartment of Sociology, Umeå University, Umeå 901 87, Sweden.ORCID https://orcid.org/0000-0001-8335-9235
Thijs BolDepartment of Sociology, University of Amsterdam, Amsterdam 1012 WP, Netherlands.ORCID https://orcid.org/0000-0001-9509-8423
Amie BosticDepartment of Sociology, The University of Texas Rio Grande Valley, Edinburg TX 78539, USA.ORCID https://orcid.org/0000-0002-9809-5014
Zuzanna BrzozowskaVienna Institute of Demography, Austrian Academy of Sciences, Vienna, Austria & Austrian National Public Health Institute (Gesundheit Österreich GmbH, GÖG), Vienna, Austria.ORCID https://orcid.org/0000-0002-0235-991X
Katharina BurgdorfSchool of Social Sciences, University of Bremen, Bremen 28359, Germany.ORCID https://orcid.org/0000-0002-0928-3313
Kaspar BurgerDepartment of Education, University of Potsdam, Potsdam, 14469, Germany.ORCID https://orcid.org/0000-0001-5582-7062
Kathrin BuschIndependent Researcher.ORCID https://orcid.org/0000-0002-6951-0776
Juan-Carlos CastilloDepartment of Sociology Universidad de Chile, Millennium Nucleus on Digital Inequalities and Opportunities (NUDOS NCS2022_046), and Center for Social Conflict and Cohesion Studies (COES), University of Chile, Santiago 7800284, Chile.ORCID https://orcid.org/0000-0003-1265-7854
Nathan ChanDepartment of Political Science and International Relations, Loyola Marymount University, Los Angeles CA 90045, USA.ORCID https://orcid.org/0000-0001-7793-3157
Pablo ChristmannData and Research on Society, GESIS-Leibniz-Institute for the Social Sciences, Mannheim 68159, Germany.ORCID https://orcid.org/0000-0003-0458-9572
Roxanne ConnellySchool of Social and Political Science, University of Edinburgh, Edinburgh EH8 9LD, Scotland.ORCID https://orcid.org/0000-0002-3886-1506
Christian S CzymaraInstitute of Sociology, Goethe-University Frankfurt, Frankfurt 60323, Germany.ORCID https://orcid.org/0000-0002-9535-3559
Elena DamianEpidemiology and Public Health, Sciensano, Brussels 1050, Belgium.ORCID https://orcid.org/0000-0002-3776-6988
Eline A de RooijPolitical Science, Simon Fraser University, Burnaby V5A 1S6, Canada.ORCID https://orcid.org/0000-0002-5808-920X
Alejandro EckerInstitute of Political Science, Heidelberg University, Heidelberg 69115, Germany.ORCID https://orcid.org/0009-0009-1792-2080
Achim EdelmannMédialab Sciences Po, Paris 75007, France.ORCID https://orcid.org/0000-0001-8293-674X
Christina EderSurvey Data Curation, GESIS-Leibniz Institute for the Social Sciences, Mannheim 50667, Germany.ORCID https://orcid.org/0000-0002-7703-4108
Maureen A EgerDepartment of Sociology, Umeå University, Umeå 901 87, Sweden.ORCID https://orcid.org/0000-0001-9023-7316
Simon EllerbrockMannheim Centre for European Social Research (MZES), University of Mannheim, Mannheim 68131, Germany.ORCID https://orcid.org/0000-0002-9099-1420
Anna ForkeOrganization and Program Planning, German Institute for Adult Education-Leibniz Center of Lifelong Learning, Bonn 53175, Germany.
Andrea ForsterDepartment of Sociology, Utrecht University, Utrecht 3584 CS, Netherlands.ORCID https://orcid.org/0000-0002-5201-1452
Danilo FreireDepartment of Quantitative Theory and Methods, Emory University, Atlanta 30306, USA.ORCID https://orcid.org/0000-0002-4712-6810
Chris GaasendamDepartment of Work and Social Economy, Government of Flanders-Belgium.ORCID https://orcid.org/0000-0002-9431-5833
Konstantin GavrasSchool of Social Sciences, University of Mannheim, Mannheim 68159, Germany.ORCID https://orcid.org/0000-0002-9222-0101
Vernon GayleSchool of Social and Political Science, University of Edinburgh, Edinburgh EH8 9LD, Scotland.ORCID https://orcid.org/0000-0002-1929-5983
Theresa GesslerKulturwissenschaftliche Fakultät, Europa Universität Viadrina, Frankfurt 15230, Germany.ORCID https://orcid.org/0000-0003-2339-6266
Timo GnambsEducational Measurement, Leibniz Institute for Educational Trajectories, Bamberg 96047, Germany.ORCID https://orcid.org/0000-0002-6984-1276
Amélie GodefroidtCentre for Research on Peace and Development, KU Leuven, Leuven 3000, Belgium.ORCID https://orcid.org/0000-0002-5010-2860
Max GrömpingSchool of Government and International Relations, Griffith University, Nathan, Queensland 4111, Australia.ORCID https://orcid.org/0000-0003-1488-4436
Martin GroßDepartment of Sociology, University of Tübingen, Tübingen 72074, Germany.ORCID https://orcid.org/0000-0002-5193-9865
Stefan GruberResearch Data Center and Communication, SHARE BERLIN Institute, Berlin 10115, Germany.ORCID https://orcid.org/0000-0002-3459-421X
Tobias GummerData and Research on Society, GESIS-Leibniz Institute for the Social Sciences, Mannheim 68159, Germany.ORCID https://orcid.org/0000-0001-6469-7802
Andreas HadjarDivision Sociology, Social Policy and Social Work, University of Fribourg, Fribourg CH-1700, Switzerland.ORCID https://orcid.org/0000-0002-2641-010X
Verena HalbherrAssociation for Doctoral Studies Baden-Wuerttemberg 70174, Germany.ORCID https://orcid.org/0000-0001-7995-1738
Jan Paul HeisigResearch Group 'Health and Social Inequality', WZB Berlin Social Science Center, Berlin 10785, Germany.ORCID https://orcid.org/0000-0001-8228-1907
Sebastian HellmeierTransformations of Democracy Unit, WZB Berlin Social Science Center, Berlin 10785, Germany.ORCID https://orcid.org/0000-0002-9423-7150
Stefanie HeyneMannheim Centre for European Social Research (MZES), University of Mannheim, Mannheim 68131, Germany.ORCID https://orcid.org/0000-0002-1546-9421
Magdalena HirschResearch Unit Migration, Integration, Transnationalization, WZB Berlin Social Science Center, Berlin 10785, Germany.ORCID https://orcid.org/0000-0002-9709-9259
Mikael HjermDepartment of Sociology, Umeå University, Umeå 901 87, Sweden.ORCID https://orcid.org/0000-0003-4203-5394
Oshrat HochmanData and Research on Society, GESIS-Leibniz Institue for the Social Sciences, Mannheim 68159, Germany.ORCID https://orcid.org/0000-0002-4941-0815
Jan H HöfflerFacultad de Emprendimiento, Negocios y Economía, Universidad Espíritu Santo, ReplicationWiki, and EQ-Lab.ORCID https://orcid.org/0000-0002-8382-2071
Andreas HövermannWirtschafts- und Sozialwissenschaftliches Institut (WSI), Hans-Böckler-Foundation, Düsseldorf 40474, Germany.ORCID https://orcid.org/0000-0002-6774-6128
Sophia HungerSOCIUM-Research Center on Inequality and Social Policy, University of Bremen, Bremen 10785, Germany.ORCID https://orcid.org/0000-0002-3859-5674
Christian HunklerBerlin Institute for Integration and Migration Research (BIM), Humboldt Universität zu Berlin, Berlin 10099, Germany.ORCID https://orcid.org/0000-0002-1632-9834
Nora Huth-StöckleSchool of Human and Social Sciences, University of Wuppertal, Wuppertal 42119, Germany.ORCID https://orcid.org/0000-0002-1651-9101
Zsófia S IgnáczInstitute of Sociology, Goethe-University Frankfurt, Frankfurt 60323, Germany.ORCID https://orcid.org/0000-0002-2288-5757
Sabine IsraelIndependent Researcher.
Laura JacobsDepartment of Political Science, University of Antwerp, Antwerpen 2000, Belgium.ORCID https://orcid.org/0000-0001-5094-3531
Jannes JacobsenCluster 'Data-Methods-Monitoring', German Center for Integration and Migration Research (DeZIM).ORCID https://orcid.org/0000-0003-4358-0458
Bastian JaegerDepartment of Social Psychology, Tilburg University, Tilburg 5037AB, Netherlands.ORCID https://orcid.org/0000-0002-4398-9731
Sebastian JungkunzInstitute of Political Science and Sociology, University of Bonn, Bonn 53111, Germany.ORCID https://orcid.org/0000-0003-1040-8635
Nils JungmannSurvey Data Curation, GESIS-Leibniz Institute for the Social Sciences, Mannheim 50667, Germany.ORCID https://orcid.org/0000-0001-8849-8373
Jennifer KanjanaOrganization and Program Planning, German Institute for Adult Education-Leibniz Center of Lifelong Learning, Bonn 53175, Germany.
Mathias KauffDepartment of Psychology, Medical School Hamburg, Hamburg 20457, Germany.ORCID https://orcid.org/0000-0003-3803-3521
Salman KhanEconomics, University of Illinois, Chicago, Chicago, IL, USA.
Sayak KhatuaSchool of Public Policy, Oregon State University, Corvallis 97330, USA.ORCID https://orcid.org/0000-0003-3432-3150
Manuel KleinertInstitute of Sociology, Justus Liebig University Giessen, Giessen 35394, Germany.
Julia KlingerIndependent Researcher.ORCID https://orcid.org/0000-0001-8120-5785
Jan-Philipp KolbStatistisches Bundesamt, Statistisches Bundesamt Wiesbaden, Wiesbaden 67549, Germany.ORCID https://orcid.org/0000-0001-6982-2115
Marta KołczyńskaDepartment of Research on Social and Institutional Transformations, Institute of Political Studies of the Polish Academy of Sciences, Warsaw 00-625, Poland.ORCID https://orcid.org/0000-0003-4981-0437
John KukDepartment of Political Science, Michigan State University, East Lansing 48823, USA.ORCID https://orcid.org/0000-0003-0772-1110
Katharina KunißenInstitute of Sociology, Johannes Gutenberg University Mainz, Mainz 55128, Germany.ORCID https://orcid.org/0000-0002-8330-6392
Dafina Kurti SinatraCenter for Evaluation, Independent Researcher (Formerly Uni Cologne).ORCID https://orcid.org/0000-0001-7268-661X
Alexander LangenkampInstitute of Sociology, Goethe-University Frankfurt, Frankfurt 60323, Germany.ORCID https://orcid.org/0000-0002-3359-7179
Robin C LeeDepartment of Sociology, Princeton University, Princeton, USA.
Philipp M LerschSocio-Economic Panel, German Institute for Economic Research, Berlin 10117, Germany.ORCID https://orcid.org/0000-0003-3863-8301
David LiuOrganization and Program Planning, German Institute for Adult Education-Leibniz Center of Lifelong Learning, Bonn 53175, Germany.
Lea-Maria LöbelSocio-Economic Panel, German Institute for Economic Research, Berlin 10117, Germany.ORCID https://orcid.org/0000-0002-1541-6514
Philipp LutscherDepartment of Political Science, University of Oslo, Oslo 0851, Norway.ORCID https://orcid.org/0000-0001-6176-7297
Matthias MaderDepartment of Philosophy, Politics and Economics, Witen/Herdecke University, Witten 58488, Germany.ORCID https://orcid.org/0000-0002-4593-2392
Joan E MadiaDepartment of Primary Care and Health Sciences, University of Oxford, Oxford OX11JD, England.ORCID https://orcid.org/ 0000-0001-8398-8859
Natalia MalancuSwiss Forum for Migration and Population Studies, University of Neuchatel, Neuchâtel 1205, Switzerland.ORCID https://orcid.org/0000-0002-3576-2422
Luis MaldonadoInstituto de Sociologia, Pontifical Catholic University of Chile, Santiago 7820436, Chile.ORCID https://orcid.org/000-0002-0028-4766
Helge MarahrensMassive Data Institute, Georgetown University, Washington D.C. 20057, USA.ORCID https://orcid.org/0000-0002-1729-9104
Nicole MartinDepartment of Politics, University of Manchester, Manchester M19JS, UK.ORCID https://orcid.org/0000-0001-8480-7175
Paul MartinezDepartment of Institutional Research, Western Governors University, Millcreek 84107, USA.ORCID https://orcid.org/0000-0002-7041-4466
Jochen MayerlInstitute of Sociology, Chemnitz University of Technology, Chemnitz 09126, Germany.ORCID https://orcid.org/0000-0002-4599-9976
Oscar J MayorgaDirector of Data for Freedom, Equity Research Cooperative, 19107.ORCID https://orcid.org/0000-0002-5299-8955
Robert McDonnellOrganization and Program Planning, German Institute for Adult Education-Leibniz Center of Lifelong Learning, Bonn 53175, Germany.ORCID https://orcid.org/0000-0002-6440-2776
Patricia McManusDepartment of Sociology, Indiana University Bloomington, Bloomington, IN 47405, USA.ORCID https://orcid.org/0000-0003-0954-4517
Kyle McWagnerDepartment of Political Science, University of Wisconsin-Milwaukee, Milwaukee, WI 53211, USA.ORCID https://orcid.org/0000-0002-8144-2569
Cecil MeeusenDepartment of Sociology, Center for Sociological Research, KU Leuven, 3000.ORCID https://orcid.org/0000-0003-3071-9529
Daniel MeierrieksResearch Unit Migration, Integration, Transnationalization, WZB Berlin Social Science Center, Berlin 10785, Germany.ORCID https://orcid.org/0000-0003-2058-8385
Jonathan MellonDepartment of Politics, Westpoint Department of Systems Engineering, M19 2JS.ORCID https://orcid.org/0000-0001-6754-203X
Friedolin MerhoutDepartment of Sociology and Centre for Social Data Science, University of Copenhagen, 1353.ORCID https://orcid.org/0000-0003-3703-7651
Samuel MerkDepartment of School Development, Karlsruhe University of Education, Karlsruhe 76133, Germany.ORCID https://orcid.org/0000-0003-2594-5337
Daniel MeyerCompetence Centre for Regional Development, Federal Institute for Research on Building, Urban Affairs and Spatial Development (BBSR), 03048.ORCID https://orcid.org/0000-0002-1057-6498
Leticia MicheliDepartment of Social, Economic & Organisational Psychology, Leiden University, Leiden 2333AK, The Netherlands.ORCID https://orcid.org/0000-0003-0066-8222
Jonathan MijsDepartment of Sociology, Boston University, Boston, MA 02215, USA.ORCID https://orcid.org/0000-0002-7895-0028
Cristóbal MoyaSocio-Economic Panel, Institute for Economic Research, Berlin 10117, Germany.ORCID https://orcid.org/0000-0002-7176-4775
Marcel NeunhoefferSchool of Social Sciences, LMU Munich, Munich, Germany.ORCID https://orcid.org/0000-0002-9137-5785
Daniel NüstDepartment of Geosciences, Technische Universität Dresden, Dresden 01069, Germany.ORCID https://orcid.org/0000-0002-0024-5046
Olav NygårdDivision of Migration, Ethnicity and Society (REMESO), Linköping University, 60174.ORCID https://orcid.org/0000-0003-2272-8150
Fabian OchsenfeldAdministrative Headquarters, Max Planck Society, Munich 80539, Germany.
Gunnar OtteInstitute of Sociology, Johannes Gutenberg University Mainz, Mainz 55128, Germany.ORCID https://orcid.org/0000-0001-7025-2543
Anna PechenkinaDepartment of Political Science, Utah State University, Logan, UT 84321, USA.ORCID https://orcid.org/0000-0002-7934-9832
Mark PickupPolitical Science, Simon Fraser University, Canada.ORCID https://orcid.org/0000-0003-0539-1138
Christopher ProsserOrganization and Program Planning, German Institute for Adult Education-Leibniz Center of Lifelong Learning, Bonn 53175, Germany.ORCID https://orcid.org/0000-0002-2992-8190
Louis RaesDepartment of Economics, Tilburg University, Tilburg 5037 AB, The Netherlands.ORCID https://orcid.org/0000-0003-2640-7493
Kevin RalstonSchool of Social and Political Science, University of Edinburgh, Edinburgh EH8 9LD, Scotland.ORCID https://orcid.org/0000-0003-4344-7120
Miguel RamosDepartment of Social Policy, Sociology and Criminology, University of Birmingham, Birmingham B15 2TT, UK.ORCID https://orcid.org/0000-0001-6821-3692
Frank ReichertOrganization and Program Planning, German Institute for Adult Education-Leibniz Center of Lifelong Learning, Bonn 53175, Germany.ORCID https://orcid.org/0000-0003-0692-5082
Arne RoetsDepartment of Developmental, Personality, and Social Psychology, Ghent University, Sint-Pietersnieuwstraat, B-9000, Belgium.ORCID https://orcid.org/0000-0001-5814-1189
Jonathan RogersSchool of Law, Empirical Research Group, University of California, Los Angeles, Los Angeles, CA, USA.ORCID https://orcid.org/0000-0002-0039-608X
Guido RopersIndependent Researcher.ORCID https://orcid.org/0000-0001-5069-2699
Robin SamuelDepartment of Social Sciences, University of Luxembourg, 4366, Luxembourg.ORCID https://orcid.org/0000-0002-7598-197X
Gregor SandSHARE Operations, SHARE Berlin Institute, Berlin 10115, Germany.ORCID https://orcid.org/0000-0002-4475-0757
Constanza Sanhueza PetrarcaSchool of Politics and International Relations, Australian National University, Canberra 2132, Australia.ORCID https://orcid.org/0000-0002-8884-4771
Ariela SchachterDepartment of Sociology, Washington University in St. Louis, St. Louis MO 63130, USA.ORCID https://orcid.org/0000-0002-7404-4140
Merlin SchaefferDepartment of Sociology and Centre for Social Data Science, University of Copenhagen, 1353.ORCID https://orcid.org/0000-0003-1969-8974
David SchieferdeckerInstitute for Media and Communication Studies, Freie Universität Berlin, Berlin 14195, Germany.ORCID https://orcid.org/0000-0003-2376-0929
Elmar SchlueterInstitute of Sociology, Justus Liebig University Giessen, Giessen 35394, Germany.ORCID https://orcid.org/0000-0003-3880-4111
Katja SchmidtDepartment of Social Sciences, Humboldt University Berlin, Berlin, 10117, Germany.ORCID https://orcid.org/0000-0003-3695-1054
Regine SchmidtFaculty of Social Sciences, Economics, and Business Administration, University of Bamberg, Bamberg 96052, Germany.ORCID https://orcid.org/0000-0002-8484-5646
Alexander Schmidt-CatranInstitute of Sociology, Goethe-University Frankfurt, Frankfurt 60323, Germany.ORCID https://orcid.org/0000-0002-9485-6314
Claudia SchmiedebergDepartment of Sociology, LMU Munich, Munich 80801, Germany.ORCID https://orcid.org/0000-0002-6015-0460
Jürgen SchneiderTeacher and Teaching Quality, Leibniz Institute for Research and Information in Education, Frankfurt 60323, Germany.ORCID https://orcid.org/0000-0002-3772-4198
Martijn SchoonveldeChair group European Politics and Society, University of Groningen, Groningen 9712 EK, Netherlands.ORCID https://orcid.org/0000-0003-4370-2654
Julia Schulte-CloosDepartment of Political Science, University of Marburg, Marburg 35037, Germany.ORCID https://orcid.org/0000-0001-7223-3602
Sandy SchumannDepartment of Security and Crime Science, University College London, London WC1E 6BT, UK.ORCID https://orcid.org/0000-0002-0900-5356
Reinhard SchunckSchool of Human and Social Sciences, University of Wuppertal, Wuppertal 42119, Germany.ORCID https://orcid.org/0000-0002-8185-8919
Julian SeuringDepartment Migration, Leibniz Institute for Educational Trajectories, Bamberg 96047, Germany.ORCID https://orcid.org/0000-0001-5567-596X
Henning SilberInstitute for Social Research, University of Michigan, Ann Arbor MI 48109, USA.ORCID https://orcid.org/0000-0002-3568-3257
Willem SleegersDepartment of Social Psychology, Tilburg University, Tilburg 5037AB, Netherlands.ORCID https://orcid.org/0000-0001-9058-3817
Nico SonntagInstitute of Sociology, Johannes Gutenberg University Mainz, Mainz 55128, Germany.ORCID https://orcid.org/0000-0001-9951-9117
Alexander StaudtOrganization and Program Planning, German Institute for Adult Education-Leibniz Center of Lifelong Learning, Bonn 53175, Germany.
Nadia SteiberDepartment of Sociology, University of Vienna, Vienna 1090, Austria.ORCID https://orcid.org/0000-0002-9425-8840
Nils D SteinerInstitute for Political Science, Johannes Gutenberg University Mainz, Mainz 55099, Germany.ORCID https://orcid.org/0000-0003-3433-4079
Sebastian SternbergIndependent Researcher.ORCID https://orcid.org/0000-0003-4225-6402
Dieter StiersCentre for Political Science Research, KU Leuven, Leuven 3000, Belgium.ORCID https://orcid.org/0000-0001-7242-8477
Dragana StojmenovskaDepartment of Sociology, University of Amsterdam, Amsterdam 1012 WP, Netherlands.ORCID https://orcid.org/0000-0002-9805-7229
Nora StorzExpert Council on Intergration and Migration, Berlin 10178, Germany.ORCID https://orcid.org/0000-0001-5262-4024
Erich StriessnigDepartment of Sociology, University of Vienna, Vienna 1090, Austria.ORCID https://orcid.org/0000-0001-5419-9498
Anne-Kathrin StroppeSurvey Data Curation, GESIS-Leibniz Institute for the Social Sciences, Mannheim 50667, Germany.ORCID https://orcid.org/0000-0002-1251-9235
Jordan W SuchowSchool of Business, Stevens Institute of Technology, Hoboken 07030, USA.ORCID https://orcid.org/0000-0001-9848-4872
Janna TeltemannInstitute for Social Sciences, University of Hildesheim, Hildesheim 31141, Germany.ORCID https://orcid.org/0000-0003-0329-3104
Andrey TibajevDepartment of Women's and Children's Health, Uppsala University, Uppsala SE-751 05, Sweden.ORCID https://orcid.org/0000-0001-7348-1632
Brian TungDepartment of Sociology, Washington University in St. Louis, St. Louis 63130, USA.ORCID https://orcid.org/0000-0003-2630-6115
Giacomo VagniSocial Research Institute (UCL), University College London, London WC1E 6BT, USA.ORCID https://orcid.org/0000-0002-8140-1361
Jasper Van AsscheCenter for Social and Cultural Psychology (CESCUP), Université Libre de Bruxelles, Brussels, BE-1050, Belgium.ORCID https://orcid.org/0000-0002-2570-2928
Meta van der LindenDepartment of Interdisciplinary Social Science, Utrecht University, Utrecht 3584CH, Netherlands.ORCID https://orcid.org/0000-0003-3944-3354
Jolanda van der NollDepartment of Psychology, University of Hagen, Hagen 58097, Germany.ORCID https://orcid.org/0000-0001-7929-466X
Arno Van HootegemDepartment of Sociology and Human Geography, University of Oslo, Oslo 851, Norway.ORCID https://orcid.org/0000-0002-9559-8038
Stefan VogtenhuberEducation and Employment, Institute for Advanced Studies, Vienna 1080, Austria.ORCID https://orcid.org/0000-0003-0839-4481
Bogdan VoicuResearch Institute for Quality of Life, Romanian Academy, Bucharest, 010071, Romania.ORCID https://orcid.org/0000-0002-2221-2499
Fieke WagemansBeleidsvisies, Burgervisies en Gedragingen (Policy Perspectives, Citizen Perspectives, and Behaviors), Netherlands Institute for Social Research, The Hague 2594, Netherlands.ORCID https://orcid.org/0000-0002-8537-0187
Nadja WehlResearch Cluster 'The Politics of Inequality', University of Konstanz, Konstanz 78464, Germany.ORCID https://orcid.org/0000-0002-8828-4399
Hannah WernerDepartment of Political Science, University of Zurich, Zürich 8050, Switzerland.ORCID https://orcid.org/0000-0003-4248-5037
Brenton M WiernikIndependent Researcher.ORCID https://orcid.org/0000-0001-9560-6336
Fabian WinterMechanisms of Normative Change, Max-Planck-Institute for Research on Collective Goods, Bonn 53113, Germany.ORCID https://orcid.org/0000-0002-4838-4504
Christof WolfGESIS Leibniz-Institute for the Social Science & University of Mannheim, Mannheim 68159, Germany.ORCID https://orcid.org/0000-0002-9364-9524
Cary WuDepartment of Sociology, York University, Toronto M3J 1P3, Canada.ORCID https://orcid.org/0000-0003-2652-5684
Yuki YamadaFaculty of Social Sciences, Economics, and Business Administration, University of Bamberg, Bamberg 96052, Germany.ORCID https://orcid.org/0000-0003-1431-568X
Björn ZakulaOrganization and Program Planning, German Institute for Adult Education-Leibniz Center of Lifelong Learning, Bonn 53175, Germany.ORCID https://orcid.org/0000-0002-4191-2239
Nan ZhangFaculty of Arts and Science, Kyushu University, Fukuoka 819-0395, Japan.ORCID https://orcid.org/0009-0001-6883-1359
Conrad ZillerDepartment of Political Science, University of Duisburg-Essen, Duisburg-Essen 47057, Germany.ORCID https://orcid.org/0000-0002-2282-636X
Stefan ZinsInstitute for Employment Research, Federal Employment Agency, Nuremberg 90478, Germany.ORCID https://orcid.org/0000-0002-3097-5939
Tomasz ŻółtakDepartment of Computational Social Sciences, Institute of Philosophy and Sociology of the Polish Academy of Sciences, Warsaw 00-330, Poland.ORCID https://orcid.org/0000-0003-1354-4472
Hung H V NguyenPolitical Science, University of Bremen, Bremen 28359, Germany.ORCID https://orcid.org/0000-0001-9496-6217
Funding
No grant is acknowledged in the PubMed record.
Abstract
This study investigates researcher variability in computational reproduction, an activity for which it is least expected. Eighty-five independent teams attempted numerical replication of results from an original study of policy preferences and immigration. Reproduction teams were randomly grouped into a 'transparent group' receiving original study and code or 'opaque group' receiving only a method and results description and no code. The transparent group mostly verified original results (95.7% same sign and
Indexed as
computational reproductionreliabilityreplicationssocial and behavioural sciences
Identifiers
What OpenQuestion holds
Textmetadata
LicenceCC BY
Read underepoch 390
Registered trials
None linked
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.