The reliability of replications: a study in computational reproductions
This study investigates researcher variability in computational reproduction, an activity for which it is least expected. Eighty-five independent teams attempted numerical replication of results from an original study of policy preferences and immigration. Reproduction teams were randomly grouped in...
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
Mar 2025
|
| In: |
Royal Society Open Science
Year: 2025, Jahrgang: 12, Heft: 3, Pages: 1-23 |
| ISSN: | 2054-5703 |
| DOI: | 10.1098/rsos.241038 |
| Online-Zugang: | Verlag, kostenfrei, Volltext: https://doi.org/10.1098/rsos.241038 Verlag, kostenfrei, Volltext: https://royalsocietypublishing.org/doi/10.1098/rsos.241038 
|
| Verfasserangaben: | Nate Breznau, Alejandro Ecker [und viele weitere] |
MARC
| LEADER | 00000naa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1938856325 | ||
| 003 | DE-627 | ||
| 005 | 20251020080920.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 251020s2025 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1098/rsos.241038 |2 doi | |
| 035 | |a (DE-627)1938856325 | ||
| 035 | |a (DE-599)KXP1938856325 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 16 |2 sdnb | ||
| 100 | 1 | |a Breznau, Nate |e VerfasserIn |0 (DE-588)1147192065 |0 (DE-627)1008549479 |0 (DE-576)421126264 |4 aut | |
| 245 | 1 | 4 | |a The reliability of replications |b a study in computational reproductions |c Nate Breznau, Alejandro Ecker [und viele weitere] |
| 264 | 1 | |c Mar 2025 | |
| 300 | |b Diagramme | ||
| 300 | |a 23 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 20.10.2025 | ||
| 520 | |a This study investigates researcher variability in computational reproduction, an activity for which it is least expected. Eighty-five independent teams attempted numerical replication of results from an original study of policy preferences and immigration. Reproduction teams were randomly grouped into a ‘transparent group’ receiving original study and code or ‘opaque group’ receiving only a method and results description and no code. The transparent group mostly verified original results (95.7% same sign and p-value cutoff), while the opaque group had less success (89.3%). Second-decimal place exact numerical reproductions were less common (76.9 and 48.1%). Qualitative investigation of the workflows revealed many causes of error, including mistakes and procedural variations. When curating mistakes, we still find that only the transparent group was reliably successful. Our findings imply a need for transparency, but also more. Institutional checks and less subjective difficulty for researchers ‘doing reproduction’ would help, implying a need for better training. We also urge increased awareness of complexity in the research process and in ‘push button’ replications. | ||
| 650 | 4 | |a computational reproduction | |
| 650 | 4 | |a reliability | |
| 650 | 4 | |a replications | |
| 650 | 4 | |a social and behavioural sciences | |
| 700 | 1 | |a Ecker, Alejandro |e VerfasserIn |0 (DE-588)1200650905 |0 (DE-627)1683812131 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |a Royal Society (London) |t Royal Society Open Science |d London : Royal Soc. Publ., 2014 |g 12(2025), 3 vom: März, Artikel-ID 241038, Seite 1-23 |h Online-Ressource |w (DE-627)798561173 |w (DE-600)2787755-3 |w (DE-576)415647460 |x 2054-5703 |7 nnas |
| 773 | 1 | 8 | |g volume:12 |g year:2025 |g number:3 |g month:03 |g elocationid:241038 |g pages:1-23 |g extent:23 |a The reliability of replications a study in computational reproductions |
| 856 | 4 | 0 | |u https://doi.org/10.1098/rsos.241038 |x Verlag |x Resolving-System |z kostenfrei |3 Volltext |
| 856 | 4 | 0 | |u https://royalsocietypublishing.org/doi/10.1098/rsos.241038 |x Verlag |z kostenfrei |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20251020 | ||
| 993 | |a Article | ||
| 994 | |a 2025 | ||
| 998 | |g 1200650905 |a Ecker, Alejandro |m 1200650905:Ecker, Alejandro |d 700000 |d 728600 |e 700000PE1200650905 |e 728600PE1200650905 |k 0/700000/ |k 1/700000/728600/ |p 41 | ||
| 999 | |a KXP-PPN1938856325 |e 4789729761 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"note":["Gesehen am 20.10.2025"],"type":{"media":"Online-Ressource","bibl":"article-journal"},"recId":"1938856325","language":["eng"],"person":[{"given":"Nate","family":"Breznau","role":"aut","display":"Breznau, Nate","roleDisplay":"VerfasserIn"},{"role":"aut","display":"Ecker, Alejandro","roleDisplay":"VerfasserIn","given":"Alejandro","family":"Ecker"}],"title":[{"subtitle":"a study in computational reproductions","title":"The reliability of replications","title_sort":"reliability of replications"}],"physDesc":[{"extent":"23 S.","noteIll":"Diagramme"}],"relHost":[{"type":{"bibl":"periodical","media":"Online-Ressource"},"note":["Gesehen am 22.07.15"],"disp":"Royal Society (London)Royal Society Open Science","corporate":[{"display":"Royal Society","roleDisplay":"VerfasserIn","role":"aut"}],"language":["eng"],"recId":"798561173","pubHistory":["1.2014 -"],"part":{"extent":"23","text":"12(2025), 3 vom: März, Artikel-ID 241038, Seite 1-23","volume":"12","pages":"1-23","issue":"3","year":"2025"},"title":[{"title":"Royal Society Open Science","title_sort":"Royal Society Open Science"}],"physDesc":[{"extent":"Online-Ressource"}],"origin":[{"dateIssuedKey":"2014","publisher":"Royal Soc. Publ.","dateIssuedDisp":"2014-","publisherPlace":"London"}],"id":{"issn":["2054-5703"],"zdb":["2787755-3"],"eki":["798561173"]}}],"name":{"displayForm":["Nate Breznau, Alejandro Ecker [und viele weitere]"]},"origin":[{"dateIssuedKey":"2025","dateIssuedDisp":"Mar 2025"}],"id":{"doi":["10.1098/rsos.241038"],"eki":["1938856325"]}} | ||
| SRT | |a BREZNAUNATRELIABILIT2025 | ||