Template-Type: ReDIF-Paper 1.0
Series: Tinbergen Institute Discussion Papers
Creation-Date: 2021-11-26
Number: 21-102/IV
Authors: Albert J. Menkveld1,2, Anna Dreber3, Felix Holzmeister4, Juergen Huber4, Magnus Johannesson3, Michael Kirchler4, Sebastian Neusüss5, Michael Razen4, Utz Weitzel1,6, David Abad-Díaz7, Menachem Abudy8, Tobias Adrian9, Yacine Ait-Sahalia10, Oliver Akmansoy11,12, 
Jamie T. Alcock13, Vitali Alexeev14, Arash Aloosh15, Livia Amato16, Diego Amaya17, James J. Angel18, Alejandro T. Avetikian19, Amadeus Bach20, Edwin Baidoo21, Gaetan Bakalli22, Li Bao23, Andrea Barbon24, Oksana Bashenko25, Parampreet C. Bindra4, Geir H. Bjønnes26, Jeffrey R. Black27, Bernard S. Black28, Dimitar Bogoev29, Santiago Bohorquez Correa30, Oleg Bondarenko31, Charles S. Bos1, Ciril Bosch-Rosa32, Elie Bouri33, Christian Brownlees34, Anna Calamia35, Viet Nga Cao36, Gunther Capelle-Blancard37, Laura M. Caprera1, Massimiliano Caporin38, Allen Carion27, Tolga Caskurlu38, Bidisha Chakrabarty39, Jian Chen40, Mikhail Chernov41, William Cheung42, Ludwig B. Chincarini43, Tarun Chordia44, Sheung Chi Chow45, Benjamin Clapham 46, Jean-Edouard Colliard47, Carole Comerton-Forde48, Edward Curran49, Thong Dao50, Wale Dare51, Ryan J. Davies52, Riccardo De Blasis53, Gianluca F. De Nard54, Fany Declerck23, Oleg Deev55, Hans Degryse56, Solomon Y. Deku50, Christophe Dasgre57, Matthijs A. van Dijk58, Chukwuma Dim59, Thomas Dimpfl60, Yun Jiang Dong40, Philip A. Drummond36, Tom Dudda61, Teodor Duevski47, Adriana Dumitrescu62, Teodor Dyakov63, Anne Haubo Dyhrberg64, Michal Dzieliński65, Asli Eksi66, Izidin El Kalak67, Saskia ter Ellen68, Nicolas Eugster69,Martin D.D. Evans18, Michael Farrell70, Ester Felez-Vinas14, Gerardo Ferrara71, El Mehdi Ferrouhi72, Andrea Flori73, Jonathan T Fluharty74, Sean D. Foley49, Kingsley Y.L. Fong75, Thierry Foucault47, Tatiana Franus76, Francesco Franzoni77, Bart Frijns78, Michael Frömmel79, Servanna M. Fu80, Sascha C. Füllbrunn6, Baoqing Gan14, Ge Gao81, Thomas P. Gehrig82, Roland Gemayel83, Dirk Gerritsen84, Javier Gil-Bazo34, Dudley Gilder67, Lawrence R. Glosten86, Arseny Gorbenko36, Joachim Grammig87, Vincent Grégoire88, Ufuk Güçbilmez89, Björn Hagströmer65, Julien Hambuckers51, Erik Hapnes5, Jeffrey H. Harris90, Lawrence Harris91, Simon Hartmann92, Jean-Baptiste Hasse93, Nikolaus Hautsch82, Xue-Zhong He94, Davidson Heath95, Simon Hediger54, Terrence Hendershott96, Ann Marie Hibbert74, Erik Hjalmarsson97, Seth Hoelscher98, Peter Hoffmann99, Craig W. Holden100, Alex R. Horenstein101, Wenqian Huang102, Da Huang95, Christophe Hurlin103, Konrad Ilczuk5, Alexey Ivashchenko1, Subramanian R. Iyer104, Hossein Jahanshahloo67, Naji O. Jalkh105, Charles M. Jones106, Simon Jurkatis71, Petri Jylhä5, Andreas T. Kaeck107, Gabriel Kaiser108, Arzé Karam109, Egle Karmaziene1, Bernhard Kassner110, Markku Kaustia5, Ekaterina Kazak111, Fearghal Kearney112, Vincent van Kervel113, Saad A. Khan88, Marta K. Khomyn14, Tony Klein112, Olga Klein114, Alexander Klos115, Michael Koetter116, Aleksey Kolokolov111, Robert A. Korajczyk28, Roman Kozhan114, Jan P. Krahnen46, Paul Kuhle117, Amy Kwan64, Quentin Lajaunie118, F.Y. Eric C. Lam119, Marie Lambert51, Hugues Langlois47, Jens Lausen46, Tobias Lauter120, Markus Leippold54, Vladimir Levin108, Yijie Li121, Hui Li122, Chee Yoong Liew123, Thomas Lindner124, Oliver Linton125, Jiacheng Liu126, Anqi Liu64, Guillermo Llorente117, Matthijs Lof5, Ariel Lohr127, Francis Logstaff41, Alejandro Lopez-Lira128, Shawn Mankad129, Nicola Mano130, Alexis Marchal131, Charles Martineau132, Francesco Mazzola58, Debrah Meloso35, Michael G. Mi64, Roxana Mihet133, V|ijay Mohan134, Sophie Moinas23, David Moore135, Liangyi Mu136, Dimitry Muravyev137, Dermot Murphy31, Gabor Neszveda138, Christian Neumeier139, Ulf Nielssson140, Mahendrarajah Nimalendran128, Sven Nolte6, Lars L. Norden65, Peter W. O’Neill141, Khaled Obaid142, Bernt A. Ødegaard143, Per Östberg54, Emiliano Pagnotta144, Marcus Painter39, Stefan Palan145, Imon J Palit134, Andreas Park146, Roberto Pascual147, Paolo Pasquariello148, Lubos Pastor16, Vinay Patel14, Andrwe J. Patton146, Neil D. Pearson150,151, Loriana Pelizzon46, Michele Pelli152, Matthias Pelster153, Christophe Pérignon47,12, Cameron Pfiffer154, Richard Philip64, Tomáš Plíhal55, Puneet Prakash98, Oliver-Alexander Press140, Tina Prodromou155, Marcel Prokopczuk120, Talis Putnins14, Ya Qian5, Gaurav Raizada156, David Rakowski157, Angelo Ranaldo24, Luca Regis158, Stefan Reitz115, Thomas Renault159, Rex W. Renjie1, Roberto Reno160, Steven J. Riddiough132, Kalle Rinne108, Paul J. Rintamäki5, Ryan Riordan40, Thomas Rittmansberger161, Iñaki Rodríguez Longarela65, Dominik Roesch162, Lavinia Rognone111, Brian Roseman163, Ioanid Rosu47, Saurabh Roy164, Nicolas Rudolf165, Stephen R. Rush166, Khaladdin Rzayev167,168, Aleksandra A. Rzeźnik169, Anthony Sanford170, Harikumar Sankaran171, Asani Sarkar172, Lucio Sarno125, Olivier Scaillet173, Stefan Scharnowski20, Klaus R. Schenk-Hoppé111, Andrea Schertler145, Michael Schneider174,175, Florian Schroeder49, Norman Schürhoff25, Philipp Schuster176, Marco A. Schwarz177,178, Mark S. Seasholes127, Norman J. Seeger1, Or Shachar172, Andiry Shkilko17, Jessica Shui179, Mario Sikic54, Giorgia Simion124, Lee A. Smales180, Paul Söderlind24, Elvira Sojli75, Konstantin Sokolov27, Jantje Sönksen87, Laima Spokeviciute67, Denitsa Stefanova108, Marti G. Subtahmanyam181, Barnabas Szaszi183, Oleksandr Talavera81, Yuehua Tang128, Nick Taylor184, Wing Wah Tham75, Erik Theissen20, Julian Thimme185, Ian Tonks184, Hai Tran135, Luca Trapin186, Anders B. Trolle140, M. Andreea Vaduva187, Giorgio Valente188, Robert A. van Ness189, Aurelio Vasquez190, Thanos Verousis80, Patrick Verwijmeren58, Anders Vilhelmsson191, Grigory Vilkov59, Vladimir Vladimirov38, Sebastian Vogel58, Stefan Voigt192, Wolf Wagner58, Thomas Walther84, Patrick Weiss193, Michel van der Wel58, Ingrid M. Werner194, Joakim Westerholm64, Christian Westheide82, Hans C. Wika195, Evert Wipplinger1, Michael Wolf54, Christian C.P. Wolff108, Leonard Wolk1, Wing-Keung Wong196, Jan Wrampelmeyer1, Zhen-Xing Wu197, Shuo Xia116, Dacheng Xiu16, Ke Xu198, Caihong Xu65, Pradeep K. Yadav199, José Yagüe200, Cheng Yan80, Antti Yang58, Woongsun Yoo201, Wenjia Yu5, Yihe Yu202, Shihao Yu1, Bart Z. Yushen203, Darya Yuferova204, Marcin Zamojski97, Abalfazl Zareei65, Stefan M. Zeisberger6, Lu Zhang108, S.Sarah Zhang111, Xiaoyu Zhang1, Lu Zhao205, Zhuo Zhong48, Zeyang Zhou155, Chen Zhou58, Xingyu S. Zhu3, Marius Zoican146, Remco Zwinkels1
Affiliations:1 Vrije Universiteit Amsterdam, 2 Tinbergen Institute, 3 Stockholm School of Economics, 4 University of Innsbruck, 5 Aalto University, 6 Radboud University, 7 University of Alicante, 8 Bar-Ilan University, 9 International Monetary Fund, 10 Princeton University, 11 CNRS, 12 Cascad, 13 University of Oxford, 14 University of Technology Sydney, 15 Neoma Business School, 16 University of Chicago Booth School of Business, 17 Wilfrid Laurier University, 18 Georgetown University, 19 Pontificia Universidad Católica de Chile, 20 University of Mannheim, 21 Tennessee Technological University, 
22 Auburn University, 23 Toulouse 1 Capitole University, 24 University of St. Gallen, 25 SFI at University of Lausanne, 26 BI Norwegian Business School, 27 University of Memphis, 
28 Northwestern University, 29 EDF Energy London, 30 Universidad EAFIT, 31 University of Illinois at Chicago, 32 Technische Universität Berlin, 33 Lebanese American University, 34 Universitat Pompeu Fabra, 35 TBS Education, 36 Monash University, 37 University Paris 1 Pantheon-Sorbonne, 38 University of Padova, 39 Saint Louis University, 40 Queen's University, 41 UCLA, 42 Waseda University, 43 University of San Francisco, 44 Emory University, 45 Australian National University, 46 Goethe University Frankfurt, 47 HEC Paris, 48 University of Melbourne, 49 Macquarie University, 50 Nottingham Trent University, 51 HEC Liège - University of Liège, 52 Babson College, 53 LUM University, 54 University of Zurich, 55 Masaryk University, 
56 KU Leuven, 57 UC Louvain, 58 Erasmus University Rotterdam, 59 Frankfurt School of Finance and Management, 60 University of Hohenheim, 61 Technische Universität Dresden, 62 ESADE Business School, Univ. Ramon Llull, 63 EDHEC Business School, 64 University of Sydney, 65 Stockholm University, 66 Salisbury University, 67, Cardif University, 68 Central Bank of Norway, 96 University of Queensland, 70 University of Virginia, 71 Bank of England, 72 Ibn Tofail University, 73 Politecnico di Milano, 74 West Virginia University, 
75 University of New South Wales, 76 Bayes Business School, 77 Università della Svizzera italiana, 78 Open Universiteit, 79 Ghent University, 80 University of Essex, 81 University of Birmingham, 82 University of Vienna, 83 King’s College London, 84 Utrecht University, 85 Barcelona School of Economics, 86 Columbia University, 87 University of Tübingen, 88 HEC Montréal, 89 University of Glasgow, 90 American University, 
91 University of Southern California, 92 Vienna University of Economics and Business, 93 Aix-Marseille University, 94 Xi'an Jiaotong-Liverpool University, 95 University of Utah, 96 UC Berkeley, 97 University of Gothenburg, 98 Missouri State University, 99 European Central Bank, 100 Indiana University, 101 University of Miami, 102 Bank for International Settlements, 103 University of Orléans, 104 University of New Mexico, 105 Saint Joseph University, 106 Columbia Business School, 107 University of Sussex, 108 University of Luxembourg, 109 Durham University, 110 University of Munich (LMU), 111 University of Manchester, 112 Queen’s University Belfast, 113 Pontificial University of Chile, 114 University of Warwick, 115 Kiel University, 116 Halle Institute for Economic Research, 117 Universidad Autónoma de Madrid, 118 Paris Dauphine University, 119 Hong Kong Institute for Monetary and Financial Research, 120 Leibniz University Hannover, 121 SandP Global Ratings, 122 La Trobe University, 123 UCSI University, 124 WU Vienna University of Economics and Business, 125 University of Cambridge, 126 Purdue University, 127 Arizona State University, 128 University of Florida, 129 Cornell University, 130 SFI at USI Lugano, 131 EPFL, 132 University of Toronto, 133 SFI at HEC Lausanne, 134 RMIT University, 135 Loyola Marymount University, 136 The University of Manchester, 137 Michigan state University, 138 John von Neumann University, 139 Justus-Liebig University, 140 Copenhagen Business School, 141 Financial Conduct Authority, 142 California State University - East Bay, 143 University of Stavanger, 144 Singapore Management University, 145 University of Graz, 146 University of Toronto Mississauga, 147 University of Balearic Islands, 148 University of Michigan, 149 Duke University, 150 University of Illinois at Urbana-Champaign, 151 Canadian Derivatives Institute, 152 SFI at University of Zurich, 153 Paderborn University, 154 University of Oregon, 155 University of Wollongong, 156 IIM Ahmedabad, 157 University of Texas at Arlington, 158 University of Torino, 159 Université Paris 1 Panthéon-Sorbonne, 160 University of Verona, 161 University of Innsbruck, 162 State University of New York at Buffalo, 163 Oklahoma State University, 164 University of Massachusetts, Amherst, 165 University of Lausanne, 166 Bowling Green State University, 167 University of Edinburgh, 168 Koç University, 169 York University, 170 University of Maryland, 171 New Mexico state University, 172 Federal Reserve Bank of New York, 173 SFI at University of Geneva, 174 Deutsche Bundesbank, 175 Leibniz Institute for Financial Research, 176 University of Stuttgart, 177 Düsseldorf Institute for Competition Economics, 178 CESifo, 179 Federal Housing Finance Agency, 180 University of Western Australia, 181 NYU Stern, 182 NYU Shanghai, 183 ELTE, Eotvos Lorand University, 184 University of Bristol, 185 Karlsruhe Institute of Technology, 186 University of Bologna, 187 Universidad Carlos III de Madrid, 188 Hong Kong Monetary Authority, 189 University of Mississippi, 190 ITAM, 191 Lund University, 192 University of København, 193 Vienna Graduate School of Finance, 194 The Ohio State University, 195 Norges Bank, 196 Asia University, 197 Zhongnan University of Economics and Law, 198 University of Victoria, 199 University of Oklahoma, 200 University of Murcia, 201 Central Michigan University, 202 University at Buffalo, 203 INSEAD, 204 Norwegian School of Economics (NHH), 205 Southwestern University of Finance and Economics
Title: Non-Standard Errors
Abstract: In statistics, samples are drawn from a population in a data- generating process (DGP). Standard errors measure the uncer- tainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.
Classification-JEL: G1, C12, C18
Keywords:
File-URL: https://papers.tinbergen.nl/21102.pdf
File-Format: application/pdf
File-Size: 1.382.380 bytes
Handle: RePEc:tin:wpaper:20210102