metajelo: A metadata package for journals to support external linked objects

We propose a metadata package that is intended to provide academic journals with a lightweight means of registering, at the time of publication, the existence and disposition of supplementary materials. Information about the supplementary materials is, in most cases, critical for the reproducibility and replicability of scholarly results. In many instances, these materials are curated by a third party, which may or may not follow developing standards for the identification and description of those materials. As such, the vocabulary described here complements existing initiatives that specify vocabularies to describe the supplementary materials or the repositories and archives in which they have been deposited. Where possible, it reuses elements of relevant other vocabularies, facilitating coexistence with them. Furthermore, it provides an “at publication” record of reproducibility characteristics of a particular article that has been selected for publication. The proposed metadata package documents the key characteristics that journals care about in the case of supplementary materials that are held by third parties: existence, accessibility, and permanence. It does so in a robust, time-invariant fashion at the time of publication, when the editorial decisions are made. It also allows for better documentation of less accessible (non-public data), by treating it symmetrically from the point of view of the journal, therefore increasing the transparency of what up until now has been very opaque.

[1]  Phillip Li,et al.  A Preanalysis Plan to Replicate Sixty Economics Research Papers That Worked Half of the Time , 2017 .

[2]  W. Robert Reed,et al.  What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics? , 2016 .

[3]  V. Stodden,et al.  Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals , 2013, PloS one.

[4]  Lars Vilhuber,et al.  Report by the AEA Data Editor , 2019, AEA Papers and Proceedings.

[5]  Reginald B. Adams,et al.  Investigating Variation in Replicability: A “Many Labs” Replication Project , 2022 .

[6]  Victoria Stodden,et al.  An empirical analysis of journal policy effectiveness for computational reproducibility , 2018, Proceedings of the National Academy of Sciences.

[7]  Lars Vilhuber,et al.  The Sensitivity of Economic Statistics to Coding Errors in Personal Identifiers , 2003 .

[8]  L. Vilhuber,et al.  The Reproducibility of Economics Research: A Case Study , 2023, SSRN Electronic Journal.

[9]  Iain Hrynaszkiewicz,et al.  Standardising and Harmonising Research Data Policy in Scholary Publishing , 2017, Int. J. Digit. Curation.

[10]  Daniele Fanelli,et al.  Opinion: Is science really facing a reproducibility crisis, and do we need it to? , 2018, Proceedings of the National Academy of Sciences.

[11]  Michael C. Frank,et al.  Estimating the reproducibility of psychological science , 2015, Science.

[12]  Kevin L. McKinney,et al.  The Link between Human Capital, Mass Layoffs, and Firm Deaths , 2009 .

[13]  C. Lagoze,et al.  Making Confidential Data Part of Reproducible Research , 2017 .

[14]  Jerry G. Thursby,et al.  Replication in Empirical Economics: The Journal of Money, Credit and Banking Project , 1986 .

[15]  Malcolm Hill,et al.  The Panel Study of Income Dynamics , 1991 .

[16]  Stacy Konkiel ReplicationWiki: Improving Transparency in Social Sciences Research , 2017 .

[17]  Laura Rothfritz,et al.  FAIR Data Principles , 2019, The ACS Guide to Scholarly Communication.

[18]  Jan H. Höffler Replication and Economics Journal Policies , 2017 .

[19]  Bruce D. McCullough,et al.  Econometrics and Software: Comments , 2003 .

[20]  Sarah Callaghan,et al.  Joint declaration of data citation principles , 2014 .

[21]  Division on Earth,et al.  Reproducibility and Replicability in Science , 2019 .

[22]  E. Duflo,et al.  Report of the Search Committee to Appoint a Data Editor for the AEA , 2018 .

[23]  Jesus M. de la Garza Data Availability Policy , 2017 .

[24]  Phillip Li,et al.  Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say 'Usually Not' , 2015 .

[25]  Yi Wang,et al.  Metadata Schema for the Description of Research Data Repositories: version 3.1 , 2021 .

[26]  Garret Christensen,et al.  Transparent and Reproducible Social Science Research , 2019 .

[27]  M. Wooden,et al.  The Household, Income and Labour Dynamics in Australia (HILDA) Survey , 2000 .

[28]  Teresa D. Harrison,et al.  Lessons from the JMCB Archive , 2006 .

[29]  Paolo Manghi,et al.  Scholix Metadata Schema for Exchange of Scholarly Communication Links , 2017 .

[30]  Yolanda Gil,et al.  Enhancing reproducibility for computational methods , 2016, Science.

[31]  Lars Vilhuber,et al.  Total Error and Variability Measures with Integrated Disclosure Limitation for Quarterly Workforce Indicators and LEHD Origin Destination Employment Statistics in OnThe Map , 2017 .

[32]  Peter J. McDonough,et al.  Inter-University Consortium for Political and Social Research , 1986 .

[33]  Gideon Nave,et al.  Evaluating replicability of laboratory experiments in economics , 2016, Science.