A Link is not Enough – Reproducibility of Data
Fulltext URI
Document type
Additional Information
relationships.isAuthorOf
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Although many works in the database community use open data in their experimental evaluation, repeating the empirical results of previous works remains a challenge. This holds true even if the source code or binaries of the tested algorithms are available. In this paper, we argue that providing access to the raw, original datasets is not enough. Real-world datasets are rarely processed without modification. Instead, the data is adapted to the needs of the experimental evaluation in the data preparation process. We showcase that the details of the data preparation process matter and subtle differences during data conversion can have a large impact on the outcome of runtime results. We introduce a data reproducibility model, identify three levels of data reproducibility, report about our own experience, and exemplify our best practices.
Description
Keywords
Citation
URI
Collections
Endorsement
Review
Supplemented By
Referenced By
Number of citations to item: 18
- Werner Svellingen, Geir Torgersen, Tone Merete Muthanna, Oddbjørn Bruland (2025): Enhancing Hydrological Monitoring: A Comprehensive AHP–Based Protocol for Instrument Selection, doi:10.2139/ssrn.5520186
- Michael Beurskens, Stefanie Scherzinger (2023): Datenbankherstellerrecht und Datenbankforschung, In: Datenbank-Spektrum 2(23), doi:10.1007/s13222-023-00446-1
- Wolfgang Mauerer, Stefanie Scherzinger (2021): Nullius in Verba: Reproducibility for Database Systems Research, Revisited, In: 2021 IEEE 37th International Conference on Data Engineering (ICDE), doi:10.1109/icde51399.2021.00270
- Xinggang Yang, Juan Wu, Xiyang Chen (2023): Application of Artificial Intelligence to the Diagnosis and Therapy of Nasopharyngeal Carcinoma, In: Journal of Clinical Medicine 9(12), doi:10.3390/jcm12093077
- Vicky Steeves, Rémi Rampin, Fernando Chirigati (2020): Reproducibility, preservation, and access to research with ReproZip and ReproServer, In: IASSIST Quarterly 1-2(44), doi:10.29173/iq969
- Shashank Shekher Tripathi, Simon Tschirner, Mathias Röper, Markus M. Becker, Volker Skwarek (2024): Moving Smart Contracts for Proving Integrity of Off-Chain Data, In: Lecture Notes in Networks and Systems, doi:10.1007/978-3-031-53963-3_19
- Kevin Wittek, Neslihan Wittek, James Lawton, Iryna Dohndorf, Alexander Weinert, Andrei Ionita (2021): A Blockchain-Based Approach to Provenance and Reproducibility in Research Workflows, In: 2021 IEEE International Conference on Blockchain and Cryptocurrency (ICBC), doi:10.1109/icbc51069.2021.9461139
- Evanthia Kaimaklioti Samota, Robert P. Davey (2019): Knowledge and attitudes among life scientists towards reproducibility within journal articles: a research survey, doi:10.1101/581033
- Akhil Pandey Akella, Sagnik Ray Choudhury, David Koop, Hamed Alhoori (2024): Navigating the Landscape of Reproducible Research: A Predictive Modeling Approach, In: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, doi:10.1145/3627673.3679831
- Evanthia Kaimaklioti Samota, Robert P. Davey (2021): Knowledge and Attitudes Among Life Scientists Toward Reproducibility Within Journal Articles: A Research Survey, In: Frontiers in Research Metrics and Analytics, doi:10.3389/frma.2021.678554
- Christine Schäler, Thomas Hütter, Martin Schäler (2023): Benchmarking the Utility of <i>w</i> -Event Differential Privacy Mechanisms - When Baselines Become Mighty Competitors, In: Proceedings of the VLDB Endowment 8(16), doi:10.14778/3594512.3594515
- Dimitri Braininger, Wolfgang Mauerer, Stefanie Scherzinger (2020): Replicability and Reproducibility of a Schema Evolution Study in Embedded Databases, In: Lecture Notes in Computer Science, doi:10.1007/978-3-030-65847-2_19
- Tim Fischer, Denis Hirn, Gökhan Kul (2024): A Reproducible Tutorial on Reproducibility in Database Systems Research, In: Proceedings of the VLDB Endowment 12(17), doi:10.14778/3685800.3685840
- Thomas Weißgerber, Mehdi Ben Amor, Christofer Fellicious, Michael Granitzer (2023): PyPads, In: Datenbank-Spektrum 1(24), doi:10.1007/s13222-023-00459-w
- Lamia Salsabil, Jian Wu, Muntabir Hasan Choudhury, William A. Ingram, Edward A. Fox, Sarah M. Rajtmajer, C. Lee Giles (2022): A Study of Computational Reproducibility using URLs Linking to Open Access Datasets and Software, In: Companion Proceedings of the Web Conference 2022, doi:10.1145/3487553.3524658
- Claudia Mazo, Claudia Aura, Arman Rahman, William M. Gallagher, Catherine Mooney (2022): Application of Artificial Intelligence Techniques to Predict Risk of Recurrence of Breast Cancer: A Systematic Review, In: Journal of Personalized Medicine 9(12), doi:10.3390/jpm12091496
