Show your work: attested code, data, proofs, and workflows on the Valency Hub
Products supporting a publication are recorded, timestamped, and versioned to facilitate replicability, reproducibility, and trust.
The open source software community has long valued the ability to inspect, copy, reuse, and remix software. Do the tests pass? Does it work on my data? Can I safely build that code into my dependency tree? Fork the code repository and you’re off to the races.
In research, we often have deeper (but somewhat similar) questions about a published work we encounter: are the methods sound? Do the claims hold up? Can I comfortably build on top of these results? Replicability, reproducibility, generalizability, and correctness1 are all different facets of the fundamental trust question. But unlike with open source code, where at least the source is there to inspect, publications are themselves just the tip of the iceberg.
To let readers dig deeper than the paper itself, the Valency Hub now lets authors attach four kinds of supporting material:
- Code — repositories on GitHub, GitLab, Codeberg, Hugging Face, and Software Heritage.
- Data — datasets on Zenodo, Figshare, OSF, Dryad, Harvard Dataverse, and Hugging Face.
- Proofs — formal proofs in Lean 4 or Bend 2, as a project on GitHub, a single
.leanor.bendfile, or a project archive; or an entry in the Palomar Registry or a prove2.me mission. - Agentic workflows — ASTRA (the Agentic Schema for Transparent Research Analysis), as an
astra.yamlanalysis file or a project.zip.
ASTRA and Bend 2 are two early formats in the agentic science ecosystem. ASTRA is a declarative YAML format designed to record the consequential choices in an analysis (its inputs, outputs, methodological decisions, and the evidence behind them) so that an agent, a workflow runner, or a human can rerun and audit it. It comes from Lightcone Research, an open-source initiative based at the Berkeley Institute for Data Science and France’s CNRS, with philanthropic support from Eric and Wendy Schmidt. The spec itself is openly licensed and community-owned, and its path to v1 is being worked out in public RFCs (see the write-up of Lightcone’s first developer summit and the perspective paper behind the project). Bend 2, from Victor Taelin and the Higher Order Company, is a fast programming language whose type checker is also a proof checker, as in Lean: you state what a program must do as laws, and an AI agent writes the proofs that the code obeys them. Its proof kernel comes with a Lean proof of its own consistency (here is an early hands-on look from launch week).
You can see more about which ancillary data we currently support in the Valency Hub docs. To see what an attached ASTRA workflow looks like on a published work, open the Ancillary data tab of “UNIONS-3500 Weak Lensing: II. B-mode validation for cosmic shear.”
At submission time we retrieve the attested sources, compute a SHA-256 of the files or repository where there are bytes to hash, and record the link, the resolved commit, the hash, and the retrieval time alongside the version of the publication. Every linked source must be public at the time of the ancillary stamping. For those in math and theoretical computer science, if you do not want to share the proof yet but want to attest that you have a proof in hand, you can instead give us the SHA-256 of your proof and we’ll store it. That way, at a later date, you can reveal the proof and someone can check that against the timestamped value that landed next to your publication.
.bend file in the project.Attesting to ancillary sources does not, of course, guarantee the work will be replicable, reproducible, generalizable, or correct. Ancillary files, archived DOIs, and artifact badges already let authors point at supporting material. What’s new here is that each source is retrieved, hashed, and pinned to the specific version of the paper. It doesn’t make research forkable, but it pins down exactly what a reader would need to try.
If you have other places where public data and code live, if there are other formal proof languages you’d like to see, or if you’re working on a new workflow schema, please get in touch.
On what these terms mean and how they differ, see Goodman, Fanelli, and Ioannidis, “What does research reproducibility mean?” (Science Translational Medicine, 2016), and the National Academies’ Reproducibility and Replicability in Science (2019). Classic entry points into each facet include Buckheit and Donoho, “WaveLab and Reproducible Research” (1995), which argued that a computational paper only advertises the scholarship, while the code and data are the scholarship itself; Ioannidis, “Why Most Published Research Findings Are False” (2005); the Open Science Collaboration, “Estimating the reproducibility of psychological science” (2015); and Yarkoni, “The generalizability crisis” (2022). ↩︎
