When we share knowledge today — be it technical essays or scientific papers — most of the time we only share the final result. The analysis behind this result is mostly inaccessible. Even if we’re lucky and the code is available on GitHub or similar services, running it is hard if not impossible. Software rot, dependency hell, missing data files or hardware requirements are just a few of the problems we experience.
Research is stuck in a reproducibility crisis and no, your ten simple rules for reproducibility won’t fix that.
In the face of climate change, it’s critical to address these problems in a timely manner so that researchers can:
easily share their work with others,
experiment and build on top of each others work,
and generate trust by exposing their methods in a fully-reproducible way.
Read on to learn more.
Sharing shouldn’t be hard
Nextjournal was built from the ground up to address the reproducibility crisis. Long-term reproducibility is enabled by storing the document, data and computational environment immutably in append-only storage.
Once a notebook is ready for publication, you can assign it a permanent URL (or a DOI upon request) and share it with others.
You can decide wether a published notebook is accessible by everybody or just by a selected group of people. Published notebooks are read-only but can be remixed by anyone who can see them.
Walls make collaboration difficult
Nextjournal doesn’t have the same boundaries as traditional code repositories or notebooks. Enabled by immutability, our “Remix” feature allows to quickly build off a copy of any previously published notebook, including all of its dependencies.
A notebook’s entire file system can be published as Docker image with a single click. These images can be pulled to run locally or reused in other Nextjournal notebooks.
Public or private groups can be created to work with collaborators under a shared profile or on a per-invite basis. Secrets can be shared among collaborators.
Stop wasting time on setup
Ready-to-use templates for different programming languages and popular libraries make starting out easy. You can also import your existing Jupyter, Markdown or RMarkdown files and get them to run with minimal changes.
Furthermore, automatic provisioning & shutdown of compute instances save time on setup and save costs on idling machines.
Every notebook that is run, gets its own compute instance provisioned automatically. We keep a pool of small runners around with 4GB of RAM per instance that will boot instantly but larger instances are also available on-demand. We also have full support for up to 8 NVIDIA Tesla K80, P100 or P100 Workstation GPUs per runtime with minimal setup necessary.
Don’t worry about making changes
Nextjournal tracks changes on all levels of the system. Changes to the document are automatically versioned to an immutable database (Datomic). Previous versions of the document can be restored any time.
Any uploaded data or generated result files are automatically versioned in append-only content-addressed storage and can’t be accidentally overwritten.
Changes to the file system state can be committed as Docker images. Reproducibility is ensured by referencing these images only by their immutable hashes.
There is no need to learn any version control system, like Git, or containerization software, like Docker.
Multiple dependency stacks? Multiple language runtimes?
Nextjournal is a polyglot notebook system that lets you use multiple programming language runtimes together in a single notebook. Runtimes are independent from each other but can exchanges values between them by using files. At its core, Nextjournal runs anything you can put into a Docker container.
You can sign up for a Nextjournal account here. Check out our Quickstart guide or the general help pages to see what you can do and how to best bring your content to Nextjournal. Get in touch any time if you have questions or feedback.