Environments: A Deep Dive

What Are Environments?

An environment is a filesystem snapshot of the runtime. It includes everything that is necessary to run whatever is in its associated code cells - from the operating system to the smallest dependency.

Environments make articles a fruitful basis for easy experimentation and collaboration.

Nextjournal automatically versions an article's entire dependency stack. If new dependencies are added, a new version of the environment is created. A researcher can always go back to a previous version, transclude the environment in an entirely new article, or remix their work in an collaborative effort. Nextjournal environments mean no more messy downloads or missing dependencies - everything runs immediately in the browser!

What's Happening

An environment is a filesystem snapshot of the runtime stored as a Docker image. These images are a sophisticated software abstraction hidden behind our notebook interface. You will never have to worry about how to install Docker, manage images, or prepare them for others to use. This quick tour of Nextjournal's architecture will help convey the power of the platform.

Docker Images

Most images used in Nextjournal use Ubuntu 18.04 LTS as a base. On top of that are system libraries that extend the functionality of the Linux core.

The stuff you care most about - your carefully crafted code, written descriptions, and imported datasets - all sit atop of this stack of dependencies. This is so tightly coupled that running a little code and some data on two different computers can require hours of effort to install and configure the correct software stack. Nextjournal's environments eliminate concerns around dependencies so you can focus on your work.

Versioning

While Nextjournal does make setup easier, what we're really passionate about is making changes easier.

Version Control Systems like Git have revolutionized the development of software by recording every committed change to source code. This simplifies collaboration by reducing risk and ultimately incentivizes openness; if someone commits an ill-advised change, simply rewind to a previous version of the source code.

Nextjournal offers versioning across your entire project: from your source code all the way down to the most basic dependency. This means:

  • Sharing research with a peer is no longer difficult. Provide a URL and it immediately runs in their web browser.
  • Updating a depreciated library is no longer a risky position. Install the new software, tinker with your code, and if it's not working out, rewind.
  • Collaborating is no longer painful. Others can take your fully-attributed work, hit the remix button, and experiment with different datasets and easily alter code.

Default Environments

These are the default environments available in Nextjournal. Each language runtime is its own environment, all running live code, all contained in this single article!

Default environments are no different then your own custom environments. They run by way of transclusion. To see an environment's features and how it is built, click on the transclusion link in the lower right corner of any cell.

Minimal Bash

Based on Ubuntu 18.04.

cat /etc/*release

Python

Python default environments with support for code completion through the jedi package.

Python 3

import sys
import jedi, numpy, scipy, matplotlib
sys.version
'3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34) \n[GCC 7.3.0]'

Python 2

import sys
import jedi, numpy, scipy, matplotlib
sys.version
'2.7.16 |Anaconda, Inc.| (default, Mar 14 2019, 21:00:58) \n[GCC 7.3.0]'

Tensorflow

import platform,tensorflow as tf
print("Python version: %s.\nTensorflow version: %s.\n" % 
      (platform.python_version(),tf.__version__))

TFLearn

import platform, tflearn, tensorflow
print("Python version: %s.\nTensorflow version: %s.\n" % 
      (platform.python_version(),tensorflow.__version__))

Keras

import platform, keras, tensorflow
print("Python version: %s.\nTensorflow version: %s.\nKeras version: %s." % 
      (platform.python_version(), tensorflow.__version__, keras.__version__))

Pytorch

import platform, torch
print("Python version: %s.\nPyTorch version: %s." %
      (platform.python_version(),torch.__version__))

R

R.version.string

Julia

"$VERSION"
"1.1.0"

Clojure

(clojure-version)
"1.10.1"

Jupyter

IPython

__IPYTHON__
True

IR

IRkernel::log_error("Hello, world.")

IJulia

IJulia.set_verbose(false)
false

More Than a Notebook

Nextjournal promises complete reproducibility across your entire project. Read the introduction to transclusions to learn more about how environments can be shared. Refer to the introduction to remixing to learn more about how this enables collaboration. For a broad overview what what makes Nextjournal different from other notebooks and developent enviornments, check out Why Nextjournal. If you'd like to try it yourself, get in touch for Private Beta access.