Using Bash

1.
Working with the filesystem

Each Nextjournal code cell runs in a runtime, and each runtime has an environment, which is a Docker container with its own filesystem. We're just using Bash, so we can stick with the default Python runtime. Let's look around, make directories, and create a simple config file.

pwd
ls -al
mkdir -p /banana/nut
echo '[config_section]
config_var="config_statement"' > /banana/nut/cake

By default, each new code cell of a given language will be in the same runtime, and so it will see all the files created by any cells run previously, or even prior runs of itself.

if [ -d /banana/nut ]; then
	echo "Directory exists!"
else
	echo "Directory doesn't exist!"
fi

However, we can create a new runtime using the ··· cell menu in the upper left: Change Runtime...Add New Runtime...

That last menu selects what the new runtime's environment will be—you can select any of the stock Nextjournal environments, or Transclude to search for an environment exported by another article on the site. For this new runtime, we'll transclude the lightweight Minimal Bash environment from the nextjournal/environments article.

Since this new runtime has its own environment and filesystem, our created files no longer exist.

if [ -d /banana/nut ]; then
	echo "Directory exists!"
else
	echo "Directory doesn't exist!"
fi

The underlying Linux distribution is Ubuntu — so we can install packages with apt-get; however, the stock Nextjournal environments do not have package lists stored, so an apt-get update is required first.

apt-get update
apt-get install -y fortune cowsay

/usr/games/fortune | /usr/games/cowsay

2.
File handling

We can download files from within a code cell using wget or curl. To make files available for download from your article, copy them to /results/, where they will be stored in a persistent database. Images will also be displayed directly in the article, while other files will show as a link.

echo '[config_section]
config_var="config_statement"' > /results/test.conf

wget https://upload.wikimedia.org/wikipedia/commons/b/bd/Test.svg
cp Test.svg /results/
test.conf

Files downloaded into the filesystem can be accessed normally from other cells in the same runtime. We can also upload files directly to the article via the insert menu.

We'll upload data points along a simple cubic polynomial.

cubic.csv

This will store the file in a persistent, versioned database. To access the file in a code cell, type @ to bring up the reference menu and search for the file's name, select it or hit Enter to insert the file reference.

cat cubic.csv

We can also access files that were copied to /results/. These files are stored in the same database, but to reference these we start by typing the name of the cell that created the file, then again find the file in the autocomplete list.

cat test.confsummer-brook

3.
Saving the filesystem state

Now, what if we want to save the modifications we've made to a filesystem? This is a great way to achieve reproducibility, both now and in the future: we can install a set of programs and packages, configure the filesystem just as we like, and then save everything—OS, programs, and files—together for future use. To do this in Nextjournal, we need to export the runtime's state as a new environment.

Click the gear icon to the right of the runtime's entry in the sidebar...

...to pop up the runtime configuration menu.

Check Export the environment. This will make the Review & Save button appear below the runtime. Clicking that button will yield a window for reviewing how the current filesystem state differs from the original environment, and two choices: to either Save immediately, or to reset the runtime to its initial state and then Run Clean in top-to-bottom order before saving.

Saving can take a while, and the status bar will show when the newly created environment (a Docker image) is being uploaded to the GCR (Google Container Registry). When the icon returns to green, the upload is complete.

Now that we've got an environment, we can do whatever we want with it.

  • If we go to create a new runtime in our article, the exported environment will be available for use as the new runtime's environment.
  • We (or any other user) can Transclude it into another article.
  • Back on the runtime config menu, we can even grab a link to the image on the GCR, and download it for use elsewhere.

4.
Bash Runtime vs. [Langauge] Bash Runtime

What's the difference between a Bash runtime and a [langauge] Bash runtime (i.e. Python Bash runtime)?

Each runtime contains its own file state. This is how Nextjournal guarantees reproducibility across environments. This is true across all languages: Python, Clojure, Julia, R, and Bash.

If a researcher wants to update a language (i.e. Python, Clojure, Julia, R) file system, they may do so by creating a Bash cell within that runtime.

To run a bash cell in another language:

Insert > Code Cell: Bash > ... > Change Runtime > [select other environment]

© 2018 Nextjournal GmbH