Installing and configuring packages

Nextjournal’s runtimes give you full control over their underlying file system. This means you can install any package or library and configure them in whatever way works best for you. For most programming languages, the preferred way of installing packages will be to use a Bash script but some, like R, might use their own runtime for it.

Bash Scripts vs. Bash Runtimes

Note: This is will change soon in favor of dedicated Install Scripts that can use any language.

Currently, the language used for code cells and their corresponding runtimes are coupled meaning a Python runtime will only use Python cells, a R runtime only R cells and so on. Bash is an exception here. Because it is often used to install packages, Nextjournal allows assigning Bash cells to language-specific runtimes. There are some rules in place that try to automate this process by guessing your intent when you add a Bash cell:

If no other runtimes exist, Nextjournal will add a standalone Bash runtime.

If other non-Bash runtimes exist, Nextjournal will take the most recent cell’s runtime and add a Bash script to that runtime, assuming that you want to install packages for it. This will be indicated by Bash in [Language] in the bottom right corner of a code cell:

pip install climt

Manually assigning cells to runtimes

You can always manually assign your Bash cells to runtimes: Simply open the Bash cell’s options menu and select Change runtime…, then select the runtime you want it to assign to:

Installing Packages

Each Nextjournal runtime comes with an environment that is based on Ubuntu 18.04 LTS. This means you can install packages from files via dpkg or fetch them from repositories with apt-get. Nextjournal’s default environments don’t have package lists stored so an apt-get update might be required first:

apt-get update
apt-get install fortune cowsay
/usr/games/fortune | /usr/games/cowsay

Prevent having to re-install packages

A Nextjournal runtime’s file system state is transient by default. This means that whenever the runtime shuts down or is reset (either by your action or because it automatically shut down after 20 minutes of idle time), all installed packages are lost and have to be re-installed again. To prevent this you can save your environment after installing packages to persist their file system state and all installed packages. We have dedicated guides explaining this in detail. You might want to read Runtimes & Environments for a high-level overview and Saving & Importing Environments for a short tutorial. Once you saved the runtime’s environment you can assign it to your runtime so your code cells have the packages available to them.

What follows here is a collection of examples on how to install packages for individual programming languages:

Installing Python Packages

Nextjournal's default Python 2 and 3 environments have a number of basic data and plotting packages installed, including numpy, matplotlib, and plotly:

pip freeze

When additional Python packages are required, you can install them in multiple ways. The easiest is to use conda, which will attempt to install all packages and dependencies in a consistent manner, including system packages and libraries. The Anaconda Cloud has a searchable database of packages and channels — by default conda will select only from the anaconda channel.

conda install descartes pysal

Additional channels can be added with the -c flag.

conda install -c conda-forge cartopy

For packages and versions unavailable via conda, or for installing packages distributed as wheel files, pip is available. For any packages that require compilation, gcc is available along with the most common build tools.

pip install quilt mapclassify

You can also use pip to install development versions from git repos.

pip install git+https://github.com/geopandas/geopandas@master

Finally, if a package has a setup.py, you can download and install with that.

git clone https://github.com/ResidentMario/geoplot
cd geoplot
git checkout 328cc9d5f8c02470c6257f2bf2fb2f3c5304530f
python setup.py install

Installing Clojure Packages

The default Clojure environment provides Clojure v1.10.0. Other versions, and any required libraries or packages, can be installed using a deps.edn file. In order to ensure loading when the runtime boots, this file must be created by inserting a Code Listing. Simply click the + button between nodes and select Code Listing, then set its language (in the bottom right corner) to Extensible Data Notation, and assing it deps.edn as name using its options menu (•••) in the upper left corner. The following example will install the clj-xchart library:

{ :deps { com.hypirion/clj-xchart {:mvn/version "0.2.0"} 
          org.clojure/clojure { :mvn/version "1.10.0-beta8" }}}
deps.edn
Extensible Data Notation

Your deps.edn can then be mounted into the Clojure runtime’s file system to be read at boot time using the runtime’s settings panel:

Booting will take an extended length of time because of the installation happening through deps.edn. It is advisable to save the environment once all of the packages are installed so you not have to run through this process whenever you reset the runtime.

System packages that are required for clj-xchart can be installed via a Bash cell:

apt-get update
apt-get install --no-install-recommends \
  fontconfig libxext6 libxrender1 libxtst6 libxi6

Once everything has been installed, we can require clj-xchart:

(require '[com.hypirion.clj-xchart :as c])

Installing Julia Packages

The Julia 1.1 and 0.7 default environments include a wide variety of file-handling and data processing packages, as well as the Plots and Makie graphical frameworks.

inst = Pkg.installed()
[println("$key v$(inst[key])") for key in keys(inst)];

Installation of additional packages is done via the the standard Julia Pkg interface: update will upgrade all packages with newer versions available, and add will install new packages. Running precompile before saving an environment will speed up package loading in the future.

pkg"add LightGraphs GraphPlot"
pkg"precompile"

Installing R Packages

The R environment includes the tidyverse, and the plotly and ggplot2 graphical packages.

inst = installed.packages()[,c("Package","Version")]
inst[order(tolower(inst[,"Package"])),]
0 items

As always, a Bash cell can install additional dependencies:

apt-get update
apt-get install libudunits2-dev
apt-get clean

Installation of additional R packages is done via the the standard R function install.packages(), or the devtools function install_github(). Build tools, including gcc, are included for package compilation.

install.packages(c("ggraph","igraph"))
devtools::install_github("sjmgarnier/viridis")

Once the packages have been installed, we can require them:

library(ggraph)
library(igraph)
library(tidyverse)
library(viridis)

Configuring Packages

Environment Variables

Besides using a Bash script for configuration or a deps.edn code listing mounted into Clojure runtimes (see Installing Clojure Packages), any runtime also has full control over its environment variables. To create or override env variables, simply open a runtime’s settings panel and expand the Environment Variables & Secrets section: