Reusing Environments: A Deep Dive

Importing allows an environment built in one notebook to be reused in another notebook. The following cell runs in an environment built in a notebook called Color Palette Extractor and analyzes the above photograph to find the dominant hue:

from haishoku.haishoku import Haishoku
from PIL import Image

def new_image(mode, size, color):
    return Image.new(mode, size, color)

path = JMW_Turner_-_Modern_Rome_-_Campo_Vacino.jpg

# getPalette/showDominant api
palette = Haishoku.getPalette(path)
dominantImage = Haishoku.showDominant(path)

# Haishoku object
h = Haishoku.loadHaishoku(path)
dominantColor = new_image('RGB', (100, 100), h.dominant)
dominantColor.save("/results/dominateColor.jpg", "JPEG")

1.5s

Python

1. What Makes "Import Environment..." Unique?

Nextjournal's Import environment... feature is a more powerful tool than you might realize at first glance.

The code cell provides a link back to the exact version of the notebook which created the imported environment. This is known as an immutable reference.
If the original creator of the environment updates the notebook, it will not affect the version you have referenced.

An imported environment will never lead to a broken reference, unlike a hyperlinked document on the World Wide Web. Unlike a typical file import, the context which created the environment will always be available.

Since Nextjournal automatically saves the exact version of all dependencies, transcluded environments offer the promise of equivalency - an essential step towards rigorous reproducibility and a way to engender collaboration through ease and confidence.

2. Concrete Usecase

The Hues of Turner is a modest study in the wide-ranging field of art, culture, and the use of color. To help reveal larger correlations, the author, Sandra, collaborates with a machine learning expert, Abduba.

Abduba uses Nextjournal's Tensorflow template and transcludes Sandra's environment. He can immediately get to work analyzing the color space for tens of thousands of images in a new article.

Remember that an import is not a simple reference, it is an entirely new copy that points back to its origin. This subtle detail means Sandra can refine the tooling without worrying about breaking Abduba's work and vice versa.

The top track depicts Sandra's timeline with Abduba just below. Sandra publishes her environment as color-extractor and Abduba imports it as a basis for his article, abduba/ML.

Sandra updates color-extractor twice: once to remove unused dependencies and once to add new dependencies. Each change creates a new Docker image with a unique SHA key. From Sandra's perspective they are all color-extractor, from Nextjournal's perspective there exists three Docker images in time. If Sandra makes an undesired change at any point, it's simple to revert to any point in the past.

Abduba can complete his task using the original dependencies, so he publishes using the original color-extractor environment (addressed as SHA 111). The diagram above illustrates two important features of Nextjournal's environments:

Changes to Sandra's environment do not affect Abduba's work. This is a core benefit of immutable references.
Changes in Abduba's article and associated data do not alter its environment. Articles, data, and environments are fundamentally decoupled.

The latter allows Nextjournal to automatically version control your environments, articles, and data separately. This means Nextjournal creates a fully history on your behalf while you focus on your work!

3. Understanding Immutable References

A transclusion is a special type of immutable reference to content in another article. Their immutability means that state cannot be changed, eliminating common problems associated with hyperlinks and modifying shared assets.

The difference between a transclusion and a conventional reference like hyperlinks is nuanced. Rather than refer to content in another article, transclusions include content from another article.

Ted Nelson, the person that coined the term, describes the unique fluidity of interconnections offered by transclusions in this clip from Warner Herzog's Lo and Behold (2016):

4. More Than a Notebook

Nextjournal promises complete reproducibility across your entire project. Read the introduction to environments to learn more about how this enables experimentation and sharing. Refer to the introduction to remixing to learn more about how the platform enables collaboration. For a broad overview what what makes Nextjournal different from other notebooks and development environments, check out Why Nextjournal. If you'd like to try it yourself, get in touch for Private Beta access.