The Nextjournal Python Environment

Default environments for Python 3 and Python 2

Nexjournal's Python 3 environment runs version  nil↩ while Python 2 runs version  nil↩.

Learn more about environments on Nextjournal.

1.
Showcase

These packages are included in Nextjournal's Python 3 environment. 

1.1.
Package Management

Packages are installed using conda or pip version  nil↩. setuptools version  nil↩ is also included for convenience. Please refer to How to Install Python Packages for more detailed information.  

1.2.
Plotting

The default environment comes with plotly version  nil↩  and matplotlib version  nil↩. Here are some examples of how they are used in Nextjournal: 

1.2.1.
Plotly

Plot a histogram of    500 values using plotly, a plotting library for making interactive graphs online.

import plotly.graph_objs as go
import numpy as np

x0 = np.random.randn(quiet-butterfly↩)
x1 = np.random.randn(quiet-butterfly↩)+1

trace1 = go.Histogram(x=x0, opacity=0.75)
trace2 = go.Histogram(x=x1, opacity=0.75)

layout = go.Layout(barmode='overlay')
go.Figure(data=[trace1, trace2], layout=layout)

1.7s

Language:Python

1.2.2.
Matplotlib

Plot a  5 hertz sine wave using matplotlib, a Python plotting library.

import matplotlib.pyplot as plt

# Data for plotting
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin((sine-cycles↩ * 2)* np.pi * t)

# Note that using plt.subplots below is equivalent to using
# fig = plt.figure() and then ax = fig.add_subplot(111)
fig, ax = plt.subplots()
ax.plot(t, s)

ax.set(xlabel='time (s)', ylabel='voltage (mV)', title='Sine Wave')
ax.grid()
fig

1.2s

Language:Python

1.3.
Data Structures

Nextjournal's default Python environment contains several packages for data manipulation and parsing. 

The SciPy ecosystem is available, including scipy version  nil↩, numpy version  nil↩, and  pandas version  nil↩. 
simplejson version  nil↩  makes it easy to encode/decode JSON data structures.
six version  nil↩ is included to help smooth differences between Python 2 and 3.

1.3.1.
Numpy

Numpy's main object is a N-dimensional array useful for linear algebra, Fourier transforms, and random number capabilities. Here it is used to create a Mandelbrot set which is ultimately plotted using matplotlib.

def mandelbrot( h,w, maxit=10):
    y,x = np.ogrid[ -1.4:1.4:h*1j, -2:0.8:w*1j ]
    c = x+y*1j
    z = c
    divtime = maxit + np.zeros(z.shape, dtype=int)

    for i in range(maxit):
        z  = z**2 + c
        diverge = z * np.conj(z) > 2**2       # who is diverging
        div_now = diverge & (divtime==maxit)  # who is diverging now
        divtime[div_now] = i + 100            # note when
        z[diverge] = 2                        # avoid diverging too much

    return divtime

fig = plt.subplots(1,figsize=(20,20))
plt.imshow(mandelbrot(1000,1000)) 
plt.axis('off')
plt.savefig("/results/temp.png")

1.9s

Language:Python

1.3.2.
Pandas

Pandas makes data analysis easier with Python. For example, a single instantiation of pandas' Series class can include all label and data information.  996 random values are generated by numpy and the final graph is plotted with matplotlib.

import pandas as pd                                                                           

ts = pd.Series(np.random.randn(pandas↩), index=pd.date_range('1/1/2000', periods=pandas↩))

ts = ts.cumsum()

fig, ax = plt.subplots()
ax = ts.plot()

fig

1.2s

Language:Python

1.3.3.
Simplejson

Import and export JSON on Nextjournal using simplejson. In the example below, a Python data structure input results in JSON output. The change from None to null is a clear indicator.

import simplejson as json
json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])

0.6s

Language:Python

1.3.4.
Six

Six makes it easy to write Python code that is compatible with both Python 2 and Python 3. 

For example, Python 2's urllib, urllib2, and urlparse modules have been combined in the urllib package in Python 3. The six.moves.urllib package is a version-independent location for this functionality.

Python 2:

from six.moves.urllib.request import urlopen

url = urlopen("http://nextjournal.com")
print url.read()

0.9s

Language:Python

Python 3:

from six.moves.urllib.request import urlopen

url = urlopen("http://nextjournal.com")
print(url.read())

1.3s

Language:Python

2.
Setup

2.1.
Build a Minimal Python 3 Environment

Download and install conda.

CONDA_VER="4.5.4"

curl -sSL -o ~/anaconda.sh \
    https://repo.continuum.io/miniconda/Miniconda3-${CONDA_VER}-Linux-x86_64.sh
/bin/bash ~/anaconda.sh -b -p /opt/conda
rm ~/anaconda.sh

15.7s

Language:Bash

Add conda's library directory so ldconfig will pick it up, set conda config, and ensure pip is reasonably updated.  We also pin Python to the installed minor version, allowing only patch-version up/downgrades.

echo "/opt/conda/lib" >> /etc/ld.so.conf.d/conda.conf

conda config --set always_yes True

printf "[global]\ndisable-pip-version-check = True\n" > /etc/pip.conf

echo 'pip >=18.1' > /opt/conda/conda-meta/pinned # prevent pip downgrade

# upgrade Python within minor version
PYTHON_MINOR=`python --version 2>&1 | sed 's/Python //;s/.[0-9] ::.*//;'`
echo "python =$PYTHON_MINOR" >> /opt/conda/conda-meta/pinned

conda update python
conda clean -tipsy

ldconfig

python -V
pip -V

39.0s

Language:Bash

2.2.
Build the Default Python 3 Environment

2.2.1.
Install

We'll install gcc and other build tools, since some pip installs require compilation.

apt-get -qq update
apt-get install \
  build-essential gfortran cmake automake libtool libltdl-dev pkg-config
apt-get clean
rm -r /var/lib/apt/lists/* # Clear package list so it isn't stale

37.5s

Language:Bash

This default image has support for pandas, scipy, matplotlib, and plotly.  We'll also install some basic utilities, as well as setuptools to make any additional installs less difficult.  We're installing Jedi to have code completions for Python.

Some version interactions in conda centered on openssl currently make it better to install matplotlib 2.2 here, lest we be forced to downgrade to Python 3.6.6.

conda install \
  setuptools six simplejson \
  plotly 'matplotlib>=2,<3.0' \
  numpy scipy pandas cython jedi

conda clean -tipsy

ldconfig

python -V
pip -V

212.4s

Language:Bash

Now we can upgrade matplotlib with pip.

pip install --upgrade matplotlib pyqt5

11.4s

Language:Bash

2.2.2.
Test

python --version

0.7s

Language:Bash

import sys; sys.version

0.9s

Language:Python

import pip;
pip.__version__

0.5s

Language:Python

import plotly; plotly.__version__

0.5s

Language:Python

np.__version__

0.6s

Language:Python

import matplotlib; matplotlib.__version__

0.9s

Language:Python

import setuptools; setuptools.__version__

0.6s

Language:Python

import six; six.__version__

0.9s

Language:Python

import simplejson; simplejson.__version__

0.5s

Language:Python

import pandas; pandas.__version__

0.5s

Language:Python

import scipy; scipy.__version__

0.6s

Language:Python

2.3.
Minimal Python 2

Download and install conda.

CONDA_VER="4.5.4"

curl -sSL -o ~/anaconda.sh \
    https://repo.continuum.io/miniconda/Miniconda2-${CONDA_VER}-Linux-x86_64.sh
/bin/bash ~/anaconda.sh -b -p /opt/conda
rm ~/anaconda.sh

15.2s

Language:Bash

Setup conda, ld, and pip.

59.0s

Language:Bash

2.4.
Default Python 2

2.4.1.
Install

255.4s

Language:Bash

2.4.2.
Test

python --version

0.8s

Language:Bash

import sys
import jedi
import numpy
import scipy
sys.version

0.7s

Language:Python

The Nextjournal Python Environment

Default environments for Python 3 and Python 2

1. Showcase

1.1. Package Management

1.2. Plotting

1.2.1. Plotly

1.2.2. Matplotlib

1.3. Data Structures

1.3.1. Numpy

1.3.2. Pandas

1.3.3. Simplejson

1.3.4. Six

2. Setup

2.1. Build a Minimal Python 3 Environment

2.2. Build the Default Python 3 Environment

2.2.1. Install

2.2.2. Test

2.3. Minimal Python 2

2.4. Default Python 2

2.4.1. Install

2.4.2. Test

1.
Showcase

1.1.
Package Management

1.2.
Plotting

1.2.1.
Plotly

1.2.2.
Matplotlib

1.3.
Data Structures

1.3.1.
Numpy

1.3.2.
Pandas

1.3.3.
Simplejson

1.3.4.
Six

2.
Setup

2.1.
Build a Minimal Python 3 Environment

2.2.
Build the Default Python 3 Environment

2.2.1.
Install

2.2.2.
Test

2.3.
Minimal Python 2

2.4.
Default Python 2

2.4.1.
Install

2.4.2.
Test