Haskell Environment

Showcase

Built packages:

echo "Package,Version" > /results/pkg.csv
stack exec ghc-pkg -- list | grep -v -e '/' -e "^$" -e "(no packages)" | sort | \
  sed 's/-\([0-9]\)/,\1/' >> /results/pkg.csv

0.5s

IHaskell Test (Bash in Haskell)

0 items

Hello World

import IHaskell.Display
html "<strong>Works!</strong>"

1.1s

IHaskell Test (Haskell)

Diagrams

:extension NoMonomorphismRestriction FlexibleContexts TypeFamilies
import Control.Monad
import Control.Monad.State

import System.Random
import Data.Random
import Data.Random.Distribution.Pareto
import Data.RVar

import Diagrams.Prelude hiding (normal)
--import Data.Colour.Palette.BrewerSet
import qualified Diagrams.Color.XKCD as XKCD
import Data.Colour (withOpacity)

0.6s

IHaskell Test (Haskell)

randomNormal :: (Num a, Distribution Normal a) => RVar a
randomNormal = normal 0 1

randomPareto :: (Floating a, Distribution StdUniform a) => RVar a
randomPareto = pareto 1 1

randomVariable :: (Floating b, Distribution StdUniform b, Distribution Normal b)
                   => RVarT Identity b
randomVariable = (*) <$> randomPareto <*> randomNormal

sampleNormal :: State StdGen Double
sampleNormal = sampleRVar randomNormal

sampleVariable = sampleRVar randomVariable

evalState sampleNormal $ mkStdGen 1
evalState sampleVariable $ mkStdGen 1

0.7s

IHaskell Test (Haskell)

-0.10940815185138648

bubble r = circle r # lw none # fcA (blue `withOpacity` 0.2)

-- RVarT Identity b
randBubbles n = do
    rs <- replicateM n $ normal 1 1
    xs <- replicateM n $ normal 0 20
    ys <- replicateM n $ normal 0 20
    let points = map p2 $ zip xs ys
        bubbles  = map bubble rs
    return $ position (zip points bubbles)
          
-- (MonadRandom m, ...) m a
sampleBubbles n = sampleRVar $ randBubbles n

withImgHeight 500 $ diagram $ evalState (sampleBubbles 500) $ mkStdGen 1

1.4s

IHaskell Test (Haskell)

Setup

This build is a mix of several methods, with the addition of some simple management scripts to help with adding new packages. The stack development environment manager is geared towards projects, which makes it a little awkward for a more open-ended case like an IHaskell notebook environment. As has been stated many times in many forums, "Stack is not a package manager," so we have to pick up the slack.

The biggest problem is that, for some packages that require dependencies outside of the curated Stackage system, we must add specific versions from Hackage to the extra-deps section of stack.yaml. This is easy enough with sed, but changing stack.yaml also has the effect of resetting things so that the next stack build will make many currently built packages inaccessible (their builds are still cached, but they cannot be imported).

In order to get around this and have the effect of 'adding' packages to our current setup, we need to keep a list of currently installed packages, add to that, and then rebuild the whole environment. To accomplish this:

The global stack.yaml is initialized to a reasonable state for IHaskell.
The script stack-add-deps.sh is provided. Run with a list of versioned Hackage packages, and it will insert them into the extra-deps section of stack.yaml.
The file /opt/stack/build.list has the set of packages to build.
The file /opt/stack/install.list has packages with binaries to make globally available (i.e. they can be run directly in Bash).
The script update.sh reads the two lists above, and makes the current state of the environment match what they say.

The workflow is thus:

stack-add-deps.sh <deps>
echo <new packages> >> /opt/stack/build.list
echo <new global binaries> >> /opt/stack/install.list (optional)
stack-update.sh

An example of steps 1, 2, and 4 can be seen in the first cell of the last Setup section.

Setup System, Global GHC, and Stack

Build following instructions https://github.com/gibiansky/IHaskell#installation

Install build tools and lots of system dependencies, as well as a dummy package that lets us keep our conda install as the system Python.

# dummy package to use conda as system python
wget -qO nextjournal-conda-python_3.6.8_all.deb \
  https://nextjournal.com/data/Qmcy5rjq2jui3VTvyfktPMrLASs4vWJSGAKpz3PTzXQhho

apt-get -qq update
DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends \
  build-essential gfortran cmake automake libtool libltdl-dev pkg-config \
  libmagic-dev libtinfo-dev libzmq3-dev libblas-dev liblapack-dev zlib1g-dev \
  netbase graphviz libffi-dev libgmp-dev libcairo2-dev libpango1.0-dev \
  ./nextjournal-conda-python_3.6.8_all.deb
apt-get clean
rm -r /var/lib/apt/lists/* nextjournal-conda-python*.deb

54.5s

Haskell Stack (Bash)

Install Jupyter and dependencies.

pip install --upgrade jupyter jupyter_console jupyter_client jupyter_core \
  ipykernel ipywidgets \
  jupyter_nbextensions_configurator jupyter_contrib_nbextensions

23.3s

Haskell Stack (Bash)

Install the stack development tool. The STACK_ROOT environment variable is configured in the Runtime Settings.

echo "Stack root: $STACK_ROOT"
mkdir -p $STACK_ROOT/global-project
mkdir -p $STACK_ROOT/bin
chmod -R g+rwX $STACK_ROOT

echo "local-bin-path: $STACK_ROOT/bin" > $STACK_ROOT/config.yaml

curl -qsSL https://get.haskellstack.org/ | sh -s -- -d /opt/stack/bin

2.7s

Haskell Stack (Bash)

Initialize the package index.

stack update

76.1s

Haskell Stack (Bash)

Check.

du -hsx /
stack --version

1.4s

Haskell Stack (Bash)

Install Depends and Build IHaskell

We're using a Docker building strategy similar to the one followed by https://github.com/jamesdbrock/ihaskell-notebook to avoid issue https://github.com/gibiansky/IHaskell/issues/715.

Custom stack.yaml template, mounted to /tmp in Runtime Settings:

# Stack global project /opt/stack/global-project/stack.yaml in the Docker image.
# https://docs.haskellstack.org/en/stable/yaml_configuration/#yaml-configuration

# All the IHaskell packages are listed as extra-deps rather than packages,
# because we never want to build anything automatically, we always want to
# select exactly what we build for the IHaskell notebook environment.
# For example, `stack ghci` tries to load every package listed in `packages`,
# and we don't want that behavior. Several of these packages are unbuildable
# at the time of this writing.
#
# See the Dockerfile for the list of packages which are pre-built into the
# Docker image.

packages: []
extra-deps:
- /opt/IHaskell
- /opt/IHaskell/ipython-kernel
- /opt/IHaskell/ghc-parser
- /opt/IHaskell/ihaskell-display/ihaskell-aeson
- /opt/IHaskell/ihaskell-display/ihaskell-blaze
- /opt/IHaskell/ihaskell-display/ihaskell-charts
- /opt/IHaskell/ihaskell-display/ihaskell-diagrams
- /opt/IHaskell/ihaskell-display/ihaskell-gnuplot
- /opt/IHaskell/ihaskell-display/ihaskell-graphviz
- /opt/IHaskell/ihaskell-display/ihaskell-hatex
- /opt/IHaskell/ihaskell-display/ihaskell-juicypixels
- /opt/IHaskell/ihaskell-display/ihaskell-magic
- /opt/IHaskell/ihaskell-display/ihaskell-plot
- /opt/IHaskell/ihaskell-display/ihaskell-rlangqq
- /opt/IHaskell/ihaskell-display/ihaskell-static-canvas

# - /opt/IHaskell/ihaskell-display/ihaskell-widgets

# Build Vega
# - /opt/hvega/hvega
# - /opt/hvega/ihaskell-hvega

# Resolver copied from IHaskell stack.yaml, see the Dockerfile:

stack-template.yaml

Install stack.yaml template.

mkdir -p /opt/bin
cp /tmp/stack-template.yaml $STACK_ROOT/global-project/stack.yaml

0.1s

IHaskell Deps (Bash)

Clone and install IHaskell, and copy its resolver to global stack.yaml.

cd /opt
git clone https://github.com/gibiansky/IHaskell
grep 'resolver:' IHaskell/stack.yaml >> \
  $STACK_ROOT/global-project/stack.yaml

2.3s

IHaskell Deps (Bash)

Final stack.yaml:

cat $STACK_ROOT/global-project/stack.yaml

0.5s

IHaskell Deps (Bash)

A script to add dependencies to the extra-deps section in the global stack.yaml.

#!/bin/sh
for dep in $@; do
  sed -i "s/extra-deps:/extra-deps:\n- ${dep}/" \
    ${STACK_ROOT}/global-project/stack.yaml
done

stack-add-deps.sh

And another script, to build all packages in build.list and then globally install those listed in install.list.

#!/bin/sh
# Build
for package in $(cat ${STACK_ROOT}/build.list); do
  stack build ${package}
done

# Install global binaries
for package in $(cat ${STACK_ROOT}/install.list); do
  stack install ${package}
done

stack-update.sh

Initial package lists for the above script.

hpack cabal-install ghc-parser ipython-kernel
ihaskell ihaskell-aeson ihaskell-blaze ihaskell-gnuplot ihaskell-juicypixels ihaskell-graphviz

build.list

hpack cabal-install ihaskell

install.list

Install IHaskell and the base setup.

cd /opt
stack setup

chmod +x ${STACK_ROOT}/bin/stack*.sh
stack-update.sh

# Install IHaskell.Display libraries
# https://github.com/gibiansky/IHaskell/tree/master/ihaskell-display

# Skip install of ihaskell-widgets, they don't work.
# See https://github.com/gibiansky/IHaskell/issues/870
#   stack build --fast ihaskell-widgets

1693.7s

IHaskell Deps (Bash)

Check.

du -hsx /
/opt/stack/programs/x86_64-linux/ghc-8.6.5/bin/ghc --version

1.3s

IHaskell Deps (Bash)

Install Additional Packages & IHaskell Kernel

Install extra packages for graphics work.

echo 'rvar random-fu colour palette diagrams diagrams-cairo Chart Chart-cairo
ihaskell-charts ihaskell-diagrams' >> ${STACK_ROOT}/build.list

# These are all deps for the Cairo chain:
stack-add-deps.sh gtk2hs-buildtools-0.13.5.0 glib-0.13.7.0 \
  cairo-0.13.6.0 pango-0.13.6.0 diagrams-cairo-1.4.1 Chart-cairo-1.9.1

stack-update.sh

768.7s

Haskell (Bash)

echo "Package,Version" > /results/pkg.csv
stack exec ghc-pkg -- list | grep -v -e '/' -e "^$" -e "(no packages)" | sort | \
  sed 's/-\([0-9]\)/,\1/' >> /results/pkg.csv

0.5s

Haskell (Bash)

0 items

Install the kernel.

ihaskell install --debug --stack

0.8s

Haskell (Bash)

Check.

du -hsx /
jupyter kernelspec list

1.6s

Haskell (Bash)