Haskell Environment

(With IHaskell)

This notebook describes and creates the Haskell environment in Nextjournal. Check out the showcase if you want to see what the environment contains. To see how it’s built, see setup.

Showcase

Built packages:

echo "Package,Version" > /results/pkg.csv

stack exec ghc-pkg -- list | grep -v -e '/' -e "^$" -e "(no packages)" | \

  sort | sed 's/-\([0-9]\)/,\1/' >> /results/pkg.csv

Hello World

import IHaskell.Display

html "<strong>Works!</strong>"

Diagrams

:extension NoMonomorphismRestriction FlexibleContexts TypeFamilies

import Control.Monad

import Control.Monad.State

import System.Random

import Data.Random

import Data.Random.Distribution.Pareto

import Data.RVar

import Diagrams.Prelude hiding (normal)

--import Data.Colour.Palette.BrewerSet

import qualified Diagrams.Color.XKCD as XKCD

import Data.Colour (withOpacity)

randomNormal :: (Num a, Distribution Normal a) => RVar a

randomNormal = normal 0 1

randomPareto :: (Floating a, Distribution StdUniform a) => RVar a

randomPareto = pareto 1 1

randomVariable :: (Floating b, Distribution StdUniform b, Distribution Normal b)

                   => RVarT Identity b

randomVariable = (*) <$> randomPareto <*> randomNormal

sampleNormal :: State StdGen Double

sampleNormal = sampleRVar randomNormal

sampleVariable = sampleRVar randomVariable

evalState sampleNormal $ mkStdGen 1

evalState sampleVariable $ mkStdGen 1

bubble r = circle r # lw none # fcA (blue `withOpacity` 0.2)

-- RVarT Identity b

randBubbles n = do

    rs <- replicateM n $ normal 1 1

    xs <- replicateM n $ normal 0 20

    ys <- replicateM n $ normal 0 20

    let points = map p2 $ zip xs ys

        bubbles  = map bubble rs

    return $ position (zip points bubbles)

-- (MonadRandom m, ...) m a

sampleBubbles n = sampleRVar $ randBubbles n

withImgHeight 500 $ diagram $ evalState (sampleBubbles 250) $ mkStdGen 1

Setup

This build is a mix of several methods, with the addition of some simple management scripts to help with adding new packages. The stack development environment manager is geared towards projects, which makes it a little awkward for a more open-ended case like an IHaskell notebook environment. As has been stated many times in many forums, "Stack is not a package manager," so we have to pick up the slack.

The biggest problem is that, for some packages that require dependencies outside of the curated Stackage system, we must add specific versions from Hackage to the extra-deps section of stack.yaml. This is easy enough with sed, but changing stack.yaml also has the effect of resetting things so that the next stack build will make many currently built packages inaccessible (their builds are still cached, but they cannot be imported).

In order to get around this and have the effect of 'adding' packages to our current setup, we need to keep a list of currently installed packages, add to that, and then rebuild the whole environment. To accomplish this:

• The global stack.yaml is initialized to a reasonable state for IHaskell.

• The script stack-add-deps.sh is provided. Run with a list of versioned Hackage packages, and it will insert them into the extra-deps section of stack.yaml.

• The file /opt/stack/build.list has the set of packages to build.

• The file /opt/stack/install.list has packages with binaries to make globally available (i.e. they can be run directly in Bash).

• The script update.sh reads the two lists above, and makes the current state of the environment match what they say.

The workflow is thus:

1. stack-add-deps.sh <deps>

2. echo <new packages> >> /opt/stack/build.list

3. echo <new global binaries> >> /opt/stack/install.list (optional)

4. stack-update.sh

An example of steps 1, 2, and 4 can be seen in the first cell of the last Setup section.

Base Environment

This environment is used for compilation, and also as a base for the install. Kept minimal so that it can be rebuilt and an existing Stack/Haskell compilation carried over via tarfile, eliminating recompilation for small changes.

Install system dependencies.

apt-get -qq update

DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends \

  libmagic-dev libtinfo-dev libblas-dev liblapack-dev \

  netbase graphviz libgmp-dev libicu-dev alex happy

apt-get clean

rm -r /var/lib/apt/lists/*

Install Jupyter extras.

pip3 install \

  jupyter_nbextensions_configurator jupyter_contrib_nbextensions

du -hsx /

Stack

Install and build here generally follow instructions for IHaskell https://github.com/gibiansky/IHaskell#installation

Install the stack development tool. The STACK_ROOT environment variable is configured in the Runtime Settings.

echo "Stack root: $STACK_ROOT"

mkdir -p $STACK_ROOT/global-project

mkdir -p $STACK_ROOT/bin

chmod -R g+rwX $STACK_ROOT

echo "local-bin-path: $STACK_ROOT/bin" > $STACK_ROOT/config.yaml

curl -qsSL https://get.haskellstack.org/ | sh -s -- -d /opt/stack/bin

Initialize the package index.

stack update

Barebones stack.yaml.

# Stack global project /opt/stack/global-project/stack.yaml

# https://docs.haskellstack.org/en/stable/yaml_configuration/#yaml-configuration

packages: []

extra-deps:

# - extra insertion point -

# Default resolver

resolver: lts-14.17

A script to add dependencies to the extra-deps section in the global stack.yaml.

#!/bin/sh

for dep in $@; do

  sed -i "s/extra-deps:/extra-deps:\n- ${dep}/" \

    ${STACK_ROOT}/global-project/stack.yaml

done

And another script, to build all packages in build.list and then globally install those listed in install.list.

#!/bin/sh

# Build

for package in $(cat ${STACK_ROOT}/build.list); do

  stack build ${package}

done

# Install global binaries

for package in $(cat ${STACK_ROOT}/install.list); do

  stack install ${package}

done

Initial package lists for the above script.

hpack cabal-install

hpack cabal-install

Base install.

cd /opt

stack setup

chmod +x ${STACK_ROOT}/bin/stack*.sh

stack-update.sh

Link for global running.

ln -s /opt/stack/programs/x86_64-linux/ghc-8.6.5/bin/ghc \

  /opt/stack/bin/

Check.

du -hsx /

stack --version

ghc --version

Save. Locked for safety.

cd /opt

tar -zcf /results/haskell-stack-njinst.tar.gz stack

IHaskell Build

Setup Depends

We're using a Docker building strategy similar to the one followed by https://github.com/jamesdbrock/ihaskell-notebook to avoid issue https://github.com/gibiansky/IHaskell/issues/715.

Clone and install IHaskell, and copy its resolver to global stack.yaml.

cd /opt

git clone https://github.com/gibiansky/IHaskell

Insert IHaskell deps and suchlike. All the IHaskell packages are listed as extra-deps rather than packages, because we never want to build anything automatically, we always want to select exactly what we build for the IHaskell notebook environment. For example, `stack ghci` tries to load every package listed in `packages`, and we don't want that behavior. Several of these packages are unbuildable at the time of this writing.

See the Dockerfile for the list of packages which are pre-built into the Docker image.

extra-deps:

- /opt/IHaskell

- /opt/IHaskell/ipython-kernel

- /opt/IHaskell/ghc-parser

- /opt/IHaskell/ihaskell-display/ihaskell-aeson

- /opt/IHaskell/ihaskell-display/ihaskell-blaze

- /opt/IHaskell/ihaskell-display/ihaskell-charts

- /opt/IHaskell/ihaskell-display/ihaskell-diagrams

- /opt/IHaskell/ihaskell-display/ihaskell-gnuplot

- /opt/IHaskell/ihaskell-display/ihaskell-graphviz

- /opt/IHaskell/ihaskell-display/ihaskell-hatex

- /opt/IHaskell/ihaskell-display/ihaskell-juicypixels

- /opt/IHaskell/ihaskell-display/ihaskell-magic

- /opt/IHaskell/ihaskell-display/ihaskell-plot

- /opt/IHaskell/ihaskell-display/ihaskell-rlangqq

- /opt/IHaskell/ihaskell-display/ihaskell-static-canvas

# - /opt/IHaskell/ihaskell-display/ihaskell-widgets

# Build Vega

# - /opt/hvega/hvega

# - /opt/hvega/ihaskell-hvega

sed -i 's!extra-deps:!cat /tmp/ihaskell-edeps!e' \

  /opt/stack/global-project/stack.yaml

echo 'ipython-kernel

ihaskell ihaskell-aeson ihaskell-blaze ihaskell-gnuplot ihaskell-juicypixels ihaskell-graphviz' >> ${STACK_ROOT}/build.list

echo 'ihaskell' >> ${STACK_ROOT}/install.list

Final stack.yaml:

cat ${STACK_ROOT}/global-project/stack.yaml

Build

Install IHaskell and the base setup.

cd /opt

stack setup

chmod +x ${STACK_ROOT}/bin/stack*.sh

stack-update.sh

# Install IHaskell.Display libraries

# https://github.com/gibiansky/IHaskell/tree/master/ihaskell-display

# Skip install of ihaskell-widgets, they don't work.

# See https://github.com/gibiansky/IHaskell/issues/870

#   stack build --fast ihaskell-widgets

ihaskell install --debug --stack

Install Additional Packages

Install extra packages for graphics work.

echo 'rvar random-fu colour palette diagrams diagrams-cairo Chart Chart-cairo

ihaskell-charts ihaskell-diagrams' >> ${STACK_ROOT}/build.list

# These are all deps for the Cairo chain:

stack-add-deps.sh gtk2hs-buildtools-0.13.5.0 glib-0.13.7.0 \

  cairo-0.13.6.0 pango-0.13.6.0 diagrams-cairo-1.4.1 Chart-cairo-1.9.1

stack-update.sh

echo "Package,Version" > /results/pkg.csv

stack exec ghc-pkg -- list | grep -v -e '/' -e "^$" -e "(no packages)" | \

  sort | sed 's/-\([0-9]\)/,\1/' >> /results/pkg.csv

Check.

du -hsx /

Tar up the whole shebang. Locked for safety.

Last compile: GHC v8.6.5, IHaskell v0.10.0.2, Dec 19th 2019.

cd /opt

tar -zcf /results/ihaskell-njinst.tar.gz stack IHaskell

Main Environment

Untar the built Stack/Haskell/IHaskell blob.

mkdir -p /opt

cd /opt

tar -zxf NJ__REF_

Install the IHaskell kernel.

ihaskell install --debug --stack

Check.

du -hsx /

stack --version

/opt/stack/programs/x86_64-linux/ghc-8.6.5/bin/ghc --version

jupyter kernelspec list