Fun With MatPlotLib

{:deps
 {org.clojure/clojure {:mvn/version "1.10.1"}
  org.clojure/tools.deps.alpha
  {:git/url "https://github.com/clojure/tools.deps.alpha.git"
   :sha "f6c080bd0049211021ea59e516d1785b08302515"}
  compliment {:mvn/version "0.3.9"}
  cnuernber/libpython-clj {:mvn/version "0.14"}}}

deps.edn

apt-get update &&\
apt-get install libpython3.6-dev python3-pip &&\
pip3 install numpy pandas matplotlib

76.8s

Bash in Clojure

Setup

Initialization uses JNA to find the python shared library on the system. You can configure which library to find via a dynamic var if the default (3.6 at this time) doesn't work for you.

(require '[libpython-clj.python :as py])
(require '[tech.v2.datatype :as dtype])
(require '[tech.v2.tensor :as dtt])
(require '[clojure.java.io :as io])
(import '[java.awt.image BufferedImage])
(import '[javax.imageio ImageIO])


;;Uncomment this line to load a different version of your python shared library:
;;(alter-var-root #'libpython-clj.jna.base/*python-library* (constantly "python3.7m"))


(py/initialize!)

31.4s

Clojure

:ok

Numpy & Basic Python

Using libpython, we can interact with python very naturally with good (and getting better) REPL support.

(def np (py/import-module "numpy"))
(py/call-attr np "ones" [2 3])

0.6s

Clojure

Vector(4) [libpython_clj.python.bridge$generic_python_as_jvm$fn$reify__24453, "0x6074d326", "[[1. 1. 1.] [1. 1. 1.]]", Map]

For this plot, we will compare three different functions on the domain [0 2]:

(def x (py/call-attr np "linspace" 0 2 100))
x

0.7s

Clojure

Python objects have a python-type which is their actual type object name in python but ->kebab-case and keyworded.

(py/python-type x)

0.1s

Clojure

:ndarray

In addition, one useful thing is the get the att-type-map which tells you all the available attributes and their types. This doesn't always work because querying an attribute in python may actually call some side-effecting code that isn't setup right. When it does work it can be extremely helpful:

(py/att-type-map x)

0.6s

Clojure

The keyword variants of all functions are supported. For example the above linspace call could have been called like so:

(def nx (py/call-attr-kw np "linspace" [0 2] {"num" 100}))
nx

0.5s

Clojure

All python objects implement iterable via the object's `__iter__` attribute. So you can iterate through a numpy object.

(seq x)

0.7s

Clojure

For multiple dimension objects, the seq of the numpy object will produce a slice of the object from the outer dimenson. Again, this isn't libpython-clj doing this; it is just calling the iter method on the object.

(def ones (py/call-attr np "ones" [3 2]))
(seq ones)

0.1s

Clojure

List(3) (Vector(4), Vector(4), Vector(4))

Plotting & Graphics

Matplotlib is a very large, full featured system for plotting data so we definitely aren't going to get into it here. But we can show one easy, side-effecty way through it that allows you to get some direct control of the result.

(def magg (py/import-module "matplotlib.backends.backend_agg"))
(def plt (py/import-module "matplotlib.pyplot"))
(def fig (py/call-attr plt "figure"))
;; Set backend to be pure in-memory
(def agg-canvas (py/call-attr magg "FigureCanvasAgg" fig))

(defn plot-it
    []
    (py/call-attr-kw plt "plot" [x x] {"label" "linear"})
    (py/call-attr-kw plt "plot" [x (py/call-attr x "__pow__" 2)] {"label" "quadratic"})
    (py/call-attr-kw plt "plot" [x (py/call-attr x "__pow__" 3)] {"label" "cubic"})
    (py/call-attr plt "xlabel" "x label")
    (py/call-attr plt "ylabel" "y label")
    (py/call-attr plt "title" "Simple Plot")
    (py/call-attr plt "legend"))

(plot-it)
(py/call-attr agg-canvas "draw")

2.1s

Clojure

Getting The Raster Data

Note that we changed the backend to be the pure-software in-memory backend for the plot. This gives us an interesting option - we can get to the actual pixels via the zero-copy pathway from the plotlib to tensors.

(def np-data (py/call-attr np "array"
                           (py/call-attr agg-canvas "buffer_rgba")))
(def tens (py/as-tensor np-data))

{:datatype (dtype/get-datatype tens)
:shape (dtype/shape tens)
 :ecount (dtype/ecount tens)
:buffer (dtt/tensor->buffer tens)}

0.2s

Clojure

Map {:datatype: :uint8, :shape: Vector(3), :ecount: 1228800, :buffer: Map}

The backing store of the tensor is a native-backed nio byte buffer. Reading data from this buffer naively will return the wrong values; values like -1 instead of 255. The datatype library, however, understands this and converts data upon read/write after first checking the ranges.

(def backing-store (get (dtt/tensor->buffer tens) :backing-store))
backing-store

0.0s

Clojure

Vector(4) [java.nio.DirectByteBuffer, "0x595e070f", "java.nio.DirectByteBuffer[pos=0 lim=1228800 cap=1228800]", Map]

(.get backing-store 0)

0.1s

Clojure

-1

;;Using the raw get-value calls on tensors causes them to be interpreted 
;;as just linear buffers
(dtype/get-value tens 0)

0.1s

Clojure

255

Seeing & Fixing Results

Now that we have a handle to the data, we can manipulate it and move it into actual buffered image objects or whatever we want.

(def bufimage (BufferedImage. 640 480 BufferedImage/TYPE_4BYTE_ABGR))
;;Pixels is a byte array
(def pixels (-> bufimage
                  (.getRaster)
                  (.getDataBuffer)
                  (.getData)))
;;Pixels is a byte array because we allocated an image of 4BYTE_ABGR
;;An integer backing store would make this harder.
(type pixels)

0.2s

Clojure

Vector(1) [byte]

A direct copy of the uint8 tensor data to the pixel data will fail because 255 is out of bounds for the value of a byte:

(dtype/copy! tens pixels)

0.3s

Clojure

Using the verbose version of the copy method allows us to turn off the range checking just like clojure's `unchecked-byte` function. Copy always returns the thing written to.

(dtype/copy! tens 0 pixels 0 (dtype/ecount tens) {:unchecked? true})

0.9s

Clojure

Let's see what we have so far

(ImageIO/write bufimage "PNG" (io/file "results/image.png"))

0.5s

Clojure

true

That isn't quite right! The reason is that above, we request a buffer of type 'rgba' and we allocated a buffered image of type 'abgr'. We can fix this with a tensor 'select' call which can reorganize the data arbitrarily:

(-> (dtt/select tens :all :all [3 2 1 0])
    (dtype/copy! 0 pixels 0 (dtype/ecount tens) {:unchecked? true}))

(ImageIO/write bufimage "PNG" (io/file "results/corrected_image.png"))

1.0s

Clojure

true