Clojure Berlin: ProseMirror Transforms with GraalVM and Clojure

These notebook is from my lightning talk at the ClojureBerlin meetup on the 12th September 2019.

Usecase at Nextjournal

Embed applying ProseMirror's transforms in a Clojure/JVM app for persisting document state.

ClojureScript app

We use this deps.edn file to bring in some minimal dependencies to build a small ClojureScript app which can be used for server side rendering.

{:deps
 {org.clojure/clojure {:mvn/version "1.10.1"}
  org.clojure/data.json {:mvn/version "0.2.6"}
  org.clojure/clojurescript {:git/url
                             "https://github.com/nextjournal/clojurescript"
                             :sha
                             "da9166015f6a28b2c18fa7706e457901d02a5d81"}}
  }

deps.edn

The sha points to a ClojureScript version with some small adaptations for GraalJS.

(ns com.nextjournal.editor.doc-transforms
  (:require [prosemirror :as pm]
            [com.nextjournal.editor.doc.schema :as doc.schema]))

(def ^:export empty-doc
  (.node doc.schema/schema "doc" nil
         #js[(.node doc.schema/schema "title" nil #js[])]))

(defn ^:export from-json [doc-json]
  (let [doc (js/JSON.parse doc-json)]
    (.nodeFromJSON doc.schema/schema doc)))

(defn ^:export apply-tx [doc tx-json]
  (let [tx (js/JSON.parse tx-json)
        steps (.map (.-steps tx)
                    #(.fromJSON pm/transform.Step doc.schema/schema %))]
    (reduce (fn [^js doc ^js step]
              (let [^js result (.apply step doc)]
                (.-doc result)))
            doc
            steps)))

app.cljs

The ClojureScript artifact:

doc_transforms.js

Running JS code from Clojure

java -version

0.9s

Bash in Clojure

Create a Graal polyglot execution context

(import '(org.graalvm.polyglot Context Source))

(def context-builder
  (doto (Context/newBuilder (into-array String ["js"]))
     (.option "js.timer-resolution" "1")
     (.option "js.java-package-globals" "false")
     (.out System/out)
     (.err System/err)
     (.allowAllAccess true)
     (.allowNativeAccess true)))

(def context (.build context-builder))

1.2s

Clojure

user/context

We use the Context/newBuilder polyglot API to create a Graal execution context and specifically only allow JavaScript execution. The js.java-package-globals is needed to prevent namespace collisions between Java packages and some ClojureScript namespaces in our codebase.

(defn execute-fn [context fn & args]
  (let [fn-ref (.eval context "js" fn)
        args (into-array Object args)]
    (assert (.canExecute fn-ref) (str "cannot execute " fn))
    (.execute fn-ref args)))

0.1s

Clojure

user/execute-fn

The execute-fn helper makes it convenient to call our Javascript functions and provide them params directly from Clojure.

Before we can actually call our Javascript functions, we first need to load our ClojureScript app into the context we created before.

(def app-js (clojure.java.io/file doc_transforms.js
))
(def app-source (.build (Source/newBuilder "js" app-js)))
(.eval context app-source)

5.2s

Clojure

Vector(4) [org.graalvm.polyglot.Value, "0x3a347e91", "undefined", Map]

We utilize Graal's Source class to load the JavaScript artifact and evaluate it in the execution context to make the our functions implemented in ClojureScript available.

(require '[clojure.data.json :as json])

(defn -->doc [context data]
  (if-not data
    (.eval context "js" "com.nextjournal.editor.doc_transforms.empty_doc")
    (let [json (json/write-str data)]
      (execute-fn context "com.nextjournal.editor.doc_transforms.from_json" json))))

(defn -apply-doc-tx
  [context doc tx]
  (let [tx-json (json/write-str tx)]
    (execute-fn context "com.nextjournal.editor.doc_transforms.apply_tx" doc tx-json)))

(defn -doc->data [context doc]
  (let [obj (.invokeMember doc "toJSON" (into-array []))
        json (.asString (execute-fn context "JSON.stringify" obj))]
    (json/read-str json)))

0.7s

Clojure

user/-doc->data

Applying Transforms

Some demo data:

(def doc-data
  {"type" "doc" "content" [{"type" "title" "attrs" {"placeholder" true}}]})

(def doc-tx
  {:version 0
   :steps [{"stepType" "replace"
            "from" 1
            "to" 1
            "slice" {"content" [{"type" "text" "text" "a"}]}
            "clientID" 4071024665}]
   :clientID 4071024665})

0.1s

Clojure

user/doc-tx

Convert the data into a ProseMirror document:

(def d (-->doc context doc-data))
d

0.2s

Clojure

Vector(4) [org.graalvm.polyglot.Value, "0x7cbca8ef", "{type: {name: "doc", schema: {spec: {...}, nodes: {...}, marks: {...}, nodeFromJSON: function bound() { [native code] }, markFromJSON: function bound() { [native code] }, topNodeType: {...}, cached: {...}}, spec: {content: "title{1} subtitle? authors? block*"}, groups: [], attrs: {}, defaultAttrs: {}, contentMatch: {validEnd: false, next: Array(2), wrapCache: []}, markSet: [], inlineContent: false, isBlock: true, isText: false}, attrs: {}, content: {content: [{...}], size: 2}, marks: []}", Map]

Apply the transformation to the document:

(def d' (-apply-doc-tx context d doc-tx))
d'

0.2s

Clojure

Vector(4) [org.graalvm.polyglot.Value, "0x28da326e", "{type: {name: "doc", schema: {spec: {...}, nodes: {...}, marks: {...}, nodeFromJSON: function bound() { [native code] }, markFromJSON: function bound() { [native code] }, topNodeType: {...}, cached: {...}}, spec: {content: "title{1} subtitle? authors? block*"}, groups: [], attrs: {}, defaultAttrs: {}, contentMatch: {validEnd: false, next: Array(2), wrapCache: []}, markSet: [], inlineContent: false, isBlock: true, isText: false}, attrs: {}, content: {content: [{...}], size: 3}, marks: []}", Map]

Convert the transformed document back into a Clojure map, e.g. for persisting:

(-doc->data context d')

0.1s

Clojure

Map {"type": "doc", "content": Vector(1)}

Performance

(dotimes [n 15]
  (time (dotimes [n 10000]
        (-apply-doc-tx context d doc-tx))))

28.5s

Clojure

(time (-apply-doc-tx context d doc-tx))

0.5s

Clojure

Vector(4) [org.graalvm.polyglot.Value, "0x2f1aeb9f", "{type: {name: "doc", schema: {spec: {...}, nodes: {...}, marks: {...}, nodeFromJSON: function bound() { [native code] }, markFromJSON: function bound() { [native code] }, topNodeType: {...}, cached: {...}}, spec: {content: "title{1} subtitle? authors? block*"}, groups: [], attrs: {}, defaultAttrs: {}, contentMatch: {validEnd: false, next: Array(2), wrapCache: []}, markSet: [], inlineContent: false, isBlock: true, isText: false}, attrs: {}, content: {content: [{...}], size: 3}, marks: []}", Map]

Upstream Issues

We ran into a number of issues and limitations, which we reported and were either resolved or worked around. We'll continue to work with the GraalVM and ClojureScript teams to get those issues resolved.

GraalJS regex engine bug affecting lookarounds: GraalJS issue #162 (the fix shipped already in Graal 19.1.0)
GraalJS namespace collisions of Java package globals with ClojureScript namespaces: GraalJS issue #164 and ClojureScript CLJS-3087
Optimized :graaljs builds from ClojureScript are missing the bootstrap code: ClojureScript CLJS-3113
Make main files produced by ClojureScript for :graaljs targets work without java package globals: ClojureScript CLJS-3089