Datomic Demo 001

This is a computational article that shows how to use Datomic Free and Datalog query language using the MusicBrainz sample database.

Setup

We start by re-using an environment built in the MusicBrainz Datomic article, it already has Datomic installed and the mbrainz sample database imported.

All we have left to do is to start the transactor.

/usr/bin/nohup /datomic-free/bin/transactor /datomic-free/config/samples/free-transactor-template.properties &> /datomic-free.log & sleep 1
tail /datomic-free.log
3.1s

Example Query

Let's see if it works. We connect to the database and get the current db value, which we can then query.

(require '[datomic.api :as d])
(def uri "datomic:free://localhost:4334/mbrainz-1968-1973")
(def conn (d/connect uri))
(def db (d/db conn))
17.6s
(into [] (d/q '[:find ?id ?type ?gender
       :in $ ?name
       :where
       [?e :artist/name ?name]
       [?e :artist/gid ?id]
       [?e :artist/type ?teid]
       [?teid :db/ident ?type]
       [?e :artist/gender ?geid]
       [?geid :db/ident ?gender]]
     db
     "Janis Joplin"))
0.7s

For documentation and basics of Datalog queries with Datomic, you can head over to the offical Datomic docs.

In the following sections we examples of querying the example dataset are provided.

Queries

What are the titles, album names, and release years of John Lennon's tracks?

(into [] (d/q '[:find ?title
 	:in $ ?artist-name
 :where
 [?a :artist/name ?artist-name]
 [?t :track/artists ?a]
 [?t :track/name ?title]]
     db "John Lennon"))
0.5s

Joins

What are the titles, album names, and release years of John Lennon's tracks?

(into [] (d/q '[:find ?title ?album ?year
 :in $ ?artist-name
 :where
 [?a :artist/name   ?artist-name]
 [?t :track/artists ?a]
 [?t :track/name    ?title]
 [?m :medium/tracks ?t]
 [?r :release/media ?m]
 [?r :release/name  ?album]
 [?r :release/year  ?year]]
     db "John Lennon"))
0.7s

Expression Clauses

What are the titles, album names, and release years of the John Lennon tracks released before 1970?

(into [] (d/q '[:find ?title ?album ?year
 :in $ ?artist-name
 :where
 [?a :artist/name   ?artist-name]
 [?t :track/artists ?a]
 [?t :track/name    ?title]
 [?m :medium/tracks ?t]
 [?r :release/media ?m]
 [?r :release/name  ?album]
 [?r :release/year  ?year]
 [(< ?year 1970)]]
     db "John Lennon"))
0.3s

Rules

To reuse query logic across many queries, you can create and use rules like the following:

(def rules
  '[;; Given ?t bound to track entity-ids, binds ?r to the corresponding
    ;; set of album release entity-ids
 [(track-release ?t ?r)
  [?m :medium/tracks ?t]
  [?r :release/media ?m]]])
0.1s

Load all rules (from the appendix) for later use:

All Rules:
user/all-rules
0.1s

What are the titles, album names, and release years of John Lennon's tracks?

(into [] (d/q '[:find ?title ?album ?year
 :in $ % ?artist-name
 :where
 [?a :artist/name   ?artist-name]
 [?t :track/artists ?a]
 [?t :track/name    ?title]
 (track-release ?t ?r)
 [?r :release/name  ?album]
 [?r :release/year  ?year]]
     db rules "John Lennon"))
0.3s

Full text

What are the titles, artists, album names, and release years of all tracks having the word "always" in their titles?

(into [] (d/q '[:find ?title ?artist ?album ?year
 :in $ % ?search
 :where
 (track-search ?search ?track)
 (track-info ?track ?title ?artist ?album ?year)]
              db all-rules "always"))
8.1s

Graph Walk

Who collaborated with one of the Beatles?

(into [] (d/q '[:find ?aname ?aname2
 :in $ % [?aname ...]
 :where (collab ?aname ?aname2)]
              db all-rules ["John Lennon" "Paul McCartney" "George Harrison" "Ringo Starr"]))
0.2s

Schema

This is the schema of the example data set.

Plotting

Nextjournal lets you also plot data with Plotly. First, lets query for every John Lennon album with a release date from 1968 to 1973.

(def albums-year 
  (into [] (d/q '[:find ?album ?year
 :in $ % ?artist-name
 :where
 [?a :artist/name   ?artist-name]
 [?t :track/artists ?a]
 [?t :track/name    ?title]
 (track-release ?t ?r)
 [?r :release/name  ?album]
 [?r :release/year  ?year]]
     db all-rules "John Lennon")))
albums-year
0.1s

First, collect albums by year. This will make them easy to count and display later.

(defn add-album-to-year [m year title]
  (if (contains? m year) 
    (assoc m year (conj (year m) title)) 
    (assoc m year [title])))
(defn albums-by-year [albums-year m]
  (if-let [album (first albums-year)]
    (let [[title year] album
           sorted-albums (add-album-to-year m (keyword (str year)) title)]
	    (albums-by-year (rest albums-year) sorted-albums))
  	  m))
(albums-by-year albums-year {})
0.1s

Second, provide the plotting software with the expected :data and :layout information.

(def result (let [final-sort (albums-by-year albums-year {})]
  {:data [{:x (keys final-sort)
           :y (map #(count (second %)) (albums-by-year albums-year {}))
           :type "bar"}]
   :layout {:autosize false :width 600 :height 500
          :type "bar"
          :xaxis1 {:title "Year"} 
          :yaxis1 {:title "# of Albums"}}}))
albums
0.1s

Finally, plot the data using ClojureScript into and interactive graph!

^{:nextjournal/viewer :plotly}
;; question: What are effective & simple ways that can I access the data and layout from `result` in the cell above?
;; {:data (:data user/result) :layout (:layout user/result)}
(.plot js/Nextjournal albums:
user/result
)
0.0s

Questions

  1. How is the above cell "accessing" the result of the prior cell as a (dynamic) variable? What are the necessary steps to reproduce this behavior from scratch?

  2. Where does the ".plot" function come from?

  3. May I store variables into memory to use across Clojure and ClojureScript contexts/environments, and how?

  4. May I render a Plotly graph in Nextjournal via a hashmap stored in memory versus a hashmap literal? (all examples are hashmap literals only @ see: https://nextjournal.com/btowers/using-plotly-with-clojure )

Appendix

(def all-rules
  '[;; Given ?t bound to track entity-ids, binds ?r to the corresponding
 ;; set of album release entity-ids
 [(track-release ?t ?r)
  [?m :medium/tracks ?t]
  [?r :release/media ?m]]
 ;; Supply track entity-ids as ?t, and the other parameters will be
 ;; bound to the corresponding information about the tracks
 [(track-info ?t ?track-name ?artist-name ?album ?year)
  [?t :track/name    ?track-name]
  [?t :track/artists ?a]
  [?a :artist/name   ?artist-name]
  (track-release ?t ?r)
  [?r :release/name  ?album]
  [?r :release/year  ?year]]
 ;; Supply ?a (artist entity-ids) and and integer ?max track duration,
 ;; and ?t, ?len will be bound to track entity-ids and lengths
 ;; (respectively) of tracks shorter than the given ?max
 [(short-track ?a ?t ?len ?max)
  [?t :track/artists ?a]
  [?t :track/duration ?len]
  [(< ?len ?max)]]
 ;; Fulltext search on track.  Supply the query string ?q, and ?track
 ;; will be bound to entity-ids of tracks whose title matches the
 ;; search.
 [(track-search ?q ?track)
  [(fulltext $ :track/name ?q) [[?track ?tname]]]]
 ;; Generic transitive network walking, used by collaboration network
 ;; rule below
 ;; Supply:
 ;; ?e1 -- an entity-id
 ;; ?attr -- an attribute ident
 ;; and ?e2 will be bound to entity-ids such that ?e1 and ?e2 are both
 ;; values of the given attribute for some entity (?x)
 [(transitive-net-1 ?attr ?e1 ?e2)
  [?x ?attr ?e1]
  [?x ?attr ?e2]
  [(!= ?e1 ?e2)]]
 ;; Same as transitive-net-1, but search one more level of depth.  We
 ;; define this rule twice, once for each case, and the rule
 ;; represents the union of the two cases:
 ;; - The entities are directly related via the attribute
 ;; - The entities are related to the given depth (in this case 2) via the attribute
 [(transitive-net-2 ?attr ?e1 ?e2)
  (transitive-net-1 ?attr ?e1 ?e2)]
 [(transitive-net-2 ?attr ?e1 ?e2)
  (transitive-net-1 ?attr ?e1 ?x)
  (transitive-net-1 ?attr ?x ?e2)
  [(!= ?e1 ?e2)]]
 ;; Same as transitive-net-2 but to depth 3
 [(transitive-net-3 ?attr ?e1 ?e2)
  (transitive-net-1 ?attr ?e1 ?e2)]
 [(transitive-net-3 ?attr ?e1 ?e2)
  (transitive-net-2 ?attr ?e1 ?x)
  (transitive-net-2 ?attr ?x ?e2)
  [(!= ?e1 ?e2)]]
 ;; Same as transitive-net-2 but to depth 4
 [(transitive-net-4 ?attr ?e1 ?e2)
  (transitive-net-1 ?attr ?e1 ?e2)]
 [(transitive-net-4 ?attr ?e1 ?e2)
  (transitive-net-3 ?attr ?e1 ?x)
  (transitive-net-3 ?attr ?x ?e2)
  [(!= ?e1 ?e2)]]
 ;; Artist collaboration graph-walking rules, based on generic
 ;; graph-walk rule above
 ;; Supply an artist name as ?artist-name-1, an ?artist-name-2 will be
 ;; bound to the names of artists who directly collaborated with the
 ;; artist(s) having that name
 [(collab ?artist-name-1 ?artist-name-2)
  [?a1 :artist/name ?artist-name-1]
  (transitive-net-1 :track/artists ?a1 ?a2)
  [?a2 :artist/name ?artist-name-2]]
 ;; Alias for collab
 [(collab-net-1 ?artist-name-1 ?artist-name-2)
  (collab ?artist-name-1 ?artist-name-2)]
 ;; Collaboration network walk to depth 2
 [(collab-net-2 ?artist-name-1 ?artist-name-2)
  [?a1 :artist/name ?artist-name-1]
  (transitive-net-2 :track/artists ?a1 ?a2)
  [?a2 :artist/name ?artist-name-2]]
 ;; Collaboration network walk to depth 3
 [(collab-net-3 ?artist-name-1 ?artist-name-2)
  [?a1 :artist/name ?artist-name-1]
  (transitive-net-3 :track/artists ?a1 ?a2)
  [?a2 :artist/name ?artist-name-2]]
 ;; Collaboration network walk to depth 4
 [(collab-net-4 ?artist-name-1 ?artist-name-2)
  [?a1 :artist/name ?artist-name-1]
  (transitive-net-4 :track/artists ?a1 ?a2)
  [?a2 :artist/name ?artist-name-2]]])
All Rules
0.3s
Runtimes (2)