Datomic

This is a computational article that shows how to use Datomic Free and Datalog query language using the MusicBrainz sample database.

1.
Setup

We start by re-using an environment built in the MusicBrainz Datomic article, it already has Datomic installed and the mbrainz sample database imported.

All we have left to do is to start the transactor.

/usr/bin/nohup /datomic-free/bin/transactor /datomic-free/config/samples/free-transactor-template.properties &> /datomic-free.log & sleep 1
tail /datomic-free.log

2.
Example Query

Let's see if it works. We connect to the database and get the current db value, which we can then query.

(require '[datomic.api :as d])

(def uri "datomic:free://localhost:4334/mbrainz-1968-1973")
(def conn (d/connect uri))

(def db (d/db conn))
user/db
(into [] (d/q '[:find ?id ?type ?gender
       :in $ ?name
       :where
       [?e :artist/name ?name]
       [?e :artist/gid ?id]
       [?e :artist/type ?teid]
       [?teid :db/ident ?type]
       [?e :artist/gender ?geid]
       [?geid :db/ident ?gender]]
     db
     "Janis Joplin"))
1[[3]]

For documentation and basics of Datalog queries with Datomic, you can head over to the offical Datomic docs.

In the following sections we examples of querying the example dataset are provided.

3.
Queries

What are the titles, album names, and release years of John Lennon's tracks?

(into [] (d/q '[:find ?title
 	:in $ ?artist-name
 :where
 [?a :artist/name ?artist-name]
 [?t :track/artists ?a]
 [?t :track/name ?title]]
     db "John Lennon"))
70[[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1]50 more…]

4.
Joins

What are the titles, album names, and release years of John Lennon's tracks?

(into [] (d/q '[:find ?title ?album ?year
 :in $ ?artist-name
 :where
 [?a :artist/name   ?artist-name]
 [?t :track/artists ?a]
 [?t :track/name    ?title]
 [?m :medium/tracks ?t]
 [?r :release/media ?m]
 [?r :release/name  ?album]
 [?r :release/year  ?year]]
     db "John Lennon"))
93[[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3]73 more…]

5.
Expression Clauses

What are the titles, album names, and release years of the John Lennon tracks released before 1970?

(into [] (d/q '[:find ?title ?album ?year
 :in $ ?artist-name
 :where
 [?a :artist/name   ?artist-name]
 [?t :track/artists ?a]
 [?t :track/name    ?title]
 [?m :medium/tracks ?t]
 [?r :release/media ?m]
 [?r :release/name  ?album]
 [?r :release/year  ?year]
 [(< ?year 1970)]]
     db "John Lennon"))
18[[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3]]

6.
Rules

To reuse query logic across many queries, you can create and use rules like the following:

(def rules
  '[;; Given ?t bound to track entity-ids, binds ?r to the corresponding
    ;; set of album release entity-ids
 [(track-release ?t ?r)
  [?m :medium/tracks ?t]
  [?r :release/media ?m]]])
user/rules

Load all rules (from the appendix) for later use:

user/all-rules

What are the titles, album names, and release years of John Lennon's tracks?

(into [] (d/q '[:find ?title ?album ?year
 :in $ % ?artist-name
 :where
 [?a :artist/name   ?artist-name]
 [?t :track/artists ?a]
 [?t :track/name    ?title]
 (track-release ?t ?r)
 [?r :release/name  ?album]
 [?r :release/year  ?year]]
     db rules "John Lennon"))
93[[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3],[3]73 more…]

7.
Full text

What are the titles, artists, album names, and release years of all tracks having the word "always" in their titles?

(into [] (d/q '[:find ?title ?artist ?album ?year
 :in $ % ?search
 :where
 (track-search ?search ?track)
 (track-info ?track ?title ?artist ?album ?year)]
              db all-rules "always"))
92[[4],[4],[4],[4],[4],[4],[4],[4],[4],[4],[4],[4],[4],[4],[4],[4],[4],[4],[4],[4]72 more…]

8.
Graph Walk

Who collaborated with one of the Beatles?

(into [] (d/q '[:find ?aname ?aname2
 :in $ % [?aname ...]
 :where (collab ?aname ?aname2)]
              db all-rules ["John Lennon" "Paul McCartney" "George Harrison" "Ringo Starr"]))
5[[2],[2],[2],[2],[2]]

9.
Schema

This is the schema of the example data set.

10.
Plotting

Nextjournal lets you also plot data with Plotly. First, lets query for every John Lennon album with a release date from 1968 to 1973.

(def albums-year 
  (into [] (d/q '[:find ?album ?year
 :in $ % ?artist-name
 :where
 [?a :artist/name   ?artist-name]
 [?t :track/artists ?a]
 [?t :track/name    ?title]
 (track-release ?t ?r)
 [?r :release/name  ?album]
 [?r :release/year  ?year]]
     db all-rules "John Lennon")))

albums-year
16[[2],[2],[2],[2],[2],[2],[2],[2],[2],[2],[2],[2],[2],[2],[2],[2]]

First, collect albums by year. This will make them easy to count and display later.

(defn add-album-to-year [m year title]
  (if (contains? m year) 
    (assoc m year (conj (year m) title)) 
    (assoc m year [title])))

(defn albums-by-year [albums-year m]
  (if-let [album (first albums-year)]
    (let [[title year] album
           sorted-albums (add-album-to-year m (keyword (str year)) title)]
	    (albums-by-year (rest albums-year) sorted-albums))
  	  m))

(albums-by-year albums-year {})
5{:1971[4]:1969[3]:1973[3]:1972[3]:1970[3]}

Second, provide the plotting software with the expected :data and :layout information.

(let [final-sort (albums-by-year albums-year {})]
  {:data [{:x (keys final-sort)
           :y (map #(count (second %)) (albums-by-year albums-year {}))
           :type "bar"}]
   :layout {:autosize false :width 600 :height 500
          :type "bar"
          :xaxis1 {:title "Year"} 
          :yaxis1 {:title "# of Albums"}}})
2{:data[1]:layout{6}}

Finally, plot the data using ClojureScript into and interactive graph!

(.plot js/Nextjournal albums)