Dustin Getz / Sep 03 2022
Datafy/nav implementations for java.io.File
{:deps {org.clojure/clojure {:mvn/version "1.11.1"}
com.hyperfiddle/rcf {:mvn/version "20220902-130636"}
compliment/compliment {:mvn/version "0.3.9"}}}
Extensible Data Notation
(ns user.datafy-fs
"nav implementation for java file system traversals"
(:require [clojure.core.protocols :as ccp :refer [nav]]
[clojure.datafy :refer [datafy]]
[clojure.spec.alpha :as s]
[hyperfiddle.rcf :refer [tests]])
(:import [java.nio.file Path Paths Files]
java.io.File
java.nio.file.LinkOption
[java.nio.file.attribute BasicFileAttributes FileTime]))
(hyperfiddle.rcf/enable!)
2.3s
(defn get-extension [path]
(let [found (last (re-find "(\.[a-zA-Z0-9]+)$" path))
ext (and found (subs found 1))]
(or ext nil)))
(tests
"get-extension"
(tests
"empty"
(get-extension "") := nil
(get-extension ".") := nil
(get-extension "..") := nil
(get-extension "image") := nil
(get-extension "image.") := nil
(get-extension "image..") := nil)
(tests
"found"
(get-extension "image.png") := "png"
(get-extension "image.blah.png") := "png"
(get-extension "image.blah..png") := "png"))
0.7s
Document how java.io.File works
(tests
"java.io.File interop"
(def h (clojure.java.io/file "/etc"))
(.getName h) := "etc"
(.getPath h) := "/etc"
(.isDirectory h) := true
(.isFile h) := false
;(.getParent h) := nil -- ??
;(.getParentFile h) := nil -- ??
)
0.5s
What methods are available on File objects?
(-> (datafy java.io.File) :members keys)
0.2s
java.io.File and java.nio.file.Path are entangled machinery, sort it out
(defn file-path [File f]
(-> f .getAbsolutePath (java.nio.file.Paths/get (make-array String 0))))
(tests
(def p (file-path (clojure.java.io/file "/etc")))
(instance? Path p) := true
(-> (datafy Path) :members keys)
(-> p .getRoot str) := "/"
(-> p .getFileName str) := "etc"
(-> p .getParent .getFileName str) := ""
(-> p .getParent .toFile .getName) := ""
_(-> p .getParent .toFile datafy))
0.6s
File metadata is on Path
(defn path-attrs [Path p]
(Files/readAttributes p BasicFileAttributes (make-array java.nio.file.LinkOption 0)))
(tests
(def attrs (path-attrs (file-path (clojure.java.io/file "/etc"))))
(instance? BasicFileAttributes attrs) := true
(.isDirectory attrs) := true
(.isSymbolicLink attrs) := false
(.isRegularFile attrs) := false
(.isOther attrs) := false)
0.5s
Helper to get file metadata from a file. Todo extend this class with datafy. We don't need it here because it's an internal type.
(defn file-attrs [File f] (path-attrs (file-path f)))
(tests
(file-attrs (clojure.java.io/file "/etc")))
0.0s
Datafy impl here
(def ... ...) ; define a value for easy test assertions
(extend-protocol ccp/Datafiable
java.nio.file.attribute.FileTime
(datafy [o] (-> o .toInstant java.util.Date/from)))
(extend-protocol ccp/Datafiable
java.io.File
(datafy [File f]
; represent object's top layer as EDN-ready value records, for display
; datafy is partial display view of an object as value records
; nav is ability to resolve back to the underlying object pointers
; they compose to navigate display views of objects like a link
(let [attrs (file-attrs f)
n (.getName f)]
(as-> {::name n
::kind (cond (.isDirectory attrs) ::dir
(.isSymbolicLink attrs) ::symlink
(.isOther attrs) ::other
(.isRegularFile attrs) (if-let [s (get-extension n)]
(keyword (namespace ::foo) s)
::unknown-file-type)
() ::unknown-kind)
::absolute-path (-> f .getAbsolutePath)
::created (-> attrs .creationTime .toInstant java.util.Date/from)
::accessed (-> attrs .lastAccessTime .toInstant java.util.Date/from)
::modified (-> attrs .lastModifiedTime .toInstant java.util.Date/from)
::size (.size attrs)} %
(merge % (if (= ::dir (::kind %))
{::children (lazy-seq (.listFiles f))
::parent ...}))
(with-meta % {ccp/nav
(fn [xs k v]
(case k
; reverse data back to object, to be datafied again by caller
::modified (.lastModifiedTime attrs)
::created (.creationTime attrs)
::accessed (.lastAccessTime attrs)
::children (some-> v vec)
::parent (-> f file-path .getParent .toFile)
v))})))))
0.1s
Does it work?
(tests
; careful, calling seq loses metas on the underlying
(def h (clojure.java.io/file "/etc"))
(type h) := java.io.File
"(datafy file) returns an EDN-ready data view that is one layer deep"
(datafy h)
:= :user.datafy-fs{:name "etc",
:absolute-path _,
:size _,
:modified _,
:created _,
:accessed _,
:kind ::dir,
:children _
:parent ...})
0.9s
Nav, note that the (nav) contract is to return the underlying object instance not the data view. Call datafy again on the object if you want.
(tests
"datafy of a directory includes a Clojure coll of children, but child elements are native file
objects"
(as-> (datafy h) %
(nav % ::children (::children %))
(datafy %)
(take 2 (map type %)))
:= [java.io.File java.io.File]
"nav to a leaf returns the native object"
(as-> (datafy h) %
(nav % ::modified (::modified %)))
(type *1) := java.nio.file.attribute.FileTime
"datafy again to get the plain value"
(type (datafy *2)) := java.util.Date)
1.1s
Nav into the filesystem directories. You can see in the notebook result that ::children is a list of objects, not data. The ::children list is lazy
(tests
(as-> (datafy h) %
(nav % ::children (::children %))
(datafy %) ; can skip - simple data
(nav % 0 (% 0))
(datafy %)
_(s/conform ::file %))
:= :user.datafy-fs{:name "logrotate.d",
:absolute-path _,
:size _,
:modified _,
:created _,
:accessed _,
:kind ::dir,
:children _
:parent ...})
0.4s
Nav back into parent – ::parent is not realized until you actually nav it.
(tests
"nav into children and back up via parent ref"
(def m (datafy h))
(::name m) := "etc"
(as-> m %
(nav % ::children (::children %))
(datafy %) ; dir
(nav % 0 (get % 0)) ; first file in dir
(datafy %)
(nav % ::parent (::parent %)) ; dir (skip level on way up)
(datafy %)
(::name %))
:= "etc")
0.5s