Support java.time.instant

Question

Support for other types from java.Time: Year, YearMonth, LocalTime etc.

Answer 1

The lack of java.time support is getting more and more annoying as our code-base grows.
It turned out that even passing a YearMonth value to a transaction function, which would convert it to java.util.Date is not possible.

I've tried to extend the transit reader & writer handlers, but it was not sufficient, because somewhere along the way the YearMonth instance has been converted into a com.cognitect.transit.impl.TaggedValueImpl and stayed like that when it reached our transaction function.

The same transaction worked with d/with though, because it doesn't seem to involve marshalling and unmarshalling.

I've even checked within datomic.client.impl.shared.io/marshal, that the marshalled byte-array, when unmarshalled, is equal to the input (m)

(defn- marshal
  "Encodes val as Transit data in msgpack. Returns a map with keys:

  :bytes - a Java byte array of the marshalled data
  :length - the length in bytes of the marshalled data.

  Note that for efficiency's sake the byte array may be longer than
  the returned length."
  [m]
  (clojure.pprint/pprint m)
  (let [stm (BytesOutputStream.)
        w (transit/writer stm :msgpack {:handlers write-handlers})]
    (transit/write w m)
    (let [unmarshalled (unmarshal (java.io.ByteArrayInputStream. (.internalBuffer stm)))]
      (clojure.pprint/pprint [unmarshalled (= m unmarshalled)]))
    {:bytes  (.internalBuffer stm)
     :length (.length stm)}))

the result:

{:t 3857,
 :next-t 3858,
 :db-name "predict",
 :database-id "6652bfa3-2e73-445f-8379-2448f824cbf3",
 :tx-id #uuid "25c92f68-a931-44b7-b96d-c4be8fa147fb",
 :tx-data
 [[ginoco.datomic.component/loophole #time/year-month "2000-01"]],
 :op :transact}

[{:t 3857,
  :next-t 3858,
  :db-name "predict",
  :database-id "6652bfa3-2e73-445f-8379-2448f824cbf3",
  :tx-id #uuid "25c92f68-a931-44b7-b96d-c4be8fa147fb",
  :tx-data
  [[ginoco.datomic.component/loophole #time/year-month "2000-01"]],
  :op :transact}
 true]

Comments

here is the repro code i was using, but it relies on the specifics of our system.
when i tried with Datomic Local on an in-memory DB, i couldn't reproduce the issue, because i suspect some of the layers are left out in that implementation.

<pre><code>
(ns repl.java-time
  (:require
    [cognitect.transit :as transit]
    [datomic.client.api :as d]
    [datomic.client.impl.shared.io :as datomic-io]
    [gini.common :refer :all]
    [ginoco.datomic.component :as dc]
    [ginoco.repl :refer :all]
    [richelieu.core :as ri]
    [tick.core :as t])
  (:import (java.io ByteArrayInputStream)
           (java.time YearMonth)))

;;; Passing YearMonth in arguments for Datomic tx fns
(def java-time-transit-read-handlers
  {"year-month" (transit/read-handler
                  (comp (flip ? "year-month reader")
                        time-literals.data-readers/year-month))})

(alter-var-root #'datomic-io/read-handlers
                #(merge % java-time-transit-read-handlers))

(def java-time-transit-write-handlers
  {YearMonth (transit/write-handler "year-month" str str str)})

(alter-var-root #'datomic-io/write-handlers
                #(merge % java-time-transit-write-handlers))

(defn year-month-tx [year-month]
  {:tx-data [[`dc/loophole (t/year-month year-month)]]})

(comment
  (-> time-literals.read-write/tags)

  (ri/advice d/transact)
  (ri/unadvise-var #'d/transact #'ginoco.datomic.api/transact-instants)

  (-> (year-month-tx "2000-01") (->> (d/transact (:conn (gini-db)))) :tx-data)

  ;=== datomic.client.impl.shared.io/marshal ===
  ; at datomic.client.impl.shared.io$marshal.invokeStatic(io.clj:47)
  ;{:database-id "6652bfa3-2e73-445f-8379-2448f824cbf3",
  ; :db-name "predict",
  ; :next-t 3854,
  ; :op :transact,
  ; :t 3853,
  ; :tx-data [[ginoco.datomic.component/loophole
  ;            #<java.time.YearMonth@3aa12edb 2000-01>]],
  ; :tx-id #uuid "8a085222-2672-46a9-8042-1747da42ee8d"}
  ;
  ;=== LOOPHOLE ===
  ; at ginoco.datomic.component$loophole.invokeStatic(component.clj:485)
  ;{:class #<Class@19b1fd6c com.cognitect.transit.impl.TaggedValueImpl>,
  ; :rep "2000-01",
  ; :tag "year-month"}
  ;=>
  ;[#datom[13194139537166 50 #inst"2023-11-03T05:39:14.147-00:00" 13194139537166 true]
  ; #datom[83562883714830 10 :test/entity 13194139537166 true]
  ; #datom[83562883714830 63 "com.cognitect.transit.impl.TaggedValueImpl@b1ff6900"
  ;          13194139537166 true]]

  (ri/advise-var #'d/transact #'ginoco.datomic.api/transact-instants)

  (d/transact (:conn (gini-db)) {:tx-data [[:db/retractEntity :test/entity]]})

  (-> (year-month-tx "2000-01") (->> (d/with (d/with-db (:conn (gini-db))))) :tx-data)

  ;=== LOOPHOLE ===
  ; at ginoco.datomic.component$loophole.invokeStatic(component.clj:485)
  ;{:class #<Class@7438645b java.time.YearMonth>,
  ; :leapYear true,
  ; :month #<java.time.Month@66d4e153 JANUARY>,
  ; :monthValue 1,
  ; :year 2000}
  ;=>
  ;[#datom[13194139537166 50 #inst"2023-11-03T05:36:43.530-00:00" 13194139537166 true]
  ; #datom[83562883714830 10 :test/entity 13194139537166 true]
  ; #datom[83562883714830 63 "2000-01" 13194139537166 true]]

  [datomic-io/marshal]
  [datomic-io/read-handlers]
  [datomic-io/write-handlers]

  (-> (t/year-month "2000-10")
      (#'datomic-io/marshal) :bytes (ByteArrayInputStream.)
      (#'datomic-io/unmarshal))

  ;=== datomic.client.impl.shared.io/marshal ===
  ; at datomic.client.impl.shared.io$marshal.invokeStatic(io.clj:47)
  ;#<java.time.YearMonth@188506f8 2000-10>
  ;
  ;=== year-month reader ===
  ; at gini.common$flip$fn__231.invoke(common.cljc:37)
  ;#<java.time.YearMonth@69f0f656 2000-10>
  ;=> #time/year-month"2000-10"

  )
</code></pre>

the `loophole` transaction function i've used was this:

<pre><code>
(defn loophole [_db-val year-month]
  (-> year-month bean (? "LOOPHOLE"))
  [{:db/ident :test/entity :db/doc (-> year-month str)}])
</code></pre>

where `?` is just a fancy version of `(fn [x comment] (-> x (doto clojure.pprint/pprint)))`.