Datomic Knowledgebase - Recent questions and answers in Client API

Answered: Pulling :db/id will pull the entire entity

Fri, 14 Jul 2023 15:53:05 +0000

This bug boxes me in a corner ... I don't know the "type" of the entity whose :db/id I'm trying to pull, so I can't add an addtional attribute to work around the problem. And just using [*] has the big liability noted in the report above.

Answered: Duplicate var in :find and :with throws ArrayIndexOutOfBoundsException

Sun, 28 May 2023 21:30:11 +0000

(Not an answer, but another example of the scope of this bug.)

I think the same underlying error can be provoked when using pull and not using with.

(d/q {:query '[:find ?c (pull ?c [*])

                   :where 
                   , [?c :ansi.fips/state-numeric "51"]
                   , [?c :usps/state-name ?sn]]
          :args [db]})

That ^ provokes the IndexOutOfBoundsException but it can be resolved by removing either the pull or the bare ?c from the find expression. This is frustrating for automated query construction.

(And I've given up trying to get the code sample to format correctly... the embedded quote in the code wreaks havoc with this site's formatter....)

Answered: Error connecting to datomic cloud

Mon, 19 Dec 2022 14:02:51 +0000

This error indicates that you do not have the sourced AWS credentials to connect to the AWS account you are running Cloud in.

https://docs.datomic.com/cloud/operation/access-control.html#how-datomic-access-control-works

java.lang.IllegalArgumentException: Cannot resolve key: 24a96e20-f526-4f7f-ba38-4f684caa5607

Mon, 06 Jun 2022 19:37:30 +0000

I am trying to implement resursive rule (loglcal OR) by following this video - Lucas Cavalcanti & Edward Wible - Exploring four hidden superpowers of Datomic

Here the code

(defn owns? [cid pid db]
(d/q '{:find [?pur]

     :in [$ ?cus-id ?pur %]
     :where
     [(owns? ?cus-id ?pur)]}
db cid [:purchase/id pid] owner-rules))

(comment
(owns?

#uuid "0fb7ea94-44af-46fa-98ca-0ddb5eb23123"
#uuid "24a96e20-f526-4f7f-ba38-4f684caa5607"
(d/db bank-conn)))

While running the above example I am getting this error:
java.lang.IllegalArgumentException Cannot resolve key: 24a96e20-f526-4f7f-ba38-4f684caa5607

Here is the full code:

(def purchase-schema
[{:db/ident :purchase/id

:db/doc "The purchase ID"
:db/unique :db.unique/identity
:db/valueType :db.type/uuid
:db/cardinality :db.cardinality/one}

{:db/ident :purchase/account

:db/doc "Purchase Account"
:db/valueType :db.type/ref
:db/cardinality :db.cardinality/one}])

(def account-schema
[{:db/ident :account/id

:db/doc "The Account ID"
:db/unique :db.unique/identity
:db/valueType :db.type/uuid
:db/cardinality :db.cardinality/one}

{:db/ident :account/customer

:db/doc "Customer who own's this account"
:db/valueType :db.type/ref
:db/cardinality :db.cardinality/one}])

(def customer-schema
[{:db/ident :customer/id

:db/doc "Customer ID"
:db/unique :db.unique/identity
:db/valueType :db.type/uuid
:db/cardinality :db.cardinality/one}

{:db/ident :customer/name

:db/doc "Customer name"
:db/valueType :db.type/string
:db/cardinality :db.cardinality/one}])

;;
;; Create client
(def client (d/client {:server-type :dev-local

                   :system "dev"}))

;; Create connections
(def bank-conn (d/connect client {:db-name "bank"}))

;; Transact schemas
(comment
(d/transact bank-conn {:tx-data purchase-schema})
(d/transact bank-conn {:tx-data account-schema})
(d/transact bank-conn {:tx-data customer-schema})
(d/tx-range bank-conn {:start 6}))

(comment
(java.util.UUID/randomUUID))

;; Transact fake data
(def cid #uuid "0fb7ea94-44af-46fa-98ca-0ddb5eb23123")
(def sample-customer {:customer/id cid

                  :customer/name "Jon Snow"})

(def aid #uuid "c579a9f6-f6c3-4718-8ecf-c3003021ee9a")
(def sample-account {:account/id aid

                 :account/customer [:customer/id cid]})

(def pid #uuid "24a96e20-f526-4f7f-ba38-4f684caa5607")
(def sample-purchase {:purchase/id pid

                  :purchase/account [:account/id aid]})

(comment
(d/transact bank-conn {:tx-data [sample-customer]})
(d/transact bank-conn {:tx-data [sample-account]})
(d/transact bank-conn {:tx-data [sample-purchase]}))

(comment
(d/q '[:find ?cus

     :in $ ?cid ?pid
     :where
     [?pur :purchase/id ?pid]
     [?pur :purchase/account ?acc]
     [?acc :account/customer ?cus]
     [?cus :customer/id ?cid]]
  (d/db bank-conn)
  (:customer/id sample-customer)
  (:purchase/id sample-purchase)))

;; owns v1
(defn ownsv1? [cid pid db]
(d/q '[:find ?cus

     :in $ ?customer-id ?purchase-id
     :where
     [?pur :purchase/id ?purchase-id]
     [?pur :purchase/account ?acc]
     [?acc :account/customer ?cus]
     [?cus :customer/id ?customer-id]]
db cid pid))

(comment
(ownsv1?

(:customer/id sample-customer)
(:purchase/id sample-purchase)
(d/db bank-conn)))

;; recursive rule (logical OR)
(def owner-rules
'[[(owns? ?cus-id ?e)

 [?e :customer/id ?cus-id]]
[(owns? ?cus-id ?e)
 [?e ?ref-attr ?r]
 (owns? ?cus-id ?r)]])

(defn owns? [cid pid db]
(d/q '{:find [?pur]

     :in [$ ?cus-id ?pur %]
     :where
     [(owns? ?cus-id ?pur)]}
db cid [:purchase/id pid] owner-rules))

;;
(comment
(owns?

(:customer/id sample-customer)
(:purchase/id sample-purchase)
(d/db bank-conn)))

client api: clear cache when credentials change

Tue, 14 Dec 2021 08:30:00 +0000

I have my development program running and connect to datomic proper. How do I clear the caches because it brakes when I need new aws creds.

Answered: Pull identical var in two :find positions

Tue, 16 Nov 2021 16:03:15 +0000

Hi thank you for the detailed reproduction and problem statement. Each variable can appear in at most one pull expression. This behavior was not documented, but I've now added this to our docs on pull expressions in cloud and on-prem:

https://docs.datomic.com/cloud/query/query-data-reference.html#pull-expressions
https://docs.datomic.com/on-prem/query/query.html#pull-expressions

Cheers,
Jaret

How to install transaction function using the [com.datomic/client-pro "0.9.63"]

Wed, 13 Oct 2021 16:27:12 +0000

How to install transaction function using the [com.datomic/client-pro "0.9.63"] ?
I am following this example https://docs.datomic.com/on-prem/reference/database-functions.html

However there is no d/function to install the transaction function... And looking at the documentation there is only datomic dev local and datomic client pro libraries

How can I create a transaction function using the client pro lib?

How to integrate library using datomic.client.api when using the peer library?

Fri, 03 Sep 2021 17:33:57 +0000

I am using the peer library for on-prem Datomic. Now I want to use a library - ragtime.datomic - that uses the datomic.client.api. How can I do that?

Based on https://docs.datomic.com/client-api/datomic.client.api.html#var-client it seems the only option is to run a separate peer process, exposed at a port and set up authorization between the two. But it seems wasteful to run a separate server when I already have the peer library. Is there a simpler way?

Answered: `Unable to read keyfile` error when trying to initialize Datomic Cloud with Client API

Wed, 25 Aug 2021 13:38:05 +0000

Hi Julia,

That looks like your REPL in intelliJ is missing aws credentials. These can be set by editing your REPL configuration in intelliJ and setting the environment credentials for AWS_SECRET_KEY etc:

http://imgur.com/a/P4tdshR

You can also provide a named profile if you have AWS profiles configured. https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html. You can use these profiles in your connection (https://docs.datomic.com/cloud/tutorial/client.html#using-datomic-cloud) to a Datomic Cloud system, but if you are connecting locally from your laptop your REPL will also need sourced credentials that can access the AWS resources.

Answered: Should you cache d/connect calls?

Fri, 18 Jun 2021 15:03:10 +0000

At least more time on doing actions which require datomic connection. More DB data more time connection

Answered: Recommended way to handle :db/ident enums when used with d/pull?

Tue, 08 Jun 2021 10:38:29 +0000

This is the helper I'm using, I'm sure it performs better than the one you referred. Also, it doesn't require you to explicitly pull :db/ident fields and it works on pull-many and pull results. Consider replacing map with mapv if you want to retain vectors instead of lazy seqs.

(defn resolve-idents
  "Helper that resolves all idents in a pull expression return to
  their :db/ident keyword."
  [db x]
  (cond (map? x)
        (if-let [ident (d/ident db (:db/id x))]
          ident
          (reduce-kv (fn [x k v]
                       (assoc x k (resolve-idents db v)))
                     x x))
        (vector? x)
        (map #(resolve-idents db %) x)
        :else
        x))

Regarding the question how to deal with this:

Use a helper like this and wrap it in your own version of pull/pull-many:

(defn full 
  "Version of pull that resolves idents"
  [db expr id]
  (->> (d/pull db expr id) (resolve-idents db)))

Regarding this topic in general:

I'd really be interested in Cognitects reasons to implement this behavior differently from the good old entity API. There are obviously some pros and cons, but it would be interesting to know the decisive reasons.

Answered: How to gracefully handle optional query inputs?

Tue, 01 Jun 2021 19:12:29 +0000

Instead of doing it 'in' the query, use clojure's data manip to create the query you want.

This is where queries themselves being data is really cool. The map form may be more useful in this context. Here's a quick and dirty... Added assumed :product/id to make it clearer. IRL, you'd want to make sure more selective clauses are first, etc etc. If you have complex combos, you might want to look at something like meander

(defn product-q
    [name]
    (cond-> '{:find [?p]
              :in [$]
              :where [[?p :product/id _]]}
      name (-> (update :in conj '?name)
             (update :where conj '[?p :product/name ?name]))))

(product-q nil)
=> {:find [?p]
    :in [$]
    :where [[?p :product/id _]]}
    
(product-q "cheetos")
=> {:find [?p]
    :in [$ ?name]
    :where [[?p :product/id _]
            [?p :product/name ?name]]

Additional keys on anomaly part of the API?

Sun, 09 May 2021 01:33:17 +0000

Datomic sometimes attaches additional keys onto the anomaly in ex-data on the ExceptionInfos thrown. Is this additional data part of the API and able to be depended on? Further, could all keys returned be documented in the api docs? The returned keys are typically quite helpful in dispatching on different Datomic error "types."

Here's an example.

(def client (d/client {:server-type :dev-local
                       :storage-dir :mem
                       :system      "aa"}))
=> #'compute.http-api.public.task/client
(d/create-database client {:db-name "test"})
=> true
(def conn (d/connect client {:db-name "test"}))
=> #'compute.http-api.public.task/conn
(d/transact conn {:tx-data [{:db/ident       :user/id
                             :db/cardinality :db.cardinality/one
                             :db/valueType   :db.type/string
                             :db/unique      :db.unique/identity}
                            {:db/ident       :user/name
                             :db/cardinality :db.cardinality/one
                             :db/valueType   :db.type/string
                             :db/unique      :db.unique/identity}]})
=>
{:db-before #datomic.core.db.Db{:id "47c8f6f7-26f9-4575-97ec-7fc0948823c1",
                                :basisT 5,
                                :indexBasisT 0,
                                :index-root-id nil,
                                :asOfT nil,
                                :sinceT nil,
                                :raw nil},
 :db-after #datomic.core.db.Db{:id "47c8f6f7-26f9-4575-97ec-7fc0948823c1",
                               :basisT 6,
                               :indexBasisT 0,
                               :index-root-id nil,
                               :asOfT nil,
                               :sinceT nil,
                               :raw nil},
 :tx-data [#datom[13194139533318 50 #inst"2021-05-09T01:32:25.677-00:00" 13194139533318 true]
           #datom[73 10 :user/id 13194139533318 true]
           #datom[73 41 35 13194139533318 true]
           #datom[73 40 23 13194139533318 true]
           #datom[73 42 38 13194139533318 true]
           #datom[74 10 :user/name 13194139533318 true]
           #datom[74 41 35 13194139533318 true]
           #datom[74 40 23 13194139533318 true]
           #datom[74 42 38 13194139533318 true]
           #datom[0 13 73 13194139533318 true]
           #datom[0 13 74 13194139533318 true]],
 :tempids {}}
(d/transact conn {:tx-data [{:db/id     [:user/id "a"]
                             :user/name "a"}]})
Execution error (ExceptionInfo) at datomic.core.error/raise (error.clj:55).
:db.error/not-an-entity Unable to resolve entity: [:user/id "a"] in datom [[:user/id "a"] :user/name "a"]
(ex-data *e)
=>
{:cognitect.anomalies/category :cognitect.anomalies/incorrect,
 :cognitect.anomalies/message "Unable to resolve entity: [:user/id \"a\"] in datom [[:user/id \"a\"] :user/name \"a\"]",
 :entity [:user/id "a"],
 :datom [[:user/id "a"] :user/name "a"],
 :db/error :db.error/not-an-entity}

Answered: When would I want to use :server-type :cloud?

Fri, 26 Feb 2021 15:23:17 +0000

A :cloud connection uses endpoint etc. to make a client-server connection.

An :ion connection takes the same arguments, but will ignore them if the code is running on a Cloud node, creating a client implementation that runs entirely in-memory on the current node.

You could use :cloud to force one Datomic Cloud system to connect to a different system.

Answered: Usage of datomic client API's with in order to simulate facts without asserting.

Thu, 04 Feb 2021 22:13:12 +0000

According to the documentation, it looks like the d/with fn requires a DB value that comes from the d/with-db fn. So try:

(d/q '[:find (count ?e1)                             
   :in $                                                                                                                                         
   :where [?e1 :token/color :grey]]                                                                                                 
 (d/with (d/with-db conn) {:tx-data [{:token/color :grey}]}))

Add default :timeout to client api ns docstring

Thu, 24 Dec 2020 00:12:44 +0000

It'd be great to have the default value for :timeout (60000) included in the datomic.client.api ns docstring. Currently this value can only be found in the docs on the website. As of client-cloud 0.8.102, the ns docstring specifies the default value for :limit.

Datomic Client API equivalent to on-prem tx->t, t->tx

Mon, 21 Dec 2020 00:25:17 +0000

The Datomic client API does not provider tx->t or t->tx. These functions can be useful for very specific scenarios (e.g., you have a tx id and want to know if the database has "moved forward").

This was also requested on the forums. A workaround was provided for the user's specific case but the request is still relevant.

Answered: What approach do you use to detect a good connection to avoid :cognitect.anomalies/unavailable exceptions?

Thu, 29 Oct 2020 00:52:24 +0000

We use something very similar to the below code. Most of the code originates from aws-api. The retry code is generic and can be used for all sorts of stuff.

Where you want to retry is often use-case dependent. Something to keep in mind.

(defn with-retry-sync
  "Calls work-fn until retriable? is false or backoff returns nil. If work-fn
  throws, the exception will be passed to retriable?. If it is not retriable, the
  exception will be thrown. work-fn is a function of no arguments. retriable? is
  passed the result or exception from calling work-fn. backoff is a function of
  the number of times work-fn has been called."
  [work-fn retriable? backoff]
  (let [maybe-throw #(if (instance? Throwable %) (throw %) %)]
    (loop [retries 0]
      (let [resp (try (work-fn) (catch Throwable t t))]
        (if (retriable? resp)
          (if-let [bo (backoff retries)]
            (do
              (Thread/sleep bo)
              (recur (inc retries)))
            (maybe-throw resp))
          (maybe-throw resp))))))

(defn capped-exponential-backoff
  "Returns a function of the num-retries (so far), which returns the
  lesser of max-backoff and an exponentially increasing multiple of
  base, or nil when (>= num-retries max-retries).
  See with-retry to see how it is used.
  Alpha. Subject to change."
  ([] (capped-exponential-backoff 100 10000 8))
  ([base max-backoff max-retries]
   (fn [num-retries]
     (when (< num-retries max-retries)
       (min max-backoff
         (* base (bit-shift-left 1 num-retries)))))))

(defn capped-exponential-backoff-with-jitter
  ([] (capped-exponential-backoff-with-jitter {}))
  ([{:keys [base
            max-backoff
            max-retries
            max-jitter-ms]
     :or   {base          100
            max-backoff   20000
            max-retries   4
            max-jitter-ms 100}}]
   (let [backoff-fn (capped-exponential-backoff
                      base
                      max-backoff
                      max-retries)]
     (fn [num-retries]
       (when-let [backoff-ms (backoff-fn num-retries)]
         ;; adding "jitter" can help reduce throttling on retries:
         ;; https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
         (min (+ backoff-ms (rand-int max-jitter-ms)) max-backoff))))))

(defn default-retriable?
  "Returns true if x is an anomaly or if it is an ExceptionInfo with an anomaly
  in its ex-data."
  [x]
  (or
    (contains? #{:cognitect.anomalies/busy
                 :cognitect.anomalies/unavailable}
      (:cognitect.anomalies/category x))
    (and (instance? clojure.lang.ExceptionInfo x)
      (default-retriable? (ex-data x)))))

(defn connect
  [client arg-map]
  (with-retry-sync #(d/connect client arg-map)
    default-retriable? (capped-exponential-backoff-with-jitter)))

Client library for Python

Thu, 22 Oct 2020 16:45:51 +0000

Provide client implementation in Python

Client API: tx-report-queue equivalent

Thu, 22 Oct 2020 16:25:01 +0000

One of the features I miss a lot in the client API is TX report queues. Would it be possible to add something like this to the client? Perhaps tx-range could get an option that, when no :end is given, keeps the channel open and waits for future transactions?