Why does Datomic block shutting down when using :timeout in d/query?

Question

We started adding a :timeout to our d/query calls, and noticed that this made our app and test suite not shutdown properly anymore. I have pinpointed this to an thread pool-2-thread-1 that is keeping the VM alive, because it does not have the daemon flag set. If we don't use :timeout, this thread is not being created.

It seems that the ThreadFactory used by datomic.datalog/cancel-service does not set this daemon flag. Is this intentional?

As a workaround we now use this, which seems to work and doesn't block shutdown:

(.setThreadFactory datomic.datalog/cancel-service
                   (reify java.util.concurrent.ThreadFactory
                     (newThread [_this runnable]
                       (doto (Thread. runnable)
                         (.setName "cancel-service")
                         (.setDaemon true))

I think the current behavior is a bug/oversight, and it would be great if these threads were made daemons by default in the future (just like every other thread Datomic seems to create).

Answer 1

Hi @Leah,

Thanks for the report. Instead of your work around, I would ask if you have tried to utilize release in your peer applications that you intend to shut down. I believe that it will release these threads without requiring special action on your part.

https://docs.datomic.com/clojure/index.html#datomic.api/release

I am not entirely sure on the oversight/bug aspect, but will bring up this observation with the team and circle back.

Cheers,
Jaret

Comments

@Leah, Do you have a minimal reproduction of this issue? It seems you have bisected this down and we would like to look at a repro to see what you are seeing.  If you don't want to post here feel free to open a ticket by e-mailing support@datomic.com --Cheers, Jaret

---

Thanks for the suggestion. Unfortunately, calling `d/release` does not help either. I'll make a small reproducer.

---

deps.edn:

    {:paths ["."]
     :deps  {org.clojure/clojure {:mvn/version "1.11.3"}
             com.datomic/peer {:mvn/version "1.0.6733"}}}

code.clj:

    (ns code
      (:require [datomic.api :as d]))
    
    (let [conn (d/connect "datomic:...")]
      (prn conn)
    
      @(d/transact conn [{:db/ident :movie/title
                         :db/valueType :db.type/string
                         :db/cardinality :db.cardinality/one
                         :db/doc "The title of the movie"}])
      @(d/transact conn [{:movie/title "The Goonies"}])
    
      (prn (d/query {:query '{:find [?t]
                              :where [[?m :movie/title ?t]]}
                     :args [(d/db conn)]
                     :timeout 10000
                     }))
    
      (d/release conn)
      (shutdown-agents))
    
run with `clj -M code.clj`. The DB must exist, it works for any query.

If you uncomment timeout, it doesn't hang.