Datomic Knowledgebase - Recent questions tagged peer api

Why does Datomic block shutting down when using :timeout in d/query?

Tue, 16 Jul 2024 12:19:45 +0000

We started adding a :timeout to our d/query calls, and noticed that this made our app and test suite not shutdown properly anymore. I have pinpointed this to an thread pool-2-thread-1 that is keeping the VM alive, because it does not have the daemon flag set. If we don't use :timeout, this thread is not being created.

It seems that the ThreadFactory used by datomic.datalog/cancel-service does not set this daemon flag. Is this intentional?

As a workaround we now use this, which seems to work and doesn't block shutdown:

(.setThreadFactory datomic.datalog/cancel-service
                   (reify java.util.concurrent.ThreadFactory
                     (newThread [_this runnable]
                       (doto (Thread. runnable)
                         (.setName "cancel-service")
                         (.setDaemon true))

I think the current behavior is a bug/oversight, and it would be great if these threads were made daemons by default in the future (just like every other thread Datomic seems to create).

Store values encrypted, allowing key rotation.

Sun, 25 Oct 2020 19:14:02 +0000

In our application we have a requirement to store certain fields encrypted. This encryption is done in the application. However, we would like to be able to rotate the encryption key should it become compromised, and retain the historical query features that you would get with unencrypted storage. In order to rotate the key, we have to excise the data and re-transact it with the new key, or store the encrypted values outside of Datomic. It would be nice to have an encryption API that enabled this use case.

Backup and Restore available via Peer API

Thu, 22 Oct 2020 18:14:38 +0000

Allow for on-prem backup and restore via the peer-api.

Ability to limit the number of results from query

Thu, 22 Oct 2020 17:42:57 +0000

Currently there is no built in way to limit the results from a query. This puts the task on the user to limit the results by first querying for entity ids, limiting that number and then pulling the data.

Query explain for datalog

Thu, 22 Oct 2020 16:24:23 +0000

One of the places that we struggle with the most is understanding query performance. In general we get the idea that in order to write a fast query you need to reduce the search space as quickly as possible. As a result the most specific datalog clause should go first. However, we have found cases when that is not always true. For example when an attribute is a reference or when an attribute is indexed. Since we have little insight into how the query planner works its hard for us to tell what will make a query fast. We can and do optimize our queries by hand. But as our system grows and we add more queries this hand optimization becomes more burdensome. It would be great to have a tool, similar to a sql explain, that would give us the fastest ordering of the datalog clauses and provide suggestions for adding indexes.

Expose standby/active transactor status in on-prem metrics.

Thu, 22 Oct 2020 16:00:30 +0000

Provide a method through an API to allow monitoring of standby/active status of transactors in on-prem.