HACKER Q&A
📣 networked

Why GraphQL APIs but no Datalog APIs?


Why doesn't anyone use Datalog (or another limited-by-design logic programming language) as the query language in a public Web API? The adoption of GraphQL suggests demand for letting the API user write more sophisticated queries even at a performance cost per query. Datalog would let the user do more, likely saving many HTTP round-trips. A lot more research exists on optimizing Datalog than GraphQL. It is unclear that the total impact on performance would be significantly negative compared to the typical GraphQL API. If this line of thinking is valid, where are the Datalog API experiments?


  👤 Etheryte Accepted Answer ✓
For context, I'm a frontend development lead who oversees a number of projects of different shape and size. My personal and very subjective opinion is it's simply hype. If you say "Datalog" to a frontend developer, they will either hear "obsolete" or just not know what you mean. If you say "GraphQL", they hear "+5 CV points". Obviously that is slightly tongue in cheek, but the marketing side of it is a very real factor.

Facebook currently holds one of the largest levers in frontend development β€” React. If they say the officially recommended way to do X with React is Y, you can be sure that Y will get at least moderate traction.

I'm sure they exist out in the wild, but personally I haven't seen a project where GraphQL really shines through. In my experience, HTTP2 and reasonably well designed RESTful endpoints are the right default to go with. There is the argument that building good APIs is hard, but I believe someone who has reasonable experience with a stable technology will outperform someone using a new tool for the job. If you're not familiar with it, you won't know what to look out for.


👤 j-pb
It's all about the data manipulation capabilities that our current programming languages and libraries provide. Datalog and GraphQL are both graph query languages, but they differ significantly in the datastructures they return.

GraphQL queries describe Tree unfoldings of graphs, and thus return trees.

Datalog describes recursive conjunctive queries on hypergraphs (relational model) without or limited negation, and thus return a set or bag of hypergraph edges.

The reason why GraphQL is so successful is that it fits well with Reacts data-model (trees) and the way it performs its efficient delta updates (tree walking). Furthermore its somewhat easier to implement (albeit not simpler). Consider that there are very few actual GraphQL query engines for actual data (e.g. DGraph DB), instead GraphQL backends implement resolver functions which compute the "graph"/tree on the fly, based on side effects. Resolvers are what you'd call a computable in prolog or datalog knowledge bases and even though I work full time on incremental query evaluation in said knowledge bases I don't have a clue on how to make those efficient without resorting to the Big Cannon of Differential Dataflow.

Datalog queries don't return trees, they return relations a.k.a. hypergraph edges, and that simply doesn't map well to the datastructures (maps/dicts and lists/vectors/arrays, JSON and other tree description formalisms) that we have in basically every mainstream programming language that isn't prolog or some variant on logic programming. So the query results are hard to work with, and you'd want a LINQ style embeddable datalog query engine in your language of choice to work with the returned data, which is a much bigger undertaking.

So to recap: Datalog is great if you have it everywhere and your language is build around it and hypergraphs, but alas most languages we use today are build around trees, and graphql is a tree language.


👤 triska
That's a very justified question!

In my opinion, Datalog or β€” more generally β€” Prolog is where many querying APIs will eventually arrive, because this syntax has many nice advantages: It is very convenient, expressive and readable, programs and queries can be easily parsed and analyzed with Prolog's built-in mechanisms, there is an ISO standard for it etc.

When semantic web formalisms were discussed, Prolog was sometimes mentioned. However, it is so far not very widely used as a querying or modeling language even for semantic web applications. Recent advances in Prolog implementations may increase the uptake of this syntax.

Also, public perception of Datalog and Prolog lags somewhat behind recent developments in these areas and what these technologies can actually do or what they even are. For example, here on HN, many posted articles that claim to be about Datalog often show snippets and examples that use completely different notations which are no longer a subset of ISO Prolog and therefore also not Datalog. Thus, they also do not immediately benefit from recent advances in Prolog systems in that they cannot directly be parsed and interpreted by them. It may take some time to advertise these improvements, and increase interest in these formalisms.


👤 dreamcompiler
Prolog (Datalog) has always been the gold standard for query languages. GraphQL is merely the latest attempt to reinvent it by people who obviously don't know what came before (or don't care). This might be because CS programs are no longer teaching Prolog, but I think it's more likely because one doesn't get career advancement points for using something "old" unless one reinvents it and renames it so it seems "new."

👤 JoelMcCracken
You are asking for a rational explanation for why something better is less popular than something that is worse. The answer almost certainly is "just because", and more specifically "because Facebook made GraphQL, not Datalog".

Graphql had a lot of hype around it when it was first announced. It so happens that I had just learned Prolog at the time and was personally bemoaning why SQL was so popular when a Prolog-like solution would be so much better. A month or so later comes all the podcasts and articles on HN from Facebook devs about GraphQL.

Datalog is a really great technology, and the idea that you can guarantee that each statement terminates is very powerful.

GraphQL seems to be a great technology when you have an extremely specific requirements. I personally would certainly use it if i had those requirements, but IMO it seems totally unjustified for your average tech company.


👤 dustingetz
Hyperfiddle (my startup) is a datalog API experiment, wrapped up in a low-code tool. We vertically integrate Datomic, a datalog database for AWS. Can you imagine GraphQL ever being low-code?

http://www.hyperfiddle.net/

https://gist.github.com/dustingetz/654e502340070280ab9744723...


👤 andrewingram
Ultimately GraphQL is popular because the query ergonomics fit well with the component architecture everyone is using these days, and once you're familiar with it you can be extremely productive.

👤 nickik
This is actually fairly common in Clojure annd Clojure script. Sometimes using the Datomic DB as a backend for it.

This is an older presentation but its all about doing Datalog from the client to the DB.

See: https://www.youtube.com/watch?v=aI0zVzzoK_E

But if you go threw the Clojure world, there is Datalog all over the place, both client and server. Check out Datascript:

https://github.com/tonsky/datascript

This full-stack framework: http://book.fulcrologic.com/


👤 acjohnson55
It's not clear to me that GraphQL and Datalog solve the same problems. GraphQL, to me, is fundamentally about designing the interface between a client and server, decoupled from the logic that satisfies requests. I think it's actually not a query language at all, any more than HTTP is a query language. Datalog is very much a query engine. And it's not super clear to me that the services exposed by a server map cleanly to a logic model. GraphQL requires no such purity.

👤 dzonga
I will say something that is commonly known. a lot of frontend software dev revolves around hype. if React is the in thing and fb dev's are promoting Graphql, then everyone will go towards that. I don't know much about Datalog, but JSonAPiSpec meets most of things that people tout Graphql for. + if you're a person trying to keep employed then you've to follow the trends .

👤 rklaehn
We currently looking for a powerful query language for a distributed event sourcing system. It would have to be embeddable in rust.

Is there a site where I can see more complex datalog queries and how they compare to other query languages? All the examples I have found so far were relatively trivial and basically just assumed that you knew that datalog was super powerful.

A comparison of how to solve problem X, Y, Z in datalog, graphql and SQL would go a long way to convince people that it is a superior alternative.


👤 qeternity
As a company that runs both Django Rest Framework and GraphQL backends, there is one phenomenal argument in favor of GQL: Apollo Client. When you just want a simple, normalized reactive data store it’s really amazing (even though the Apollo kids seem to break it with every minor release). Loads of redux boilerplate go right out the window.

👤 ggleason
TerminusDB does use a Datalog-like language as the query language for its public web API. Datalog has long seemed to me to be the obvious "next step" for query languages as it enables richer manipulations than those provided by vanilla SQL-like languages in a clean fashion.

However, as j-pb noted in another comment:

"GraphQL queries describe Tree unfoldings of graphs, and thus return trees.

Datalog describes recursive conjunctive queries on hypergraphs (relational model) without or limited negation, and thus return a set or bag of hypergraph edges."

TerminusDB allows you to use both approaches. Due to the way we have designed our graphs, with a strongly typed schema langage, you can extract sub-graphs as unfolded trees using a special predicate exposed in our datalog language.

This approach (hopefully) gives the best of both worlds, allowing graph traversal and extraction of objects/documents in a single framework, all communicating information via JSON-LD.


👤 beders
Well, there's EQL and various implementations that use datalog under the hood.

https://edn-query-language.org/eql/1.0.0/what-is-eql.html


👤 namelosw
> It is unclear that the total impact on performance would be significantly negative compared to the typical GraphQL API. Some of the problems also apply to GraphQL, people just need to constrain the query complexity in the end.

But yeah, it would bite Datalog more if not designed well enough, and it's harder to tackle Datalog than to GraphQL.

Other thoughts on this topic:

I love GraphQL but I would love Datalog more if the toolchain is as good as GraphQL.

There's a spectrum on API of simplicity and power -- on the left, there is HTTP/REST which could be seen as a primitive/naive way of describing API it's not consumer-driven at all. On the right there's Datalog which is a powerful way of describing API, it can maximize the flexibility of both the server and the client.

Things like 'map' and 'filter' became well-known to average programmers not very long time ago, though the elite programmers already knew them well half a centry ago. Datalog is definitely a very good idea for API, but I'm afraid it wound't be as popular as GraphQL if Facebook chose this route instead.

That being said, the industry always adopts things in a very passive manner, usually because the current solution is too painful to use, or almost couldn't solve the problem on new problem scale. There's a documentary[0] for GraphQL, the creators had enough of REST, they ended up with inventing GraphQL. It originally was to solving their data fetching problems, in the meantime it would solve others too. Meanwhile, it pushed the common understanding of API for average programmer from left to right on the spectrum. If it's close enough to the right, more and more people would discover Datalog and buzz other people to use it.

Just like type system, it's hard to expect the industry have a deep understanding on the topic itself. Type systems similar to Java dominates the industry for a long time, though ML already existed almost half a centry. Recently, languages like TypeScript tookoff because people need to type existing JavaScript. Then people would discover that the type system they used to is far from enough to describe existing code design. It's like a pandora box, new type operator would came out every release, until people realize there's a spectrum of type system (Lambda Cube[1]), and Dependent T ype would be on the same spot as Datalog on the spectrum.

[0] https://youtu.be/783ccP__No8?t=645 [1] https://en.wikipedia.org/wiki/Lambda_cube


👤 imtringued
How do I implement Datalog in my application? Crickets...

👤 macca321
I've got this idea that you can chuck JMESPath into the range header on a standard json endpoint for ghetto graphql

👤 scotty79
For me the most important thing about graphql is composability.

Child component can communicate their data requirements to the parent and parent can include them in the query and feed the data to the children as it arrives.

Don't know if Datalog has anything for that.