
Ask HN: Why GraphQL APIs but no Datalog APIs? - networked
Why doesn&#x27;t anyone use Datalog (or another limited-by-design logic programming language) as the query language in a public Web API?  The adoption of GraphQL suggests demand for letting the API user write more sophisticated queries even at a performance cost per query.  Datalog would let the user do more, likely saving many HTTP round-trips.  A lot more research exists on optimizing Datalog than GraphQL.  It is unclear that the total impact on performance would be significantly negative compared to the typical GraphQL API.  If this line of thinking is valid, where are the Datalog API experiments?
======
Etheryte
For context, I'm a frontend development lead who oversees a number of projects
of different shape and size. My personal and very subjective opinion is it's
simply hype. If you say "Datalog" to a frontend developer, they will either
hear "obsolete" or just not know what you mean. If you say "GraphQL", they
hear "+5 CV points". Obviously that is slightly tongue in cheek, but the
marketing side of it is a very real factor.

Facebook currently holds one of the largest levers in frontend development —
React. If they say the officially recommended way to do X with React is Y, you
can be sure that Y will get at least moderate traction.

I'm sure they exist out in the wild, but personally I haven't seen a project
where GraphQL really shines through. In my experience, HTTP2 and reasonably
well designed RESTful endpoints are the right default to go with. There is the
argument that building good APIs is hard, but I believe someone who has
reasonable experience with a stable technology will outperform someone using a
new tool for the job. If you're not familiar with it, you won't know what to
look out for.

~~~
shantly
> I'm sure they exist out in the wild, but personally I haven't seen a project
> where GraphQL really shines through. In my experience, HTTP2 and reasonably
> well designed RESTful endpoints are the right default to go with. There is
> the argument that building good APIs is hard, but I believe someone who has
> reasonable experience with a stable technology will outperform someone using
> a new tool for the job. If you're not familiar with it, you won't know what
> to look out for.

GraphQL doesn't make a ton of sense to me, unless you're expecting many
clients with significantly different query pattern requirements and prefer to
take on performance and (maybe) security uncertainty & complexity to
accommodate that more easily.

If you only expect one client (or a few very similar ones) and you want
GraphQL-like division of labor just have your frontend and backend folks
actually talk to each other and maybe have the backend folks also write a
client library for their services (my preference, most of the time).

[EDIT] but you bet your ass I'm not putting up much of a fight against it if
I'm in anything lower than a development lead or architect position—no use
fighting that fight from poor footing when, as you mention, it's +5 CV points
anyway, so who cares if it's maybe not the best solution :-)

~~~
zo1
> _" GraphQL doesn't make a ton of sense to me, unless you're expecting many
> clients with significantly different query pattern requirements and prefer
> to take on performance and (maybe) security uncertainty & complexity to
> accommodate that more easily."_

My impression of it is that it's an "API" that's masquerading as a dumb relay,
essentially giving almost datastore-level access to the frontend client (even
though they don't market it that way). And this is done so that "Frontend
devs" can just do "Frontend" things with access to all the data they might end
up using. A bit snarky, but I do see it as a way to just "offload" the problem
down the line, instead of thinking about the API-usage and what it means for
potential frontend clients. That's the "old way" of doing things of course.

It fits in to the rest of the "movement" or trend currently happening: Thick
(web) clients that do everything and the kitchen sink (including auth. See JWT
in arb HTTP headers instead of cookies), SPAs so that there is no server side
page rendering (this must all be done in the thick web client you see),
document databases instead of SQL so the "frontend" devs don't have to worry
about schemas, SPAs so that we break half the bloody internet because we now
need to retrofit browser navigation using hashtags, everything has to be an
"API" now because we can't convey data using server-rendering, etc.

Honestly, just that navigation thing alone is nuts and I can't believe we ever
allowed it to be imposed on us. At this point, I'm _afraid_ to touch those
buttons on a new website without first trying them to learn their specific
behavior on that site.

~~~
sundbry
You can handle proper URLs in a SPA if you know what you're doing. It's not
something your average frontend dev would pay attention to though, sadly.
Server-side "isomorphic" rendering of pages is also possible but a lot more
complicated to resolve all the async requests that have to run to render the
page.

~~~
rienbdj
Unless you use GraphQl of course!

------
j-pb
It's all about the data manipulation capabilities that our current programming
languages and libraries provide. Datalog and GraphQL are both graph query
languages, but they differ significantly in the datastructures they return.

GraphQL queries describe Tree unfoldings of graphs, and thus return trees.

Datalog describes recursive conjunctive queries on hypergraphs (relational
model) without or limited negation, and thus return a set or bag of hypergraph
edges.

The reason why GraphQL is so successful is that it fits well with Reacts data-
model (trees) and the way it performs its efficient delta updates (tree
walking). Furthermore its somewhat easier to implement (albeit not simpler).
Consider that there are very few actual GraphQL query engines for actual data
(e.g. DGraph DB), instead GraphQL backends implement resolver functions which
compute the "graph"/tree on the fly, based on side effects. Resolvers are what
you'd call a computable in prolog or datalog knowledge bases and even though I
work full time on incremental query evaluation in said knowledge bases I don't
have a clue on how to make those efficient without resorting to the Big Cannon
of Differential Dataflow.

Datalog queries don't return trees, they return relations a.k.a. hypergraph
edges, and that simply doesn't map well to the datastructures (maps/dicts and
lists/vectors/arrays, JSON and other tree description formalisms) that we have
in basically every mainstream programming language that isn't prolog or some
variant on logic programming. So the query results are hard to work with, and
you'd want a LINQ style embeddable datalog query engine in your language of
choice to work with the returned data, which is a much bigger undertaking.

So to recap: Datalog is great if you have it everywhere and your language is
build around it and hypergraphs, but alas most languages we use today are
build around trees, and graphql is a tree language.

------
triska
That's a very justified question!

In my opinion, Datalog or — more generally — Prolog is where many querying
APIs will eventually arrive, because this syntax has many nice advantages: It
is very convenient, expressive and readable, programs and queries can be
easily parsed and analyzed with Prolog's built-in mechanisms, there is an ISO
standard for it etc.

When semantic web formalisms were discussed, Prolog was sometimes mentioned.
However, it is so far not very widely used as a querying or modeling language
even for semantic web applications. Recent advances in Prolog implementations
may increase the uptake of this syntax.

Also, public perception of Datalog and Prolog lags somewhat behind recent
developments in these areas and what these technologies can actually do or
what they even are. For example, here on HN, many posted articles that claim
to be about Datalog often show snippets and examples that use completely
different notations which are no longer a subset of ISO Prolog and therefore
also not Datalog. Thus, they also do not immediately benefit from recent
advances in Prolog systems in that they cannot directly be parsed and
interpreted by them. It may take some time to advertise these improvements,
and increase interest in these formalisms.

~~~
patrickmn
What's the best example of modern Datalog?

~~~
triska
A good way to try Datalog is to use a state-of-the-art Prolog system that
supports _SLG resolution_ (tabling).

SLG resolution is an evaluation strategy of Prolog programs that provides
favourable termination properties compared to SLDNF resolution which is
Prolog's default execution strategy.

A great example for this is XSB Prolog, since it has pioneered and refined
many of these techniques and serves as an example and benchmark for many other
Prolog systems in this area:

[https://xsb.com/xsb-prolog](https://xsb.com/xsb-prolog)

The nice thing is that this approach allows you to use Datalog, and at the
same time gives you a Turing-complete programming language that is
comparatively easy to grasp once you understand Datalog, while sharing many of
the same advantages and being syntactically only a slight extension of it.

~~~
mycall
A turning complete query language sounds like a recipe for DoS attacks.

~~~
maweki
It is only Turing complete if you allow new rules. Usually a client would only
be allowed to formulate query goals.

In that sense, GraphQL is Turing complete as well, as the underlying
implementation could just simulate a Turing machine.

~~~
onion2k
What I read from your post here is that GraphQL is safe by default and would
require effort to make it dangerous while Datalog is dangerous by default and
would require configuration to make it safe. I know which one I'd rather put
on a server.

~~~
maweki
You have read that incorrectly. A Datalog/Prolog query does not define new
rules. That's like calling arbitrary code on remote machines. You would never
allow that.

But IF you were allowed to define new rules, THEN you could simulate a turing
machine or create infinite reductions.

------
dreamcompiler
Prolog (Datalog) has always been the gold standard for query languages.
GraphQL is merely the latest attempt to reinvent it by people who obviously
don't know what came before (or don't care). This might be because CS programs
are no longer teaching Prolog, but I think it's more likely because one
doesn't get career advancement points for using something "old" unless one
reinvents it and renames it so it seems "new."

~~~
erik_seaberg
I _know_ Prolog (from college) and SPARQL (from hobbyist dinking around with
SemWeb). I love DSLs and weird languages. I should be almost the ideal Datalog
user, so how did it stay completely off my radar until this post?

~~~
OJFord
Because it's a niche tangent of an already niche language.

I'm only aware of it because a databases course taught it (relational algebra
to datalog to SQL) and I thought it was superb, and a couple of times since
I've tried to find if I can use it, but it seems to suffer from the
combination of age + academia + proprietariness that makes such things hard to
discover and use. (Compared to graphql and plethora of open source libs on
Github, etc.)

------
JoelMcCracken
You are asking for a rational explanation for why something better is less
popular than something that is worse. The answer almost certainly is "just
because", and more specifically "because Facebook made GraphQL, not Datalog".

Graphql had a lot of hype around it when it was first announced. It so happens
that I had just learned Prolog at the time and was personally bemoaning why
SQL was so popular when a Prolog-like solution would be so much better. A
month or so later comes all the podcasts and articles on HN from Facebook devs
about GraphQL.

Datalog is a really great technology, and the idea that you can guarantee that
each statement terminates is very powerful.

GraphQL seems to be a great technology when you have an extremely specific
requirements. I personally would certainly use it if i had those requirements,
but IMO it seems totally unjustified for your average tech company.

------
dustingetz
Hyperfiddle (my startup) is a datalog API experiment, wrapped up in a low-code
tool. We vertically integrate Datomic, a datalog database for AWS. Can you
imagine GraphQL ever being low-code?

[http://www.hyperfiddle.net/](http://www.hyperfiddle.net/)

[https://gist.github.com/dustingetz/654e502340070280ab9744723...](https://gist.github.com/dustingetz/654e502340070280ab9744723a8ae250)

~~~
leetrout
That is really cool. I’ll have to check this out this week.

FYI Your HTTPS cert is tied to cloudfront and not configured correctly.

------
andrewingram
Ultimately GraphQL is popular because the query ergonomics fit well with the
component architecture everyone is using these days, and once you're familiar
with it you can be extremely productive.

~~~
dominotw
sorry not a frontend dev, where can i get basic info about 'component
architecture'

~~~
hmcdona1
Modern SPA frameworks are usually built on components. You can combine
components to define a tree of elements that represents your UI. This pattern
is also often referred to as a Declarative UI pattern.

GraphQL works well with declarative UIs, because you can define fragments and
combine them to create queries that best match the data the current set of
components you are loading require. In this way, it's flexible in a way REST
isn't (at least typically) and syncs well with Declarative UIs.

I can't speak to datalog though.

------
nickik
This is actually fairly common in Clojure annd Clojure script. Sometimes using
the Datomic DB as a backend for it.

This is an older presentation but its all about doing Datalog from the client
to the DB.

See:
[https://www.youtube.com/watch?v=aI0zVzzoK_E](https://www.youtube.com/watch?v=aI0zVzzoK_E)

But if you go threw the Clojure world, there is Datalog all over the place,
both client and server. Check out Datascript:

[https://github.com/tonsky/datascript](https://github.com/tonsky/datascript)

This full-stack framework:
[http://book.fulcrologic.com/](http://book.fulcrologic.com/)

~~~
myguidingstar
> This is actually fairly common in Clojure annd Clojure script. Sometimes
> using the Datomic DB as a backend for it. What do you mean by "this" which
> is fairly common? Datascript is a Datalog implementation, however you can't
> just bring your query from backend to frontend, not only for incompatiblity
> but also for security issues. And Datascript is not that popular among
> Clojurescript apps.

Fulcro is defintely the go to framework. It makes use of EQL, which is a
superior alternate to Graphql, for server-client communication.

------
acjohnson55
It's not clear to me that GraphQL and Datalog solve the same problems.
GraphQL, to me, is fundamentally about designing the interface between a
client and server, decoupled from the logic that satisfies requests. I think
it's actually not a query language at all, any more than HTTP is a query
language. Datalog is very much a query engine. And it's not super clear to me
that the services exposed by a server map cleanly to a logic model. GraphQL
requires no such purity.

~~~
fmjrey
That: GraphQL is a mechanism for clients to specify the data document they
want, and for servers to specify the documents they can serve. Datalog, or SQL
for that matter, is a mechanism to expose data within a database. The GraphQL
data model must be geared towards the end client point of view, e.g. some
internal IDs can be hidden. Datalog on the other hand exposes the internal
model of a data store, which is domain driven not client driven. Also you do
not want end client devices to have the full power of datalog or SQL on your
database, or else be ready to always watch those crazy queries that bring your
server down to a crawl. GraphQL at least provides an indirection point where
you can apply some form of resource control.

------
dzonga
I will say something that is commonly known. a lot of frontend software dev
revolves around hype. if React is the in thing and fb dev's are promoting
Graphql, then everyone will go towards that. I don't know much about Datalog,
but JSonAPiSpec meets most of things that people tout Graphql for. + if you're
a person trying to keep employed then you've to follow the trends .

~~~
JMTQp8lwXL
React came out in 2013. That's six years and counting of "hype".

------
rklaehn
We currently looking for a powerful query language for a distributed event
sourcing system. It would have to be embeddable in rust.

Is there a site where I can see more complex datalog queries and how they
compare to other query languages? All the examples I have found so far were
relatively trivial and basically just assumed that you knew that datalog was
super powerful.

A comparison of how to solve problem X, Y, Z in datalog, graphql and SQL would
go a long way to convince people that it is a superior alternative.

------
qeternity
As a company that runs both Django Rest Framework and GraphQL backends, there
is one phenomenal argument in favor of GQL: Apollo Client. When you just want
a simple, normalized reactive data store it’s really amazing (even though the
Apollo kids seem to break it with every minor release). Loads of redux
boilerplate go right out the window.

~~~
acemarke
Per the last comment, please check out our new official Redux Toolkit package.
It includes utilities to simplify several common Redux use cases, including
store setup, defining reducers, immutable update logic, and even creating
entire "slices" of state at once without writing any action creators or action
types by hand:

[https://redux-toolkit.js.org](https://redux-toolkit.js.org)

Not saying it's a replacement for Apollo (since Redux itself doesn't provide
anything around data fetching), but we specifically created RTK to eliminate
the concerns around "Redux boilerplate".

~~~
damidekronik
Still requires boilerplate for thunks, which in my case is most redux actions.
Also I think you still don't support for specifying same reducer for many
actions (for example as an array - [https://github.com/piotrwitek/typesafe-
actions](https://github.com/piotrwitek/typesafe-actions)). The immutable
update logic however is amazing and I am glad I got introduced to it by you!

~~~
acemarke
Not sure what you mean by "boilerplate for thunks". You write thunks the same
way you always have.

And yes, `createSlice` explicitly supports responding to actions that were
defined elsewhere in the app, using the `extraReducers` argument:

[https://redux-toolkit.js.org/api/createSlice#extrareducers](https://redux-
toolkit.js.org/api/createSlice#extrareducers)

Although... are you saying that you want one reducer function to handle many
different actions in one slice? Just define the reducer outside the slice
itself, and pass it as the handler for each action:

    
    
        createSlice({
            name: "someSlice",
            initialState,
            reducers: {
                action1: someCaseReducer,
                action2: someCaseReducer,    
            }
        })
    

Did something similar in my own app recently at work, where multiple slices
all needed to handle loading data similarly. So, I wrote a couple generic
reducers, and passed them in to each of the slices.

------
ggleason
TerminusDB does use a Datalog-like language as the query language for its
public web API. Datalog has long seemed to me to be the obvious "next step"
for query languages as it enables richer manipulations than those provided by
vanilla SQL-like languages in a clean fashion.

However, as j-pb noted in another comment:

"GraphQL queries describe Tree unfoldings of graphs, and thus return trees.

Datalog describes recursive conjunctive queries on hypergraphs (relational
model) without or limited negation, and thus return a set or bag of hypergraph
edges."

TerminusDB allows you to use both approaches. Due to the way we have designed
our graphs, with a strongly typed schema langage, you can extract sub-graphs
as unfolded trees using a special predicate exposed in our datalog language.

This approach (hopefully) gives the best of both worlds, allowing graph
traversal and extraction of objects/documents in a single framework, all
communicating information via JSON-LD.

~~~
cheukting
Sounds reasonable to me :-)

------
beders
Well, there's EQL and various implementations that use datalog under the hood.

[https://edn-query-language.org/eql/1.0.0/what-is-eql.html](https://edn-query-
language.org/eql/1.0.0/what-is-eql.html)

~~~
myguidingstar
EQL is surely a great alternative to Graphql because of namespaced keywords
and rich data structure (in contrast of Graphql's string base). It is also
inspired by Datomic's Pull API. However, it's a specification and the most
popular imlementation (Pathom) has nothing to do with Datalog.

~~~
ptttr
The original question made this mistake probably because the poster didn't
know it but Pathom is the actual alternative to GraphQL-based APIs. Datalog
solves a subtly different problem of querying the data, while
Pathom/GraphQL/REST is about exposing the data (interfacing?) as API.

It's more like:

Pathom vs GraphQL-based API (vs REST)

and

Datalog vs SQL (vs NoSQL)

The mainstream is in the process of migrating from REST to GraphQL, but I
think Pathom could be the black horse in this race. Just watch the last part
of Wilker Silva's The Maximal Graph talk to see the potential:
[https://youtu.be/IS3i3DTUnAI?t=2080](https://youtu.be/IS3i3DTUnAI?t=2080)

------
namelosw
> It is unclear that the total impact on performance would be significantly
> negative compared to the typical GraphQL API. Some of the problems also
> apply to GraphQL, people just need to constrain the query complexity in the
> end.

But yeah, it would bite Datalog more if not designed well enough, and it's
harder to tackle Datalog than to GraphQL.

Other thoughts on this topic:

I love GraphQL but I would love Datalog more if the toolchain is as good as
GraphQL.

There's a spectrum on API of simplicity and power -- on the left, there is
HTTP/REST which could be seen as a primitive/naive way of describing API it's
not consumer-driven at all. On the right there's Datalog which is a powerful
way of describing API, it can maximize the flexibility of both the server and
the client.

Things like 'map' and 'filter' became well-known to average programmers not
very long time ago, though the elite programmers already knew them well half a
centry ago. Datalog is definitely a very good idea for API, but I'm afraid it
wound't be as popular as GraphQL if Facebook chose this route instead.

That being said, the industry always adopts things in a very passive manner,
usually because the current solution is too painful to use, or almost couldn't
solve the problem on new problem scale. There's a documentary[0] for GraphQL,
the creators had enough of REST, they ended up with inventing GraphQL. It
originally was to solving their data fetching problems, in the meantime it
would solve others too. Meanwhile, it pushed the common understanding of API
for average programmer from left to right on the spectrum. If it's close
enough to the right, more and more people would discover Datalog and buzz
other people to use it.

Just like type system, it's hard to expect the industry have a deep
understanding on the topic itself. Type systems similar to Java dominates the
industry for a long time, though ML already existed almost half a centry.
Recently, languages like TypeScript tookoff because people need to type
existing JavaScript. Then people would discover that the type system they used
to is far from enough to describe existing code design. It's like a pandora
box, new type operator would came out every release, until people realize
there's a spectrum of type system (Lambda Cube[1]), and Dependent T ype would
be on the same spot as Datalog on the spectrum.

[0] [https://youtu.be/783ccP__No8?t=645](https://youtu.be/783ccP__No8?t=645)
[1]
[https://en.wikipedia.org/wiki/Lambda_cube](https://en.wikipedia.org/wiki/Lambda_cube)

------
imtringued
How do I implement Datalog in my application? Crickets...

~~~
vchak1
Depending on the stack, you can use some open source libraries. Datascript is
quite robust, and useable from javascript/clojurescript.
[https://github.com/tonsky/datascript](https://github.com/tonsky/datascript)

~~~
tveita
"An immutable in-memory database and Datalog query engine"

Most uses of GraphQL is to expose existing data from a relational database
through an API, so although this looks like an interesting project, is doesn't
look like it would be a contender against any GraphQL implementation.

I'd argue there are two faulty assumptions in the OP: 1. "GraphQL serves the
same use case as Datalog.", and 2. "Datalog is better at that use case."

GraphQL is not a general purpose query language! There are practically no
query operations at all built into the specification - it's all about field
selection and parameter passing. At heart it's more like a RPC protocol geared
towards requesting tree-like data. You would evaluate GraphQL against options
like plain REST, OpenAPI, gRPC, not against Datatalog.

A more relevant question to ask is why Datalog never took over for _SQL_ , but
it would be harder to pretend that SQLs dominance is due to it being the
current passing fad.

------
macca321
I've got this idea that you can chuck JMESPath into the range header on a
standard json endpoint for ghetto graphql

------
scotty79
For me the most important thing about graphql is composability.

Child component can communicate their data requirements to the parent and
parent can include them in the query and feed the data to the children as it
arrives.

Don't know if Datalog has anything for that.

