
How and why GraphQL will influence the Sourcehut alpha - michaelanckaert
https://sourcehut.org/blog/2020-06-10-how-graphql-will-shape-the-alpha/
======
WhatIsDukkha
I don't understand the attraction to Graphql. (I do understand it if maybe you
actually want the things that gRPC or Thrift etc gives you)

It seems like exactly the ORM solution/problem but even more abstract and less
under control since it pushes the orm out to browser clients and the frontend
devs.

ORM suffer from being at beyond arms length from the query analyzer in the
database server.

[https://en.wikipedia.org/wiki/Query_optimization](https://en.wikipedia.org/wiki/Query_optimization)

A query optimizer that's been tuned over decades by pretty serious people.

Bad queries, overfetching, sudden performance cliffs everywhere.

Graphql actually adds another query language on top of the normal orm problem.
(Maybe the answer is that graphql is so simple by design that it has no dark
corners but that seems like a matter of mathematical proof that I haven't seen
alluded to).

Why is graphql not going to have exactly this problem as we see people
actually start to work seriously with it?

Four or five implementations in javascript, haskell and now go. From what I
could see none of them were mentioning query optimization as an aspiration.

~~~
dmitriid
> I don't understand the attraction to Graphql.

It's attractive primarily to frontend developers. Instead of juggling various
APIs (oftne poorly designed or underdesigned due to conflicting requirements
and time constraints) you have a single entry into the system with almost any
view of the data you want.

Almost no one ever talks about what a nightmare it becomes on the server-side,
and how inane the implementations are. And how you have to re-do so many
things from scratch, inefficiently, because you really have no control of the
queries coming into the system.

My takeaway from GraphQL so far has been:

\- good for frontend

\- usable only for internal projects where you have full control of who has
access to your system, and can't bring it down because you forgot an
authorisation on a field somewhere or a protection against unlimited nested
queries.

~~~
rhlsthrm
As a full-stack dev, I'm going to always reach for things like Hasura for
building my backend from now on. It auto generates a full CRUD GraphQL API
from my Postgres DB schema. Most of the backend boilerplate is eliminated this
way. If I need to add additional business logic, I can use serverless
functions that run between the GraphQL query from the front end and the DB
operations (actions in Hasura). Most of the heavy lifting is through the front
end anyways, and this keeps everything neatly in sync.

~~~
dgellow
What about authentication, authorization?

Also, how do you handle transactional logic?

~~~
rhlsthrm
They have great recipes for authentication/authorization. It's much better IMO
because it actually provides per-request authorization. There is also great
support for transactions using the GraphQL mutations. I'm not affiliated with
Hasura in any way, it's just changed the way I view backend development.
Backends (in most cases, my day job is actually not part of this
generalization) should basically be a thin wrapper around your database, and
any work you can outsource to the database, you should do that rather than
building business logic.

------
jorams
In my experience GraphQL can be much nicer to implement than REST, and it
offers a good structure around things that many REST APIs implement in
particular ways (like selecting which fields you want). The pain you'll
experience depends heavily on your data model and the abuse potential that
brings.

I think the biggest problem with GraphQL is the JavaScript ecosystem around
it, and all of its implicit context. It seems to be built entirely on specific
servers and clients, instead of on the general concepts.

Relay[1], a popular client-side library, adds all kinds of requirements in
addition to the use of GraphQL. One of those is that until version 8, it
required all mutation inputs and outputs to contain a "clientMutationId",
which had to be round-tripped. It was an obvious hack for some client-side
problem which added requirements to the backend. Somehow it had a
specification written for it instead of being fixed before release. This hack
is now in public APIs, like every single mutation in the GitHub API v4.

GraphQL also includes "subscriptions", which are described incredibly vaguely
and frankly underspecified. There are all kinds of libraries and frameworks
that "support subscriptions", but in practice they mean they just support the
websocket transport[2] created by Apollo GraphQL.

If you just use it as a way to implement a well-structured API, and use the
simplest tools possible to get you there, it's a pleasure to work with.

[1]: [https://relay.dev/](https://relay.dev/)

[2]: [https://github.com/apollographql/subscriptions-transport-
ws](https://github.com/apollographql/subscriptions-transport-ws)

~~~
kasbah
I don't think Relay is used that much outside of Facebook. The community seems
to have settled on Apollo. Personally I find Apollo over-engineered. When I
couldn't delete things from the cache, because of a bug, and was faced with
digging into the complex code-base, I ended up just using straight JSON with a
HTTP client/fetch and caching in a simple JS object.

Other users of my API [1] just use straight HTTP with JSON as well. GraphQL
clients seem to solve something we are not encountering. If urql [2] or gqless
[3] work well when I try them I'd be up for changing my mind though.

[1]:
[https://github.com/kitspace/partinfo](https://github.com/kitspace/partinfo)

[2]:
[https://github.com/FormidableLabs/urql](https://github.com/FormidableLabs/urql)

[3]: [https://gqless.dev/](https://gqless.dev/)

~~~
andrewingram
Relay has a PR problem and i'm not 100% happy with it, but I've been using it
since 2015 (never worked at Facebook) and would still choose it over Apollo.
Main reason is that Apollo is geared towards getting up and running easier,
but this leads to a worse overall experience beyond that point.

------
tannhaeuser
Tbh I'd expected a little better than framing this question in a "REST vs
GraphQL" discussion coming from sourcehut.org. If you control your backend,
you can aggregate whatever payloads you please into a single HTTP response,
and don't have to subscribe to a (naive) "RESTful" way where you have network
roundtrips for every single "resource", a practice criticized by Roy Fielding
(who coined the term "REST") himself and rooted in a mindset I'd call based
more on cultural beliefs rather than engineering. That said, a recent
discussion [1] convinced me there are practical benefits in using GraphQL if
you're working with "modern" SPA frameworks, and your backend team can't
always deliver the ever-changing interfaces you need so you're using a
backend-for-fronted (an extra fronted-facing backend that wraps your actual
backend) approach anyway, though it could be argued that organizational issues
play a larger role here.

[1]:
[https://news.ycombinator.com/item?id=23119810](https://news.ycombinator.com/item?id=23119810)

~~~
zapf
If you don't like REST, don't use it.

Whatever you do, don't even think that GraphQL will solve your problems. You
were on the right track staying away from it till now.

I can't also advise enough to stay away from a typed language (Go in this
case) serving data in a different typed language (gql). You will eventually be
pulling your hair out jumping through hoops matching types.

After my last web project that require gql and go, I did some digging around,
thinking, there has to be a better alternative to this. I have worked with
jQuery, React, GraphQL.

My conclusion was that next time I will stick to turbolinks
([https://github.com/turbolinks/turbolinks](https://github.com/turbolinks/turbolinks))
and try stimulus ([https://stimulusjs.org/](https://stimulusjs.org/)).

~~~
square_usual
> stimulus ([https://stimulusjs.org/](https://stimulusjs.org/)).

And here I thought Basecamp was still 100% rails. Interesting to see that
they're also developing backend JS frameworks.

~~~
odensc
Stimulus is a frontend framework.

------
say_it_as_it_is
Where are the GraphQL lessons learned? The author hasn't even implemented a
solution with it yet, but that hasn't stopped him from declaring it to the
world. I don't find an announcement useful.

Maybe GraphQL adopters aren't sharing their experiences with it in production
because they're realizing its faults? People are quick to announce successes
and very reluctant to own, let alone share, costly mistakes. Also, people
change jobs so often that those who influence a roll-out won't even be around
long enough for the post-mortem. GraphQL publicity is consequently positively
biased. If the HN community were to follow up with posters who announced their
use of GraphQL the last two years, maybe we can find out how things are going?

~~~
gbear605
The reason this post was written was for users of Sourcehut, especially for
people writing to its API. This post isn’t particularly relevant or
explanatory for other audiences, but I don’t think it’s supposed to be so
that’s fine.

~~~
say_it_as_it_is
I understand what you mean. That makes sense.

------
tleb_
I think this article misses the explanation of why GraphQL over REST. I
usually don't like "x versus y" articles but here both have been tested on
SourceHut, the hindsight should probably appear as useful.

Thanks Drew and others for SourceHut.

------
ianamartin
Well, that's too bad. I always thought this was a cool project. But if you
can't dev your way into decent performance for a small alpha project using
python/flask/sql, I don't think your tools are the problem. And I guarantee
that a graphql isn't the solution.

So, I mean, good luck.

~~~
searchableguy
I didn't read the post that way. I feel scaling is a small issue more so
organization. Graphql does make sense for something like source hunt.

Why not use type hints in python? Isn't that a good enough substitute?

I wonder why go instead of rust if he wanted static typing, long term ease of
maintanence and performance. Go's type system is not great especially for
something like graphql. Gqlgen relies heavily on code generation. Last time I
used it, I ran into so many issues. I ditched go together after several
painful clashes with it that community always responded with: oh you don't
need this.

(yeah except they implemented all those parts in arguably worse ways and
ditched community solutions in the next few years)

One major benefit the GP fails to mention is that with graphql, it is easy to
generate types for frontend. This makes your frontend far more sane. It's also
way easier to test graphql since there are tools to automatically generate
queries for performance testing unlike rest.

There is no need to add something for docs or interactivity like swagger.

~~~
erk__
As for the reason to use Go instead of Rust it is probably just down to the
creator of Sourcehut he has multiple times expressed that he dislikes rust
quite a bit.

He has written a blog post about how he chooses programming languages as well
[https://drewdevault.com/2019/09/08/Enough-to-
decide.html](https://drewdevault.com/2019/09/08/Enough-to-decide.html)

~~~
Kuinox
It's the first time I hear that haskell has awful package management...

~~~
Scarbutt
I hear the opposite, it's so awful they have to use Nix to keep it sane.

~~~
tome
Interesting. Where did you hear that?

------
leadingthenet
Might I suggest taking a look at FastAPI?

It’s been a 10x+ improvement on Flask, in my experience.

------
AmericanChopper
GraphQL as a query language is simply better than REST in most cases imo. REST
has too much client side state, which not only has the potential to make
things harder for clients to consume, but also has all the inconsistent states
to handle where your consumer gets part way through a multiple-REST method
workflow, and then bails. REST also absolutely sucks for mutating arrays.

Really I just look at GraphQL as a nice RPC framework. The graph theory
operations like field level resolvers are mostly useless. But if you treat
each relationship as a node rather than each field, you can get it to work
very nicely with a normalized data set. I haven’t found it hard to preserve
join efficiency in the backend either, and it so far hasn’t forced me into
redundant query operations.

Just as long as you don’t use appsync. Really, don’t even bother.

~~~
GordonS
> GraphQL as a query language is simply better than REST in most cases imo.
> REST has too much client side state, which not only has the potential to
> make things harder for clients to consume, but also has all the inconsistent
> states to handle where your consumer gets part way through a multiple-REST
> method workflow, and then bails.

How much client state you maintain seems to me to be orthogonal to
GraphQL/REST.

Take your example or a multiple-REST workflow. I presume your point was that
the workflow could be implemented by a single GraphQL query/mutation/whatever
- but just the same, you can put as much code and logic as you like behind a
REST call?

~~~
AmericanChopper
You could do that, but if you start creating endpoints for transactions rather
than method -> resource endpoints, then you’re not really making a REST
interface anymore. But even ignoring REST purity, I’d argue that GraphQL is
better suited to that design pattern in general.

~~~
GordonS
When most people refer to REST APIs, they really mean "HTTP APIs". I really
don't think we should reach for GraphQL just because of the ideological notion
of daring to not adhere 100% to REST.

~~~
AmericanChopper
If you’re going to abandon REST design principles, then you’ll eventually end
up with something that pretty much does what GraphQL does. At that stage why
not just adopt a more fit-for-purpose set of design principles?

Keep in mind that I’m only really advocating for it as a query language for
HTTP APIs (which as a side benefit has some nice existing tooling which you
may or may not find useful).

------
mehdix
> With these, you can deploy a SourceHut instance with no frontend at all,
> using the GraphQL APIs exclusively.

This reminds me of Kubernetes' design. You have an API server which is
practially the Kubernetes from user's perspective. `kubectl` is just one out
of possibily many clients that talk to this API.

Edit: typos.

------
awinter-py
> The value-add is difficult to understand

yup

------
orf
The author says that he has soured on Python for “serious, large projects”.
While it’s clearly personal opinion, and that’s fair enough , I can’t help but
think his choice of framework hasn’t helped him and has likely caused
significant slowdown when delivering features.

Looking through some of the code for Sourcehut, there’s an insane amount of
boilerplate or otherwise redundant code[1]. The shared code library is a mini-
framework, with custom email and validation components[2][3]. In the ‘main’
project we can see the views that power mailing lists and projects[4][5].

I’m totally biased, but I can’t help but think “why Flask, and why not Django”
after seeing all of this. Most of the repeated view boilerplate would have
gone ([1] could be like 20 lines), the author could have used Django rest
framework to get a quality API with not much work (rather than building it
yourself[6]) and the pluggable apps at the core of Django seem a perfect fit.

I see this all the time with flasks projects. They start off small and light,
and as long as they stay that way then Flask is a great choice. But they often
don’t, and as the grow in complexity you end up re-inventing a framework like
Django but worse whilst getting fatigued by “Python” being bad.

1\.
[https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrh...](https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrht/blueprints/public.py)

2\.
[https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/emai...](https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/email.py)

3\.
[https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/vali...](https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/validation.py)

4\.
[https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/bl...](https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/blueprints/mailing_lists.py)

5\.
[https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/bl...](https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/blueprints/projects.py)

6\.
[https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrh...](https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrht/blueprints/api/pastes.py)

~~~
StavrosK
Exactly agreed. I basically only use Flask for things I want to explicitly be
single-file these days. For anything larger, I reach for Django, because I
know that if I need at least one thing from it (and I always need the
ORM/migrations/admin), it will have been worth it.

My current favorite way of building APIs is this Frankenstein's monster of
Django/FastAPI, which actually works quite well so far:

[https://www.stavros.io/posts/fastapi-with-
django/](https://www.stavros.io/posts/fastapi-with-django/)

FastAPI is a much better way of writing APIs than DRF, I wish it were a Django
library, but hopefully compatibility will improve as Django adds async
support.

~~~
fastball
I built the backend for my knowledge-base platform[0] using Flask originally,
but performance was definitely a struggle so I rewrote the whole thing with
FastAPI. Have definitely seen a serious performance bump from that switch, and
currently am quite happy with it. Many of our users are actually impressed
with how fast everything is on the platform.

I still want to rip out SQLAlchemy ORM and replace it with pure SQL via
`asyncpg`, as SQLAlchemy ORM is not async and that causes a bunch of extra
switching in the backend that certainly doesn't help eek out more perf, but at
the moment it's a bit too much effort and users are happy.

Scaling is handled by just throwing more instances of the application at the
problem, behind a load-balancer.

[0] [https://supernotes.app](https://supernotes.app)

~~~
StavrosK
That sounds like a good solution, and is a good data point to know, thank you.
Did you try Sync FastAPI? I'm wondering how its performance compares with
async

~~~
fastball
When you say sync do you just mean having sync endpoints? If so, then yeah,
it's required since we're using SQLAlchemy ORM. Otherwise the calls to SQLA
ORM would block the main event loop. As it is, FastAPI (well really Starlette)
creates threads for sync endpoints to prevent blocking the main thread/event
loop.

So yeah, still seeing good speedups in our own benchmarks even though most of
our endpoints are sync.

What was arguably more important though was how much switching to ASGI helped
with handling WebSockets. We're using SocketIO, and trying to get a
fundamentally async protocol working within sync (Flask) land was a massive
pain. We had repeated reliability and deployment issues that were very hard to
debug. Switching to FastAPI made that much easier.

~~~
StavrosK
Oh, I can imagine, ASGI must be immeasurably easier. Where do the async
speedup gains come from, though, if your database is still sync? Wouldn't
threadpools provide comparable performance before?

~~~
fastball
Well so actually we have both now.

For WebSockets, all of the code is async, so I'm already using `asyncpg` for
any database stuff that is happening there.

With regards to why are the sync endpoints faster, I think it is a number of
things, some of which are userland changes that could've been made under
Flask, but all of which are _somewhat_ related to the switch. With regards to
things that FastAPI itself has changed, I think using a (de)serialization lib
like Pydantic and serializing to JSON by default (which is what we were doing
under Flask anyway, though with Marshmallow) makes a lot of the code paths in
the underlying lib a bit faster, because with Flask there was more "magic"
going on behind the scenes. For userland stuff, I think partly because there
is less magic going in the background (I really like FastAPIs dependency
injection system), it's made it easier to identify the bottlenecks and
optimize hot code paths.

~~~
StavrosK
That makes perfect sense, thank you. I love FastAPI just for the code clarity
and ease of working with better type objects (the Pydantic classes) alone,
though the speed benefit is nice to have too.

------
hn_throwaway_99
Based on the top comments in this thread, I can see my previous attempts to
dispel mistaken beliefs around graphql have failed, but I'll still try!

1\. The biggest mistake GraphQL made was putting 'QL' in the name so people
think it's a query language comparable to SQL. It's not:
[https://news.ycombinator.com/item?id=23120997](https://news.ycombinator.com/item?id=23120997)

2\. Some benefits of GraphQL over REST:
[https://news.ycombinator.com/item?id=23124862](https://news.ycombinator.com/item?id=23124862)

~~~
detaro
I suspect as long as [https://graphql.org](https://graphql.org) says what it
does, you're going to have a hard time with that fight...

------
FpUser
I've been using home grown RPC for my servers (POST + JSON) for HTTP or binary
serialization for lower level access. Works like a charm for years. Never felt
like missing anything.

------
iooi
I wanted to learn GraphQL recently and I wrote a small library to
automagically generate GraphQL schemas from SQLAlchemy models. [1]

It's inspired by Hasura, the schema is almost the same. It's not optimized at
all, but it's a nice way to quickly get started with GraphQL and expose your
existing models.

[1] [https://github.com/gzzo/graphql-
sqlalchemy](https://github.com/gzzo/graphql-sqlalchemy)

------
cletus
So I've now had the opportunity to use both GraphQL and protocol buffers
("protobufs" is the more typical term) professionally and I have some thoughts
on this.

1\. Protobufs use integer IDs for fields. GraphQL uses string names. IMHO this
is a clear win for protobufs. Changing the name of a field of GraphQL is
essentially impossible. Once a name is there it's there forever (eg mobile
client versions are out there forever) so you're going to have to return null
from it and create a new one. In protobufs, the name you see in code is
nothing more than the client's bindings. Get a copy of the .proto file, change
a name (but not the ID number) and recompile and everything will work. The
wire format is the same;

2\. People who talk about auto-generating GraphQL wrappers for Postgres
database schemas (not the author of this post, to be clear, but it's common
enough) are missing the point entirely. The whole point of GraphQL is to span
heterogeneous and independent data sources;

3\. Protobuf's notions of required vs optional fields was a design mistake
that's now impossible to rectify without breaking changes. Maybe protobuf
v3/gRPC did this. I'm honestly not sure.

4\. Protobuf is just a wire format plus a way of generating language bindings
for it. There are RPC extensions for this (Stubby internally at Google; gRPC
externally and no they're not the same thing). GraphQL is a query language. I
do think it's better than protobufs in this regard;

5\. GraphQL fragments are one of these things that are probably a net positive
but they aren't as good as they might appear. You will find in any large
codebase that there are key fragments that if you change in any way you'll
generate a massive recompile across hundreds or thousands of callsites. And if
just one caller uses one of the fields in that fragment, you can't remove it;

6\. GraphQL does kind of support union types (eg foo as Bar1, foo as Bar2) but
it's awkward and my understanding is the mobile code generated is... less than
ideal. Still, it's better than not having it. The protobuf equivalent is to
have many optional submessages and there's no way to express that only one of
them will be popualated;

7\. Under the hood I believe the GraphQL query is stored on the server and
identified by ID but the bindings for it are baked into the client. Perhaps
this is just how FB uses it? It always struck me as somewhat awkward. Perhaps
certain GraphQL queries are particularly large? I never bothered to look into
the reason for this but given that the bindings are baked into the code it
doesn't seem to gain you much;

8\. GraphQL usage in Facebook is pervasive and it has first class support in
iOS, Android and React. This is in stark contrast to protobufs where protobuf
v2 in Google is probably there forever and protobuf v3/gRPC is largely for the
outsiders. It's been several years now since I worked at Google but I would be
shocked if this had changed or there was even an intention of changing it at
this point;

9\. The fact that you can do a GraphQL mutation and declare what fields are
returned is, IMHO, very nice. It saves really awkward create/update then re-
query hops.

10\. This is probably a problem only for Google internally but another factor
on top of protobuf version was the API version. Build artifacts were declared
with this API version, which was actually a huge headache if you wanted to
bring in dependencies, some of which were Java APIv1 and others Java APIv2. I
don't really understand why you had to make this kind of decision in creating
build artifacts. Again, maybe this has improved. I would be surprised however.

Lastly, as for Sourcehut, I had a look at their home page. I'm honestly still
not exactly sure what they are or what value they create. There are 3 pricing
plans that provide access to all features so I'd have to dig in to find the
difference (hint: I didn't). So it's hard for me to say if GraphQL is an
appropriate choice for them. At least their pages loaded fast. That's a good
sign.

~~~
square_usual
> People who talk about auto-generating GraphQL wrappers for Postgres database
> schemas (not the author of this post, to be clear, but it's common enough)
> are missing the point entirely. The whole point of GraphQL is to span
> heterogeneous and independent data sources;

I don't think they are missing a point; rather, they have a completely
different point: eschew a backend and use GraphQL on a DB + a frontend that
gets all that data. If you're developing rapidly and don't have complex
backend logic, I can see why you'd want to do that.

~~~
ramzeus
I think the problem with this approach becomes apparent when the database
model changes, suddenly the API doesn't match the model anymore or the API
needs to change. Both can be difficult problems leading to braking changes in
the client(s). Of course, for some rapid testing and simple personal projects
this doesn't matter so much but for most other projects I think this might
bite you hard in the future.

------
crabmusket
> Another (potential) advantage of GraphQL is the ability to compose many
> different APIs into a single, federated GraphQL schema.

If anyone else can share experiences of this sort of problems and solution,
I'd be really interested to hear it. I've written non-GQL APIs before that
back onto other internal and external services; what am I missing?

~~~
maddyboo
I think Drew is referring to composition in terms of API end-user code, not
sr.ht code, e.g. making it possible for the user write a single GQL query that
combines data from multiple sr.ht services.

~~~
crabmusket
Ah, I think that makes a little more sense. Thanks!

------
camgunz
A lot of places have a hell of a time dealing with very nested graphql queries
that effectively DoS your app servers, or a resolver that’s very slow. Caching
is also an open question. But for the simplest systems, I’d hesitate to
recommend graphql at this point.

------
ddevault
Author here. Wow, there is a ton of "didn't RTFA" comments here. It seems like
half of this thread saw GraphQL in the title, it knocked two gears into place
in their head, and they started writing up their own little essay about how
bad it is.

I evaluated GraphQL twice before, and discarded it for many of the reasons
brought up here. Even this time around, I give a rather lackluster review of
it and mention that there are still many caveats. It's not magic, and I'm not
entirely in love with it - but it's better than REST.

Query optimization, scalability, authentication, and many other issues raised
here were part of the research effort and I would not have moved forward with
it if I did not feel that they were adequately addressed.

Before assuming some limitation you have had with it in the past applies to
sr.ht, I would recommend reading through the code:

[https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/api](https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/api)

[https://git.sr.ht/~sircmpwn/gql.sr.ht](https://git.sr.ht/~sircmpwn/gql.sr.ht)

If you're curious for a more detailed run-down of my problems with REST,
Python, Flask, and SQLAlchemy, I answered similar comments last night on the
Lobsters thread:

[https://lobste.rs/s/me5emr/how_why_graphql_will_influence_so...](https://lobste.rs/s/me5emr/how_why_graphql_will_influence_sourcehut)

I would also like to point out that the last time I thought a rewrite was in
order, we got wlroots, which is now the most successful project in its class.

Cheers.

------
hpen
I tried GraphQL but my resolvers became extremely complicated. Am I missing
something or is this just how it goes?

------
geitir
One thing I've noticed is it seems redux and graphql are solving the same
problem in an entirely different way

~~~
iso-8859-1
Redux brands itself as a tool for managing state. How does GraphQL manage
state?

~~~
geitir
The way I see it is Redux is a local normalized store which can then be used
via the selector pattern to generate custom objects / views. With GraphQL you
are requesting that derived data to be generated for you via the query
language.

Regarding managing state I don't see GraphQL helping with that at all.

------
pknopf
I firmly believe GraphQL is a fad. It will never work, exposing the ORM to
your clients.

I'm curious, why don't we see more public-facing APIs using gRPC?

~~~
zimpenfish
Currently working on a public-facing gRPC API and it's much more faffy than a
REST API. Browsers don't talk gRPC which means you need to support gRPC-web
endpoints. You probably need a gRPC aware proxy too and most of those have
their own challenges. Some stuff doesn't like protobufs which means you need
JSON transport which means protoc plugins. etc.etc. It's all just more work
for (IMHO) little gain.

------
tmpz22
Big OOF. Even the flow of the article is muddied to the point where it takes
significant effort to dig out the main points for OP's migration from REST to
GraphQL. What I got from it:

> The system would become more stable with the benefit of static typing, and
> more scalable with a faster and lighter-weight implementation.

OK static typing I'm with you so far. Faster and lighter weight? I'm not so
sure, sounds like you're having troubles with Flask and SQLAlchemy completely
unrelated to REST. All _production_ graphQL implementations I've seen are very
heavy when they add in authentication and more advanced query capabilities. Is
this REALLY so superior to REST?

> Another (potential) advantage of GraphQL is the ability to compose many
> different APIs into a single, federated GraphQL schema.

I guess the discoverability of GraphQL is better, but 90% of APIs on the
internet prove that large REST APIs are very effective and achieve the same
thing.

> I also mentioned earlier that I am unsatisfied with the
> Python/Flask/SQLAlchemy design that underlies most of SourceHut’s
> implementation. The performance characteristics of this design are rather
> poor, and I have limited options for improvement. The reliability is also
> not something I am especially confident in.

This is where you completely lose me. It's fine if you hate ORMs, its fine if
you hate the SQLAlchemy API, but you're blaming your hammer for the fact that
you think you built a shoddy house. Going out and buying a hammer won't fix
the fact that you're lining up your nails all wrong.

> The GraphQL services are completely standalone, and it is possible to deploy
> them independently of the web application...

> ...it is my intention to build out experimental replacement backends which
> are routed through GraphQL instead.

I think these two captions go together, are you describing microservices? This
can be achieved just fine with REST using simple load balancing strategies
based on url routing or similar.

> Almost all of SourceHut today is built with Python, Flask, and SQLAlchemy,
> which is great for quickly building a working prototype. This has been an
> effective approach to building a “good” service and understanding the
> constraints of the problem space. However, it’s become clear to me that this
> approach isn’t going to cut it in the long term, where the goal is not just
> “good”, but “excellent”

This is a classic example of using a handful of annoying issues to justify an
exciting large re-write that doesn't actually address the main issues you are
having. If you are struggling with the SQLAlchemy library you will find
alternative (and perhaps larger) struggles in all GraphQL implementations.
Best of luck, this is a road I would not follow you on. Seriously though I
wish you the best and hope your product succeeds despite this.

~~~
ianamartin
This is a better way of saying what I've tried to say in like 3 comments.

------
xvilka
Does this mean that SourceHut will become completely written in Go? Amazing
news if so!

~~~
CameronNemo
Why? I am currently converting a Flask/SQLAlchemy application to Go and gRPC.
Go has not been that much of a win from my perspective. Lots of rough edges
and missing functionality in libraries available, a lackluster type system
that does enough to get in the way but not enough to properly express
programmatic intention, and an atypical non-ideal error handling model have
left me with little reason to champion Go. Not that it is a terrible language,
I just do not see the value add compared to even Python (let alone Kotlin or
Rust).

~~~
zapf
Had a similar experience.

Typed languages are great for systems development, and I think, not so good
for writing web applications.

I also think, Ruby, Python, JS have dominated web dev world largely cause they
don't come in the way of the developer having to constantly convert HTML
(untyped) and JS (untyped)into types for the backend programming language.

Remember how ActiveRecord (not sure about SQLAlchemy) simply took away the
pain of managing types? You didn't have to explicitly parse or cast "0.001"
into 0.01.

~~~
square_usual
ActiveRecord is parsing those types, you just don't do it explicitly. IME, a
dynamic language may not get in your way when you need to go between JS and
the backend, but it also certainly won't get in your way when you're about to
shoot yourself in the foot in a large-ish project with errors that could've
been caught at compile.

------
RcouF1uZ4gsC
> My work on SourceHut has, on the whole, really soured my opinion of Python
> as a serious language for large projects.

A lot of the things thank make Python great for small projects, really bite
you on a large or long-lived project. For me, the two biggest are lack of
types and indentation for scoping. It is really easy to mess up white space
during an edit or refactor. In many languages you would just reformat. In
python, you have to be careful that a statement suddenly did not end up
outside or inside an if block.

~~~
frou_dh
Python has a better static typesystem than Go does:

[https://mypy.readthedocs.io/en/stable/kinds_of_types.html#un...](https://mypy.readthedocs.io/en/stable/kinds_of_types.html#union-
types)

[https://mypy.readthedocs.io/en/stable/kinds_of_types.html#op...](https://mypy.readthedocs.io/en/stable/kinds_of_types.html#optional-
types-and-the-none-type)

[https://mypy.readthedocs.io/en/stable/generics.html](https://mypy.readthedocs.io/en/stable/generics.html)

[https://mypy.readthedocs.io/en/stable/protocols.html](https://mypy.readthedocs.io/en/stable/protocols.html)

~~~
ianamartin
Even though I like Python a lot, I clicked on a reply to this comment because
I was _absolutely certain_ I was going to have to disagree with you. That just
didn't seem possible. But . . . I was wrong. That third link with generics
makes me cry when I think about the go code I've written.

~~~
StavrosK
I'm really liking Python static types to the point where I don't write new
programs without them. Give them a shot, they really help.

~~~
karatestomp
Do most libraries use static types? Or have something widely-used and well-
supported available like the @types project for TypeScript to provide them,
fairly seamlessly, to consumers of those libraries?

~~~
StavrosK
Some projects do, yes. Django, for example, has a library to provide types. In
my experience, types are useful even just in my own application, since the
libraries I use tend to generally only accept simple/fundamental types anyway.
Things could definitely be better when it comes to library type support,
though, I agree.

------
zapf
I like my front end team to not write queries.

I have seen too many front end developers write queries equivalent to "select
* from users, t1, t2, ... tn, left outer join on users.id = t1.user_id... etc
etc etc".

I think that is just bullshit.

I prefer to tell the frontend people - these are the endpoints that backend
team provides, go live with it. If you really have issues, talk to them, and
they might just add a new endpoint for you.

If you let frontend developers dictate backend queries, all you will get is a
big master cluster fuck. I am talking average joe developers that we usually
find to work on startups.

~~~
runawaybottle
I think Frontend has the right to take over certain domains of the backend. A
very clear example of this is templating. Templating used to be a backend
task, but it became very clear it was the frontend developers that needed
power over this domain. Going over to a backend dev to make adjustments to the
templates is an inefficiency that had to be solved, period.

I agree with you that writing optimal queries or data modeling should not be
shifted over to the frontend. With that said, there are basic aspects of this
equation that can be factored out. There should be a layer where frontend can
at the very least pick or mix the data they need. That doesn’t necessarily
mean Graphql, but could also mean a simple middle stack layer where one can do
this basic thing without being able to shoot themselves in the foot.

There’s a responsible way to do this, and most likely will be an ongoing
discussion.

------
z3t4
Im a Sourcehut user and all I care about is that I can host my public
Mercurial repositories, and that web frontend is somewhat usable (it currently
is). I dont care what languages is ised to build it nor do i care about apis
and other features. If i where to chose I would like an api that i could use
with curl, eg. Dont complicate things.

~~~
alexchamberlain
I think this is simply a technical blog post - it could be about any site.
It's simply an insight to the way another engineer is thinking, so you can
challenge your own.

~~~
z3t4
It was not a comment about the blog post! It was just my thought as a user of
the product. He should probably spend his time marketing as he already got a
working product, rather then rewriting stuff.

