Only ones I know that follow an architecture similar to ScyllaDB are aerospike (talks directly to storage) and voltdb(partitioned on cores), while most other projects are in golang/java so I think the answer is not enough.
Also, there are many posts that show that dedicated servers have much better latency/performance compared to vps, but ~everyone still follows the ec2 bandwagon.
Things that I've used in sqlalchemy that(by a quick search) aren't in django (as far as I know):
composite primary keys
complex indexes (functional etc)
arrays + json(these are in django, but I think only recently and before that in contrib)
server side cursors
the non-orm part(the lower layer)
sqlalchemy-alembic (also in django, but only recently I think, still probably less features)
server_default (define a default value for a column that will be applied to the sql-schema and not just model.field)
more customization to the lower level db driver(psycopg2, maybe this is also supported in django)
Use the models + library outside of your web app (ex: in several non-request-serving processes )
There are alot more features that I haven't used/don't know/didn't need.
Have you tried doing remote freelance work ? It can get pretty lucrative. Compared to local wages, in my country(Albania) a good wage is 500$/month. What I did is that I took a small-hastle full-time job and in the meantime I tried to build a remote-freelance career. Went from 2$-20$/hour (full time) job in 1 year. (i found jobs on odesk, the beginning was very hard but after you build a profile it gets easier).
Wouldn't the classic sharding approach at that point simply be to use a second instance? I mean, it'd be great if they handled that for you but I'd tend to assume most apps would have hit the wall on the approach of shoving everything into a single database long before maxing out a 32-core/244GB system with 64TB on SSD.
> [meme]You can't simply use a second machine/instance.
I'm assuming [meme] is shorthand “overly-broad assertion”? It's nice if your database has horizontal scaling built in but it's not like we don't have an entire generation of successful companies who had application-level sharding logic either by necessity or because they found the control it offered was valuable compared to the built-in generic logic.
> While it's true that most companies will be fine in 256GB ram, we were talking about sharding, which it doesn't have.
You still haven't supported the assertion that it's common for places to have massive, heavily-queried databases like this which would not be better split following natural application-level boundaries. This is particularly relevant when discussing AWS as some of the common reasons for keeping everything in one database are sweet-spots for other services (e.g. migrating data for long-term historical reporting over to RedShift).
Again, I'm not questioning that integrated sharding would have its uses – only your sweeping assertion that this is a likely problem for most people and that it's a dead-end (“you're stuck”) rather than merely one of many growing pains which you'll deal with on a successful product. In particular, it's unlikely that everyone will have the same right answer at that scale since access patterns vary widely.