Sure it can, and I’ve operated MySQL (Percona) at large scale for a social media company. You shard requests by user or something else, doesn’t matter if you have 50 DBs or 50,000. However in most cases you have to write the sharding mechanism yourself, and understand your workload and what such a system can and cannot do.
Given perfect knowledge of access patterns, I bet you could. Especially since it's basically all reading and not writing. Horizontally scaled with many, many read-only replicas. But then there are lies, damn lies, and benchmarks. All the big companies running huge Oracle db installations are busy running their workloads, which probably don't look like Amazon Prime day traffic.
It's also impossible to have perfect knowledge of access patterns.
Assuming you meant Redux Toolkit (RTK) Query there.
FWIW, the stance of both the Redux maintainers (myself and Lenz) and the React/TanStack Query maintainers (Tanner and Dominik) is:
- If you're using Redux at all, prefer using RTK Query
- If you're using just plain React, prefer using React Query
but don't write data fetching and caching logic by hand :)
There _are_ some API and philosophy differences between RQ and RTKQ. RTKQ emphasizes defining "endpoints" up front, and generates React hooks based on those endpoint definitions. That approach also helps enable codegen from OpenAPI and GraphQL schemas. RTKQ also has some unique capabilities around cache lifecycles that enable use cases like streaming updates:
(I'll also note that we have an `<ApiProvider>` that automatically sets up a Redux store for you internally, so you _can_ use RTKQ if you're not already using Redux in an app.)
I think it also depends on the system you’re using it on. I think one of the biggest advantages of DDB is that is scales so well (with good design to avoid hot partitions). Afaik, RDBMS simply cannot scale in the same way due to their design. Yes, they can scale somewhat, but as you said it requires lots of tuning, and you’ll still reach a hardish limit.
One partition of DDB is incredibly tiny compared to one partition of an RDBMS. You can push that one partition of RDBMS pretty far before you're forced to design sharding into your system. With DDB you are basically forced to design sharding into your partition keys up front or you will have hot partition issues. This is by far the most common problem I see with teams using DDB, so brushing it off as "with good design to avoid hot partitions" is understating the scope of the problem.
All databases scale the same way - by partitioning and sharding the dataspace. RDBMS have harder restrictions due to the features they provide and the performance expectations, but you can just as easily use a bunch of relational servers to partition a table (or several) across them by range or hashes of the primary key.
That's basically what key/value stores like DynamoDB do, and why DynamoDB was even built on MySQL (at least originally).
"can just as easily use a bunch of relational servers to partition a table" is not true at all. Managing, maintaining and tuning a sharded relational cluster is an astonishing amount of operations work. partition management, re-partioning, partition failover / promotions / demotions, query routing, shard discovery, upgrades... it goes on an on. All this work is gone if you pick dynamo.
Not saying that dynamo is always better, but IMHO people very much underestimate the ops cost of running a sharded relational cluster at scale.
I think it’s more like cross-world latency. Where going from one side of the world to the other will be slower going via fibre optic vs through space as with starlink? I think this is reliant on the satellite-satellite latency being low and the ground to satellite being not insanely high.
A switch exists for this, however they’re quite expensive and had some financial issues in the past (although they seem to be under new management now)? https://dendevices.com/products/switch/
If your server has full disk encryption it should be relatively safe against attacks where they just take the device, and so whatever you use to sync should be safe too?
