Googler opinions are my own. I'm not a spanner expert, but I use F1 at least a little.
If you start having global spanner replicas, don't you start getting into read safe times? According to the GCP docs, a given node may not know if it has the latest data, as global replicas could diverge by up to 10 seconds ( assuming I'm reading the docs right).
I am little sad that Google never made chubby a service on GCP. That is underlying locking system that many systems that Google use.
> I am little sad that Google never made chubby a service on GCP. That is underlying locking system that many systems that Google use.
I think one of the issues with this is that its far easier to use chubby incorrectly than to use it correctly. Like, you can use chubby to build really, really cool and powerful tools (https://www.usenix.org/system/files/conference/osdi16/osdi16...), but I wouldn't be surprised if most users misused or overused it, and that would likely be magnified on a public version.
From what I remember, Chubby was one of those services that was a bit more sensitive to poor client behavior. You could cause problems for other users with poor Chubby behavior, and you could accidentally end up with code that causes a high Chubby load.
Chubby is a lot like etcd except without the hindsight. Distributed locking is pretty complicated and if I remember correctly there was a great podcast from Google's Kube podcast about distributed locking that goes into a lot of this. [0]
If you are going to do any distributed locking I strongly recommend listening to it. Has a lot of pain points discussed.
On the other hand Spanner must have a perfectly fine distributed lock manager on its own. It seems odd to implement a lock manager on another lock manager (which just happens to be a database too) just because you can't directly access it. I'm sure that must cause unnecessary overhead.
It's a bit like those odd nesting cases that happen in networking, like PPPoE over Ethernet. Maybe that's the eventual fate of computing as it evolves, more and more layers everywhere, that are more and more redundant but still exist for some odd reason, with low level facilities being reinvented.
If you start having global spanner replicas, don't you start getting into read safe times? According to the GCP docs, a given node may not know if it has the latest data, as global replicas could diverge by up to 10 seconds ( assuming I'm reading the docs right).
I am little sad that Google never made chubby a service on GCP. That is underlying locking system that many systems that Google use.
Chubby: https://research.google/pubs/pub27897/
Spanner read write docs: https://cloud.google.com/spanner/docs/whitepapers/life-of-re...