
In memory Key/Value store in go using gRPC - knrt
https://github.com/knrt10/gRPC-cache
======
jeffbee
Live design review: never use type “string” for keys and values in protocol
buffers. Always use “bytes”.

~~~
kodablah
> never use type “string” for keys and values in protocol buffers. Always use
> “bytes”.

What are "keys and values" with regard to protobuf? Surely you don't mean maps
as bytes can't be "keys" anyways. And why shouldn't you use string as a field
type?

~~~
atombender
I think they're referring to this app, which uses Protobuf strings to
represent its keys and values.

------
malkia
I'm more interested in DB's that store protocol buffers, and can expose their
innards (like spanner, bigtable, etc.). What's good to use/try on the outside?

~~~
jeffbee
> (like spanner, bigtable, etc.)

Neither Spanner nor Bigtable has this feature. If you store a protobuf in a
Bigtable column, that's your own business. Bigtable itself won't notice.

~~~
jvolkman
Internal Spanner does support protobuf columns.

~~~
jeffbee
That's neat. Just it just blow it up into a struct?

~~~
jvolkman
I'm not sure what the internal representation is. Spanner is aware of the
proto type through some mixture of DDL and build rules. Updates to proto types
get pushed when updated schemas are pushed. You can use deeply-nested fields
in various query clauses, create indexes, etc. But I believe updates must
replace the whole object. Much like JSONB columns in other RDBMS.

This is from memory. I'm not at Google anymore.

~~~
malkia
My memory too ;) - Left middle of 2017, here is some article that mentions it
- [https://www.nextplatform.com/2019/01/15/spanning-the-
databas...](https://www.nextplatform.com/2019/01/15/spanning-the-database-
world-with-google/)

also some bare mention here -
[https://static.googleusercontent.com/media/research.google.c...](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/46103.pdf)

"... The effort to standardize our own implementations both covers gaps in the
SQL standard where details are unclear or left to the implementation and
covers our own Google specific extensions. Before defining our own SQL
dialect, we did consider other options. Using the front end of an open-source
SQL processing system, such as Postgres, would have accomplished several of
the same goals including filling the gaps left by the SQL standard and
ensuring uniform syntax/semantics across our systems. We ruled out this
approach because internal systems at Google make heavy use of Protocol Buffers
in their applications, and we wanted Protocol Buffer message and enum types to
be first class types in our SQL language"

------
malkia
Internally, google had best-practices (but could be outdated), that each RPC
service function should take unique input message, and unique output message,
and never reuse one, even if empty, or having same structure, at the cost of
more writing.

Later this allows one to change things independently.

~~~
jeffbee
Wait but what about EmptyMessage and Any? Yes they're both terrible choices
for any circumstance, but also widely seen.

~~~
malkia
I dunno. I'm simply recalling what I remember as best-practices by memory,
could've been wrong about some of them, but for my own protobufs that I use I
keep the same practice.

~~~
jrockway
No, you're right. EmptyMessage exists but you shouldn't make it an input or
output to an RPC if that RPC might ever in the future accept or return
arguments. Since people are super bad about predicting the future, just call
it WhateverRequest and WhateverReply and if there are no arguments today,
don't put any fields in those types.

I would also not use Any as a request or reply type, but Any, in general, is
fairly common in practical use.

------
johnbellone
I have been poking around building an eerily similar example[1] service. I am
trying to come up with a good pattern, using go-micro, where a team can
quickly and easily leverage the protobuf IDL for gRPC/REST interfaces across
large teams. It sounds like it should be easy, right?

The tooling around pulling in protobuf at compile time is a huge pain in the
ass. I've been spending an hour or so every night over past few weeks
searching issues to figure out why things aren't working. This is a great
reference point for me, thanks!

[1]: [https://github.com/johnbellone/cache-
service](https://github.com/johnbellone/cache-service)

------
cachestash
I am hearing more and more about gPRC lately, can anyone give me a run down on
why its useful and what it solves?

~~~
mav3rick
It allows you to execute code on another machine. Your code makes a function
call that returns a status / error, except the function runs on another node.
It's another RPC framework and there are more caveats like what guarantees a
framework makes. Did the function execute exactly once, at most once or at
least once.

------
danaur
We have tons of in memory KV stores, I assume this is implemented for fun? If
not advertise the killer feature better

~~~
palash25
this was implemented as a take home assignment I guess

