
Simple RPC framework in 300 lines of Go - ankuranand
https://github.com/ankur-anand/simple-go-rpc
======
leetrout
The use of a custom transport is interesting... one of the reasons I love Go's
RPC package is because it Just Works(tm) with TLS, TCP, etc etc out of the
box. For anyone just showing up to this: you do not _have_ to create a custom
transport.

Related: Scott Mansfield of Netflix gave a talk at Gophercon 2017 on their
custom serialization format if these things are interesting to you...
[https://github.com/gophercon/2017-talks/blob/master/ScottMan...](https://github.com/gophercon/2017-talks/blob/master/ScottMansfield-
CreatingACustomSerializationFormat/creating-a-custom-serialization-format.pdf)

------
vinay_ys
For a good RPC abstraction, I think a key requirement is to easily and safely
change the deployment separation between modules that use the abstraction –
deployment separations being: in-proc, inter-process-communication and remote-
procedure-call.

That is,

1\. deploy the modules within the same process then the method invocation is
most efficient;

2\. deploy the module across separate OS processes on the same machine then
the method invocation is via the best available OS IPC mechanism;

3\. deploy the module across separate machines then the method invocation is
over network with most efficient remote procedure call implementation.

A long time ago I worked on a project that used Corba (with ACE TAO) that had
facilities like this and it made development, testing environments vs various
production environment permutations a lot easier to build for and manage over
multiple product lifecycles. Now-a-days, these Microservices frameworks are
missing these (basic) features and it overly complicates the deployment
scenarios.

Rather than hand-rolling 300 line custom RPC mechanisms like this one, I would
rather prefer a more complete one that did solve the above problems.

~~~
EdSchouten
Even though I think Go's GRPC implementation is pretty good, this is something
that really annoys me about the way it's designed. The interface of a GRPC
client is not the same as the interface of a GRPC server. This means that:

\- You cannot instantiate a piece of code built on top of a GRPC client
directly on top of another piece of code that implements a GRPC service. You
will need to create a full GRPC client and server. Don't want to let them
communicate over an OS-level socket? You'll have to use something like
[https://godoc.org/google.golang.org/grpc/test/bufconn](https://godoc.org/google.golang.org/grpc/test/bufconn)
to create an in-memory socket.

\- You cannot let a GRPC server process forward requests to a GRPC client
directly. Quite annoying in case you want to multiplex requests, add an
authenticating proxy, etc. etc.

\- Unit testing a GRPC service is annoying, as the service implementation has
to take some concrete GRPC types that can only be instantiated by a GRPC
server. This means that you also have to create a full GRPC
client/server/bufconn to be able to test it.

------
garbre
I haven't studied this extensively but it looks like a great bit of sample
code for someone who wants to see how to do some networking in Go. I echo the
other comments that find the use of a custom transport interesting, since
writing a transport is one of those things that I've always been glad I
haven't had to learn how to do, especially since I'm usually just using HTTP
for which Go is very much batteries-included.

This whole thing raises the question though, is this hard to do in other
languages?

------
grepthisab
This is great, I'm very new to Go and new to RPCs. I went through the code and
had a question someone might be able to answer as I'm still learning Go.

The Server package contains the Register function. Main package implements the
QueryUser function that does the work of querying the DB. When Main calls
Server.Register to register the function with Server, it sends the function
name (QueryUser) and..something else? Is that the memory address of QueryUser
on my computer? And when Server actually runs the function, it's just pointing
to QueryUser at the memory address given to it by Main?

If that's the case, am I correct in thinking that this wouldn't work as
written if the Server package were running on a different physical server,
because it obviously wouldn't be able to access the memory location of
QueryUser on a different machine. So in this case, the Server would need to
implement QueryUser itself on its hardware, but otherwise would work.

Or maybe the use case of RPCs isn't for two servers communicating, but rather
for two different programs on the same machine only? Or maybe what
Server.Register receives is the actual function, not just the memory location
(though I see no evidence of this).

Can someone help enlighten me?

~~~
aksx
>The Server package contains the Register function. Main package implements
the QueryUser function that does the work of querying the DB. When Main calls
Server.Register to register the function with Server, it sends the function
name (QueryUser) and..something else? Is that the memory address of QueryUser
on my computer? And when Server actually runs the function, it's just pointing
to QueryUser at the memory address given to it by Main?

This is correct, let me unpack this a little more.

The 'Register' function tells the server object that when a client tries to
call the function "QueryUser" call the function passed as the second parameter
and send back the result.

the client object's "CallRPC" functions tells the client object that i know
that there is a function called "QueryUser" that the server know about and it
has the same structure as the second parameter, when i call the second
parameter, call the server with the arguments passed. The client object then
creates a stub implementation which when called, creates a connection to the
server, tell the server to call the function "QueryUser" with the given
parameters, reads the results and returns the result.

The "Remote" part of RPC is done over the transport package which the main
function is mostly unaware of.

------
yannis
Nice example of using "gob" to create RPC services. But why call it a
"framework"?

~~~
tyingq
I'd argue you don't learn much about RPC from this example code with gob
handling the less straightforward parts. Going over what gob is doing would be
a good addition.

~~~
giancarlostoro
For those curious as I am:

[https://golang.org/pkg/encoding/gob/](https://golang.org/pkg/encoding/gob/)

Gob is apparently out of the box with Go.

I'm still learning through Go, there's so much available OOTB that I love
about Go. I wish other languages would take Go's Standard Library as a good
example of things to include with a language. I can do web applications
entirely with Go's standard libraries. Course then you gotta worry about
storing data in some database, but those libraries are usually done by
Database vendors or the language community.

~~~
tybit
I don’t think binary serialisation counts as a good thing to include in the
standard library. Unless Golang found a way to do it securely unlike other
languages, which typically see binary serialisation as one of their larger
mistakes.

~~~
pjmlp
If you are speaking about Java, the only mistake they usually talk about is
how the serialization algorithm and the API implementation, there are no
regrets about the binary support.

And as for .NET, Python I never saw any reference about such regrets, while
ISO C++ is still discussing papers for some kind of future support via static
reflection.

~~~
tybit
I was thinking Java and .NET yes.

Here’s a discussion for .NET core not adding support for security reasons
(eventually overruled for backwards compatibility reasons unfortunately).
[https://github.com/dotnet/corefx/issues/6564#issuecomment-21...](https://github.com/dotnet/corefx/issues/6564#issuecomment-219579151)

Java: [https://www.bleepingcomputer.com/news/security/oracle-
plans-...](https://www.bleepingcomputer.com/news/security/oracle-plans-to-
drop-java-serialization-support-the-source-of-most-security-bugs/)

~~~
pjmlp
On the context of .NET, not only backwards compatibility, performance is also
a big reason, text serialization sucks in that regard.

Also the official path forward for WCF is gRPC, which also supports binary
serialization.

As per Java, Brian Goetz has spoken multiple times about it, but the issue was
mainly due to how the whole thing is designed, not necessarily due to using
binary formats.

------
metastew
Worth it to learn gRPC first or start with std lib RPC?

