

Cap'n Proto 0.4: Time-traveling RPC - kentonv
http://kentonv.github.io/capnproto/news/2013-12-12-capnproto-0.4-time-travel.html

======
jerf
My gut reaction was "ewwww", but upon reflection, this does seem like a great
idea. I disagree with the claim that it "solves" the RPC problem (you may be
making it _less expensive_ to make a network request, but you still can't
pretend a network call is a function call; one of the big problems with such
pretending isn't the latency but things like "total network failure" that are
poorly represented by the function call abstraction), but it does go a long
way towards mitigating it. The ability to create much more orthogonal and
composable RPC interactions is a lesson that many such protocols could stand
to learn.

~~~
kentonv
Oops, you're right, I should have said "helps solve" rather than "solves".

Yes, total network failure is still an issue. Typically, applications need to
solve this by catching a network exception somewhere high up the stack and
then starting over with a fresh connection.

There's certainly an argument to be made for using a different abstraction,
e.g. those provided by ZeroMQ, that can handle fault tolerance more
transparently. But, it all depends on your use case. A lot of things just
don't fit into stateless models -- particularly real-time collaborative types
of things. (ZeroMQ pairs nicely with Cap'n Proto serialization, BTW, if that's
the route you prefer.)

------
pkinsky
Awesome, a C++ implementation of the Future monad. Personally I would prefer
.flatMap to .then, as in Scala. That would also allow for a companion function
.map that transforms the result of a Future using a function that does not
return a promise.

ie: (in Scala, my C++-fu is weak)

    
    
        getTweetsFor("domenic") // returns a future
           .map(parseTweetsForUrls)
           .map(_.head)
           .flatMap(expandUrlUsingTwitterApi)
           .flatMap(httpGet)

~~~
kentonv
Heh, yeah, I only realized yesterday, I think, that promises are monads.

then() will actually accept functions returning regular values, too. A big
difference between then() and map() is that then() accepts an optional second
parameter for exception handling. In any case, I used "then" because that's
what Javascript promises use.

~~~
frowaway001
JavaScript promises are wrong and broken. See the huge spec of the Promises/A+
vs. the few lines of specification for a correct one.

~~~
kentonv
Are they broken in a way that I have copied? I don't actually know too much
about the details, I just copied the `then()` call.

~~~
infinite8s
I haven't looked at your implementation, so I'm basing this on the statement
"then() will actually accept functions returning regular values, too" in your
grandparent comment. Because of this you don't have a true Monad (as in your
Promise implementation doesn't satisfy the third monad law).

One easy way to think of Monads is that they are like black holes - you are
free to put something into it but can't ever pull it back out. Instead, all
you are allowed to do is chain them together (with bind). So to transform the
value inside a monad, you must write a function that takes that value and
returns the transformed value wrapped in the same type of monad (Promise in
this case). The monadic machinery in 'bind' will take care of pushing the
result from one Promise into the transform function.

~~~
kentonv
What dwrensha said, although I'd describe it like this: Allowing then() to
take a function which returns a bare value rather than a promise is just
syntax sugar. You can always wrap a bare value in a promise by creating an
immediate (already-fulfilled) promise. Giving then() such a function -- that
wraps its result in a promise -- is equivalent in behavior to giving then() a
function that just returns a bare value, except that the latter is somewhat
more efficient in implementation.

------
jmillikin
I don't understand how this is different from the standard RPC model.

A typical RPC in a C++/Java/C# language looks something like this:

    
    
      var client = EchoClient::Connect("corba://example.com:9000");
    
      var request = new EchoRequest();
      request.set_message("hello world!")
    
      var options = new RpcOptions();
      options.set_timeout_milliseconds(5000);
    
      Future<EchoResponse> futureResponse = client.Echo(request, options);
      EchoResponse response = futureResponse.Wait();
    

According to the article this is going "back in time", but that's completely
silly -- the response is being returned at the point of the Wait() call, which
blocks until the server has finished RPC processing.

And of course, languages with proper thread support (e.g. Haskell) don't need
to bother with futures at all, because it's easy to spawn a new thread for
each request instead of blocking some central main thread.

And it's fairly easy to provide syntactic sugar over this to provide closure-
based control flow

~~~
kentonv
Where does that code come from? That's not Cap'n Proto.

Sorry, but did you read the docs?

~~~
jmillikin
That's the kind of code that would be used in a typical RPC system written in
the last ~20 years. You have some sort of client stub, a request object, and a
response handle that has operations for waiting until the server has generated
the response. The actual marshaling implementation could be ASN.1 or JSON or
Thrift or whatever the local programmers want.

I want to know how the Cap'n'proto design discussed in the post is
significantly different from this standard RPC model.

~~~
philjohn
If you're executing a 2nd procedure call that uses the output of the first (or
3rd, 4th 5th etc) then it requires 1 call. A traditional RPC system would
either need a combined method (which is suboptimal from a "clean design"
perspective) or you'd get the response of the 1st back, and then just pass it
back to the server for the 2nd call.

~~~
jmillikin
So the major improvement described here is a hint to the server that it should
pass the response from one RPC directly into the request of another, rather
than returning anything to the client?

That seems like it'd only be useful in very specific circumstances; you'd need
RPC methods with the same type for request and response, and it only works if
you're using the same backend for each call (e.g. you can't have load
balancing of separate calls).

~~~
kentonv
If you read the documentation, I give an extended argument for why this is
useful: it enables more object-oriented protocol design.

[http://kentonv.github.io/capnproto/rpc.html](http://kentonv.github.io/capnproto/rpc.html)

Also, it is _not_ the case that the new RPC's input type has to match the old
one's output. You can take just one object embedded in the old response and
use it in the new request.

Please do look at the examples.

------
kentonv
It seems my post has been severely penalized by HN. Went from #2 to #30 all at
once, then dropped from the front page.

Anyone know what I did wrong?

~~~
jmillikin
You submitted a post about programming to PoliticsNews. Please try again when
you have an outrageous story about government abuse.

~~~
kentonv
I think I figured it out. Someone with moderator rights must have read the
summary, not read the documentation in detail, and decided that I had only
implemented regular old promises/futures and was calling this "time travel".
They probably missed the point of promise pipelining -- which is something
that no mainstream RPC system implements to my knowledge, only research
systems like CapTP. So they demoted it for making what they assumed to be a
ridiculous claim about a mundane piece of software.

Sigh.

------
vosper
Is anyone using Cap'n Proto in production? What has your experience been?

~~~
kentonv
Canonical is using it in Ubuntu Unity:

[http://packages.ubuntu.com/source/trusty/unity-scopes-
api](http://packages.ubuntu.com/source/trusty/unity-scopes-api)

There is also the OSX text editor, TextMate, which uses Cap'n Proto for some
kind of cache file.

Sadly, most users don't actually tell me about their experiences, but given
the number of people asking questions and filing bugs, I suspect there are
many more.

The biggest thing holding people back is lack of MSVC support.

------
codex
SMB and NFSv4 have support for something similar in their protocols. Nice to
see it generalized. The file protocol versions only chain RPCs on success. You
can't operate directly on the result.

