Great to see this project is moving forward, can you comment on whether (and which of) the criticisms and suggestions from the original HN discussion have been integrated or have influenced development for this release?
Off the top of my head, I remember that multiple people suggested that text should be allowed to contain NUL characters (not just at the end), which I agreed to. That's not terribly significant, though.
People involved on the mailing list have helped me with several design decisions. One contributor convinced me that my original syntax for unions was pretty stupid and helped me come up with something much better.
At another point, I implemented "inline" structs and lists (which are embedded into their parent without a pointer). This turned out to be overcomplicated for not all that much benefit and I was encouraged to ditch it. A week of work lost, but it was the right decision.
Contributors hacking on implementations in other languages made clear that they weren't interested in writing their code generators in Haskell, so the compiler now has a plugin system allowing code generators in any language.
This is a very incomplete list, obviously... I'd love to have more people telling me what I'm doing wrong. :)
Your last annotation example confuses the @field and $annotation syntax-- string $0 :Text $qux;
Yeah, I hear you. My personal experience is that when I look at Go code, I find it hard to parse variable definitions since the type has no separator. But I'm sure if I wrote more Go, it would get easier. I worry that there is a bit more need for Cap'n Proto schemas to be easily readable to newcomers compared to Go code. On the other hand, in Cap'n Proto there is usually an ordinal number separating the name and type anyway (though not always).
So far not many people have complained, but I could definitely be convinced to drop the colon if that's what people want.
> Your last annotation example confuses the @field and
> $annotation syntax-- string $0 :Text $qux;
Eek, fix pushed. Thanks!
This seems contrary to what the docs say:
> Text is always UTF-8 encoded and NUL-terminated.
Which is right?
The encoding page describes this a bit better:
Small question before I get there, I imagine that such things are tagged with a length so that you can actually determine where the text field really ends then?
The protobuf approach was somewhat necessitated by the fact that various parts of the protobuf implementation expected file names to be canonical, e.g. so that it could tell if two descriptors were for the same file by comparing the names. This mean it was necessary for the compiler to know where the top of the source tree was and dealing with relative paths would have been pretty ugly. This worked fine for Google-internal usage, but was problematic for a lot of open source users.
Cap'n Proto does not use file names as canonical identifiers. Instead, it requires every file to assign itself a unique ID (a random 64-bit number). This introduces some other problems, but on the whole I think it comes out a lot nicer.
In practice, the current implementation hasn't undergone security review yet, so I wouldn't recommend using it on data you don't trust. But it's intended to be secure, and will be by version 1.0.
I think the core of our protobuf library should support many different encoding formats. There is often no need to define and redfine types to support multiple wire formats: Protobuf, Thrift, Cap'n Proto, etc. One definition is enough for most.