Hacker News new | past | comments | ask | show | jobs | submit login

also the overhead of calling out to c can actually be quite high: https://www.cockroachlabs.com/blog/the-cost-and-complexity-o...

While some overhead and increased debugging effort may be inevitable, it's a mistake to blame them on the use of Cgo itself; the root cause is Go's custom ABI and user-level-threading ("goroutines") model. And I really have to dispute OP's claim that the techniques they mention do not "require significant effort, or large changes to program structure" of their own. In many ways, rewriting some portion of the code can actually be simpler.

Well, it may be true that the "blame" for why CGo is slow is up for debate, it's just a fact that it's rather slow to use.

Go isn't unique in this; there are many languages with runtimes that require similarly intensive amounts of copying and context conversion before C code can run on whatever the data is. However, most of those languages, like CPython, are themselves slow enough that the penalty isn't as noticeable against the general background noise. (In general CPython requires a lot more copying too; Go is closer to C struct and array semantics and can more often get by with some form of memcpy, the internals of Python look nothing like that.) Go is fast enough that it's much easier to get into scenarios where in a tight loop you're spending 90% on CGo overhead if you're not careful with data flow. For those languages that have to copy a lot out of their runtime and are also fairly fast, they'll face the exact same issues. It's not really a "Go" issue per se, but the challenges faced by any language that wants a runtime significantly different from C.

(One of the miracles of Rust is building an environment and runtime that isn't stuck on C's limitations but at the same time can still speak to C really, really cheaply. Plenty of languages have one or the other of those, but there aren't very many that have both. I'm not sure there's any other language that has threaded that particular needle so cleanly.)

> Go isn't unique in this; there are many languages with runtimes that require similarly intensive amounts of copying and context conversion before C code can run on whatever the data is.

I agree, and some of this overhead is even inherent in legitimate, foundational choices such as VM-interpreted/"managed" code (as in Java/.NET; but Python does this as well) or the use of tracing GC (which requires some strict discipline on heap contents, so as to enable the GC itself to reliably "trace" and discover the semantics it cares about). So, I'm definitely not saying that the choices made as part of Go's design are consistently wrong here!

Indeed, Haskell is in a very similar place overall; the "fibers" that GHC uses in its compiled code are implemented via async code underneath, much like Go's goroutines, and Haskell's performance is also well within an order of magnitude of pure C-like code. So the issues you describe are quite well-understood in that context, and they are definitely not regarded as a "reason" to avoid the use of C FFI when performance requirements call for it. But describing Cgo itself as something that's high-overhead and should not be used for that reason is misleading to an even stronger extent, and that's what I was objecting to in the grandparent comment!

Well said. Also worth noting that most of the languages like CPython are very slow because many optimizations are now prohibitively difficult given the "easy C-interop" constraint.

Isn't that more because the choice to support C interop by giving C access to interpreter internals, which is the easiest-implemented option but not the only way to get easy-to-use C interop.

That could be. Although I'm not familiar with any languages that are fast and have a GC and easy C interop. At least none of the major VM languages or (non-embedded) scripting languages. Maybe D or some other similarly non-mainstream language.

Lisp? Common lisp's ffi faculties are good, from all I've heard.

Could be. I’m not that familiar.

Applications are open for YC Winter 2020

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact