On: Ask HN: What highly scalable thing have you built with Go?

[kasey_junk]

Similar problem space (we don’t bid but capture rtb data for analysis).

Similar throughput, our bottleneck at this point is moving data around.

We’ve abandoned channels for most of this. The next major improvement would be to rebuild the http stack & that’s just not worth it.

[jonathan-kosgei → kasey_junk]

Someone shared this with me https://blog.golang.org/share-memory-by-communicating - Share memory by communicating.

And also pointed me to nats.io, a messaging system that handles 10M messages per second on a $50 server.

See the comment at: https://www.indiehackers.com/forum/how-we-handle-25m-api-cal...

[nemothekid → kasey_junk]

Have you checked out https://github.com/valyala/fasthttp

[jonathan-kosgei → nemothekid]

Should fasthttp be used in production? It seems to get a lot of flak for not fully implementing HTTP.

https://www.reddit.com/r/golang/comments/5w3ang/switching_fr...

[sagichmal → jonathan-kosgei]

Like every engineering decision it is a risk/reward spectrum. But unless you have profiled to know that net/http is your bottleneck, no, you should almost certainly not use fasthttp.

[kasey_junk → nemothekid]

We use it in production, though fair warning we have very limited endpoint and consumers that allow us to test it aggressively.

What I really meant is we'd need to take things down to the bare network stack. Lots of our memory use/bottlenecks are fairly deep into the std lib.

[mkarlsch → kasey_junk]

Same here - we don’t use channels in the hot code path and did replace the Go HTTP parser.