On: The Surprising gRPC Client Bottleneck in Low-Latency Networks

[otterley]

> First, we checked the number of TCP connections using lsof -i TCP:2137 and found that only a single TCP connection was used regardless of in-flight count.

It's head-of-line blocking. When requests are serialized, the queue will grow as long as the time to service a request is longer than the interval between arriving requests. Queue growth is bad if sufficient capacity exists to service requests in parallel.

[yuliyp → otterley]

I guess I'd thought of head-of-line-blocking as the delay from a slow request stalling subsequent ones beyond the throughput limits of the system: i.e. a slow-to-parse request causes other cheap requests to wait.

[otterley → yuliyp]

Everything in a queue is subject to head-of-line blocking by definition; it's really a question of whether the wait time meets your requirements.

[voxic11 → otterley]

Requests are not serialized though. Http2/GRPC multiplexs multiple requests over one TCP connection.

[otterley → voxic11]

The TCP connection itself is a single bidirectional queue. So if packets are lost or delayed, then multiplexing doesn't buy you anything.