Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: How important is it for telemetry apps to have something listening?
3 points by m3047 7 days ago | hide | past | favorite | 2 comments
Company A offers a network software product which is ubiquitous on the internet (seriously, it has lost some market share over the years depending on how you measure it so maybe "pervasive" or "endemic" would be better characterizations). Company B was founded by Company A's chief architect and tech leadership to do things with telemetry related to Company A's product and eventually prevailed on Company A to implement a direct telemetry tap in the product, which I'm perfectly happy about in principle.

For whatever reasons company B chose to implement unix domain sockets as the only streaming protocol / mechanism for the telemetry tap. The protocol uses a handshake initiated by the client at setup, the software is the "client" and whatever receives the telemetry is the "server".

The tap is logging, not auditing: if messages can't be received then they are dropped, instead of the application pausing processing. You might suspect that backpressure might make the application pause, but that is incorrect because the protocol client drops messages with no in-band indicator if backpressure occurs.

The C and Go language implementations of the protocol were both written by Company B; the only other implementation was written by me (pure Python).

Both the chief architect and myself have been writing networking and telemetry software for over 30 years. I'd really like the software application to send its telemetry directly to the network. I recently asked them: why not direct-to-network, and why not UDP (with a header containing a packet counter and a start/continuation indicator)? Their answer was "too easy to send the stream to somebody who's not listening". I am puzzled by that answer.

Granted, there's no positive connection indicator with UDP as opposed to TCP (where ACKs still have to be sent back to the client) but the answer as given still needs unpacking, because what kind of indicator exists in the unix socket when at the application layer the protocol client drops messages with no fanfare? Other than the initial handshake and the observable that the connection is blocked but still intact I can't see any.

Why not use UDP and let the network drop packets for free? How is that worse than network drops a TCP packet and retries, so eventually backpressure develops and the stream is stalled by the client?

UDP is going to get an ICMP response to the client/sender that the port, host or network is unreachable. I don't think it's so bad that servers can come and go and telemetry still occurs oblivious, maybe that's all there is to it (plus there's the option for multicast this way which kind of makes the point). In terms of security "rings of hell" you can restrict the traffic to localhost, a VPC, a LAN, a subnet (and you'd want to with this data). If you don't want to offload that responsibility, you could do it from the client/sender by limiting the number of hops the packet can make (IP datagram TTL).

Is there really any important difference TCP vs UDP other than those outlined above? Is the difference in operating dynamics between TCP and unix domain sockets sufficient that equivalency should even be entertained?

> because what kind of indicator exists in the unix socket when at the application layer the protocol client drops messages with no fanfare?

I think you already understand that they are unreasonable people. One answer is that the person who implemented/validated it and the person who decided it needs to be TCP were not communicating well.

It obviously works well enough as is, with the current system, so they probably aren't looking into how/if it works much.

I would just do it whatever way you want as long as you can meet whatever needs they have. Clients often specify that things need to done a certain very particular way, then never really care if it is done differently as long as it works.

I have been told before that a service only uses TCP when in fact when I tried connecting to it, it turned out to be only UDP. They didn't know, and bringing it up to them just made them confused and angry.

So I guess what you're saying is that adjacent to the kitchen in the restaurant, there's a service window that plates are served through, and for some reason, it has sliding shutters on both sides.

It's probably fine and there are no stories of any sketchy stuff going on at the service window, but.. it's kinda unusual, architecturally for a restaurant? Certainly not what you'd expect to find at a popular restaurant known for showcasing their culinary experience (and, admittedly, mistakes; they're forward-thinking).

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact