Accountability and transparency go hand in hand, though. Teams would be able to debug together much easier if stack traces were propagated across RPCs, and good RPC frameworks can do that. Unfortunately when (ab)using HTTP+JSON for RPCs, good cross-service debugging is one of the first casualities.
I have a hard time imagining what that means but it sounds like it could be helpful for a project at work. Can you point to a source that elaborates on error propagation on rpcs?
It means if thread A does an RPC to a remote service, and the thread B in that service that's handling the RPC takes an exception, the details are serialized and returned back to thread A where the exception is rebuilt and rethrown, ensuring that in a platform that supports chained exceptions and blocking threads efficiently (e.g. JVM) that you get exceptions and stack traces that cross services.
It's not hard to implement, it's just that the industry kinda gave up on using powerful RPC frameworks with language integration. gRPC is about the fanciest you'll get and that is a clone of Stubby, which was originally written for C++ in a codebase that banned the use of C++ exceptions entirely. So it doesn't try to solve that problem.
> the industry kinda gave up on using powerful RPC frameworks with language integration
The trouble is that ties both ends to the same implementation language. Web-style RPC became popular because its simplicity meant it wasn't tied to any particular features of any particular language.
Yes, and simple textual protocols that the browser's limitations made ubiquitous.
I hope that at some point someone standardizes a way to pass virtual stack IDs, exception data and other RPC metadata via HTTP headers so we can get back the sort of features you find in high end RPC stacks.
Do you know of any methods not tied to a particular RPC framework that attempts to solve this problem? I imagine that you could try to implement something of the sort using OpenTelemetry Tracing at least, although the need to (presumably) reconstruct the distributed stack trace from a separate store is a deficiency of this approach for sure.
Not off hand, there's some JSON problems spec I found once but I don't recall if it had anything for stack traces specifically.
It's not hard anyway. You can just upgrade server frameworks to render the stack trace to either a string or a json data structure (or protobuf), and when an exception bubbles up you capture it, convert it to that structure, compress it, possibly encrypt it with a shared key (just to avoid leaks in case there isn't a proxy stripping the special header), base64 encode it and stick it into a special header. Then your HTTP client can be taught to look for that header in case of a 500, decode it, turn it back into an exception and rethrow it.
If you have a unified framework like Micronaut it's probably the work of an afternoon to throw together. The hard part is the crypto. The moment you introduce keys, corporate security teams will insist on things like frequent rotation even if it's just a backstop against badly configured frontend servers and not mission critical (if you leak a stack trace, ok, not great but not the end of the world). So it might be better to obfuscate in a different way that isn't meant to be fully secure, or detect if a request comes from an internal IP in a secure way, etc.
