> I've found the "what keeps you up at night" question has the most diverse responses.
> In general, I've been pretty surprised at the amount of apparent honesty I get. It's hard to verify the responses, of course, but any time I get a response that's not completely positive or 'canned' in some way, it seems pretty legit.
In one of the better examples, someone at a very large tech company told me that one of their major data pipelines had effectively no monitoring in it while it ran, and it took many hours to complete. The monitoring before and after the pipeline was excellent, but they had no visibility while it was running.
He said there were cases where the pipeline had effectively failed early, but continued to run throughout most of the normal run period. If they had some in-pipeline monitoring, they could have intervened earlier and helped it succeed.
> In general, I've been pretty surprised at the amount of apparent honesty I get. It's hard to verify the responses, of course, but any time I get a response that's not completely positive or 'canned' in some way, it seems pretty legit.
Could you please give a few high-level examples?