I think you underestimate the value of pipelining here. You could spend time narrowing down the set of logs to download... but in the time it takes to figure that out, you might as well just download them all.
Having "home depot" locally available for analysis is never a bad thing, plus you may be racing against time re log rotation, etc.
> For someone that is all set-up and on top of things, this seems like something that is a day's work and easily more, not counting follow-up work and validation to further investigate the additional questions that would inevitably (in my opinion) be raised on such a big dataset.
In the middle of a SEV, you don't have a day to perform this kind of analysis. 15 minutes till the SLA clock starts ticking and customers are owed refunds.
OK, but it's not a "SEV", he's looking at 2 years of logs on hundreds of thousands of servers over a wide variety of applications. Nothing appears to be "down". This is more like an investigation looking for some high-value efficiency improvements (which is something he's written about).
I am sure he did it "quick", but mere minutes of work and mostly just waiting around for something like that analysis? I doubt it!
Seems pretty normal to me - we used to do this kind of thing on the regular. SEV analysis often requires digging through months of past logs to ascertain all the affected customers. If that takes a day every time you hit a customer-facing issue, your team ends up spending all their time on after action reports, and never gets any engineering done.
Having "home depot" locally available for analysis is never a bad thing, plus you may be racing against time re log rotation, etc.
> For someone that is all set-up and on top of things, this seems like something that is a day's work and easily more, not counting follow-up work and validation to further investigate the additional questions that would inevitably (in my opinion) be raised on such a big dataset.
In the middle of a SEV, you don't have a day to perform this kind of analysis. 15 minutes till the SLA clock starts ticking and customers are owed refunds.