On: Command-line tools can be faster than your Hadoop cluster

[threeseed]

Another explanation is that your CIO is not an idiot but rather they know about future projects that you don't. CIOs want to build capabilities (skills and technologies) not just one off implementations every time.

Not saying this is the case but CIO bashing is all too easy when you're an engineer.

[acdha → threeseed]

A good CIO would know that leaving out key parts of the project is unlikely to produce good results. Even if the details aren't final, a simple “… and we probably need to scale this up considerably by next year” would be useful when weighing tradeoffs

[eru → threeseed]

The CIO is probably better at playing corporate politics than a grunt. (That's why they are CIO and not a grunt.)

[JustSomeNobody → threeseed]

The last people you want to keep secrets from are your engineers.

[EpicEng → threeseed]

I think that hiding requirements from the people building the system would, in fact, be an idiotic move.