I'll push back with a quote from Richard Hamming's famous talk "You and your research" (as I'm sure I have before): http://www.cs.virginia.edu/~robins/YouAndYourResearch.pdf
"Another trait, it took me a while to notice. I noticed the following facts about people who work with the door open or the door closed. I notice that if you have the door to your office closed, you get more work done today and tomorrow, and you are more productive than most. But 10 years later somehow you don't know quite know what problems are worth working on; all the hard work you do is sort of tangential in importance.
He who works with the door open gets all kinds of interruptions, but he also occasionally gets clues as to what the world is and what might be important. Now I cannot prove the cause and effect sequence because you might say, ``The closed door is symbolic of a closed mind.'' I don't know. But I can say there is a pretty good correlation between those who work with the doors open and those who ultimately do important things, although people who work with doors closed often work harder. Somehow they seem to work on slightly the wrong thing - not much, but enough that they miss fame"
Are there any of these anti-open-office pieces which explicitly mention "I might not like it and might be less productive short term ... but that still could be a net win long term" ?
In 1986 it mattered if Einstein had a conversation outside your door, it mattered in a big way, but decades later in 2017 it matters if you read Einstein's blog and follow him on twitter.
Also note the rise of groupthink because in '86 only you had Einstein standing in your doorway and only your doorway or at most a couple people, but in 2017 "everyone in the field" and lots of people outside the official field read, perhaps, Aaronson's physics (although lately mostly politics) blog.
The push comes from expansion. If your company is poorly run maybe with enough whipping everyone in the open office you'll survive maybe even thrive. But if you want to succeed at the multi-office class of size, you'll need competent management, and those along with the line workers are repelled by open offices and can get jobs at non-open office employers. An open office selects for a company that will struggle to survive past 100 people.
The biggest recent change I can think of is at the last open office I worked at, it was a fireable offense to wear headphones; the company paid a lot of money for the remodel and refusal to collaborate is being directly and intentionally insubordinate. The beatings will continue until morale improves. I admit I'm completely mystified, from what I read here everyone is visually in sight "to collaborate" but everyone puts on headphones to drown out the noise so they can work in order to eliminate all collaboration in practice, which strikes me as complete nonsense.
Intense display of social signalling via architecture, the non-living parts of the office are all open and free and the living parts of the office all have headphones on and shush anyone who speaks, library style.
There might be an aspect of reverse psychology going on, with the whole "shush people into silence" and headphones movement, open offices are knowingly anti-collaborative and perhaps management wants it that way to eliminate palace coups or something. I mean, they can't be so stupid as to think it increases collaboration or productivity, so they must intentionally be sabotaging those characteristics in favor of ...