Collective code ownership doesn't exclude stewardship or experts working on the code they are most qualified to work on. In a healthy team this comes natural.
Assume these people are grown-ups, who can figure out for themselves that a change X to code Y can best be made by Jim (the senior expert), but it may also be helpful if Johnny (the talented junior) makes the change after which Jim reviews it.
Any ownership model other than collective ownership takes these decisions out of the hands of those most qualified to make them and shifts them to hierarchy and regulations.
Management's job should be limited to addressing those that undermine the team's functioning and remove them if necessary, not facilitating it by imposing code ownership levels.
The Martin Fowler example the author links to speaks of "undisciplined programming by a couple of developers" and "sub-par producers". Geez, guess what the real problem is here?
Why are you assuming a hierarchy? It could just as well be a flat sibling-space, where people "own" code in the property-rights sense, not the management-fiat sense. (In other words, they own it because management won't apply the force necessary to take away the ownership they hold by default.)
Imagine a conglomerate--say, Microsoft. Does the Windows division "own" Windows, or can the Office division make commits to it if they need to? In this case, management would probably have to intercede to get someone from Office commit-access to Windows. Not because management is denying that access in a command-and-control manner; that access (keys et al) is just held inside the Windows division itself, and it takes special measures to get it if you don't work there.
Same idea here, but on a smaller scale. Maybe each developer keeps their own git repo for their module and only pushes releases (e.g., .deb files of compiled code) to the central repository. Maybe each developer runs their own SOA service and everyone else is expected to only interact with it over RPC, not to muck with its internals. Etc.
Johnny wants to make a change. Jim says "no". Johnny's now pissed, because Jim's broken code is causing him problems. Jim's pissed, because Johnny wants to touch "his" code. And let's pretend, for a minute, that they aren't both 100% perfect employees (because I know the answer is - "they'll sort it out sensibly" - we need rules for when common sense breaks down).
Then there are hybrids, where you have a big company and stuff gets sliced up. eng/foo is owned by one group, but eng/bar is owned by another. Then there's hr/secret which is owned by a third group and isn't even readable outside of that group. The trick is that someone in eng/bar can submit a patch to one of the owners for eng/foo, and if that person cares to approve it, then it can be applied.
Can this be abused? Sure. Get approval for your change and then modify it to be evil before you check it in. If the systems in place aren't specific about what approval means and the people have bad intentions (or maybe are just clueless), then it's possible to make all kinds of disasters.
The better question is: who or what is going to make sure someone actually owns something? Just having some code in your depot is no assurance that anyone is taking care of it. You might not find out that it's broken until long after the last developer has moved on to greener pastures.
It simply means that anybody CAN edit the code, not that everybody WILL DO everything. After all it's a matter of common sense: If I don't understand what's going on, I will not touch it (this address the slow delivery drawback).
All the other points against Collective Code Ownership seem a little empty as well: inconsistencies are not result of collective code ownership, but poor organization. Agile Programming != Anarchy