This makes sense conceptually, as /users/22 is the same user, whether you are getting the data in version 1 of the XML rendering, or version 6 of the JSON rendering, or even an HTML page or snippet representing the user. The Accept header can handle all of this.
There is also the advantage that when returning links to other resources -- e.g. GET /users returning an array of users, each specified by a URI like /users/22 -- the server does not need to prescribe a version for the client. The server just returns the link to the resource, and the client Accepts whichever format/version it wants.
Also, api clients shouldn't give a damn about URI structure/content. If they aren't using links embedded within the representations you present then your "RESTful" API has bigger problems than the where-to-hide-the-version game.
Consider that a server may choose to provide the same resources in one of multiple versions and renderings, for instance to support older clients alongside new ones. I would consider it surprising if altering a resource at e.g. /v1/users/22 would also alter resources at other URLs, like /v2/users/22.
In addition, keeping version out of the URL allows the resources to be cross-referenced between different client apps, which are not necessarily browser-bound. One client expects the latest and greatest version, one client expects an older version, and they can pass data back and forth freely.
I agree, but I'm talking about returning e.g. /users/22 instead of /v1/users/22 or /v1/users/22.json, not instead of 22.