Depends on the degree of complexity of the tiles. We tried it (using tilecache or tilestache and mapproxy) and it becomes unresponsive when you rasterize several tiles (9). It can take several secons for a busy tile. If the user gets impacient like I do, zooms in and out and pans the map, it freezes to a halt. It's doable over clear water and maybe on the highway, but not in busy cities, at least not with hundreds of users requesting on demand tiles and without an insane amount of rendering resources. My impression is that it works if you're the only user and a really patient one at that.
Why not use vector tiles? I can generate a vector tile package of the whole planet in Esri .vtpk format and it takes up around 30Gb. A map service can be published from it easily. Takes me about 2 days for initial translation and about 1 week ti generate the vtpk. Neither of these have been optimised for speed.
I have an old Galaxy S3 upstairs running an app that displays this vtpk - so I can have the entire world down to house level on my phone.
