I have a couple sites on Slicehost and I love it. The only thing I miss from shared hosting sites is the easy to use management console for setting up email and such. Other than that Slicehost is the way to go.
A pet feature I wish they had was exposing a way to download the actual Xen image your site is hosted within. Then you could be developing your production server sans internet, and rsync the resulting image.
I generally hate those consoles, as they make things work in unexpected ways. Recently, I spent a while tracking down a bug on a managed VPS with Plesk installed where mail sent by scripts on the server to the same domain (let's call it example.com) were going to local delivery at example.com instead of the MX entry of exchange.example.com (a separate server).
Many times, I want the email system to share its login database with some web app anyway.
What I would really love to see is affordable hosting like this but with the ability to do stuff like run a custom web server. I know this thread isn't really about hosting but has anyone seen something like this or is it even technically possible without going fully dedicated?