That's pretty much exactly what I try to do. I grab the sources for the package and upgrade the underlying php version. I usually have to grab the PHP sources from a newer version of Ubuntu and modify it until it builds.
With a bit of experimenting you can easily do a network install of a headless Fedora/SL/CentOS in KVM using virt-install and a kickstart file. Use filesystem passthrough to access your source files, run php-fpm and configure its address as a backend in your web server. Keep SELinux enabled, but adjust it to your needs using setsebool if necessary.
That first link looks like exactly what I need, thank you!
I've tried Fedora in the past. I found it unstable and broken and I don't like RPM very much. It's undoubtedly improved since I last tried it (c. 2005-2006) but I don't miss RPM at all.