I've gone with this approach for a niche social networking site I'm building. The biggest benefit an HSM provides is vendor wrapped keys, which does simplify key management and allows you to lean more on your vendor to support key material confidentially. In my case I didn't feel the added cost and complexity was worth it.
