As far as I can tell (other comments suggest this is being misunderstood), the change is not about switching from softfp to hardfp-- the FPU was always being used-- but switching from an ABI designed for softfp, where float arguments to function calls are passed in general purpose registers, to one designed for hardfp, where they're passed in the VFP registers.
tl; dr: Recompile everything with FP set to hard instead of soft.
softfp was a compromise for a number of years but since we are mostly in the age of Cortex now for mobile/tablet markets, it should be possible to recompile the entire toolchain, kernel and apps for a significant speedup anywhere FP is used: 3D, OpenGL, OpenSSL etc.. However, binary only blobs such as 3D drivers need to be recompiled.
Floating point performance shouldn't be a major bottleneck on Linux except maybe for OpenGL heavy applications. The amount of cryptographic computations doesn't dominate either.
OpenGL in desktop effects doesn't count because it takes a lot more number crunching to rotate a complicated scene graph in a game rather than composite a few layers of window surfaces together to form a desktop screen.
I'm also surprised that using hardfloats with an FPU is only a few dozen percent faster than softfloats with a CPU! Makes me question if an FPU is needed at all in the future. Some FPU usage could be offloaded to the GPU as well, and that's a lot faster than a single FPU after a certain overhead.
There's a debian hardfp port as well (arch "armhf", as opposed to "armel") in the current debian testing release (aka wheezy.) http://wiki.debian.org/ArmHardFloatPort
Not all packages have been successfully ported yet, but it seems to be coming along well.
Interestingly, iOS uses the "old" ABI:
http://developer.apple.com/library/ios/documentation/Xcode/C...