A key point of IEEE floating point is that NaN propagates. Any arithmetic operation that involves a NaN returns NaN. So you don't have to check until the end of the computation. Unless, of course, branching is involved.
A floating point type that could never be NaN is quite reasonable for a language with exceptions. But in a language without them, what's supposed to happen?
And for the half-initiated in IEEE 754, who think (like I used to) that all operations defined in the standard (including non-arithmetic ones) propagate NaNs, don't forget that pow is also defined (recommended, not obligatory), and that pow(NaN, 0) == 1, as well as pow(1, NaN) == 1, so there is a way to "lose" NaNs when using standard but non-arithmetic operations.
Well if the language is flexible enough it doesn't have to be a big annoyance to have types for both fp32 and nnfp32 (no-NaN fp32).
nnfp32 guarantees things that a fp32 doesn't, for example that comparison is always legal and works as expected.
This way you can perform calculations on fp32, but then have to sanitize the output before storing it back in the data structure.
One way to use the type system to enforce sanitation could be if values in a database are of type nnfp32, and the only way to produce a nnfp32 is using a function which returns Option<nnfp32>, returning Some only if the value is not NaN.
A floating point type that could never be NaN is quite reasonable for a language with exceptions. But in a language without them, what's supposed to happen?