For me, the most important part is -fno-math-errno which allows the compiler to ignore that libm functions are allowed to set errno. This is perfectly safe (unless you rely on that rarely known side effect) and is usually the one flag I explicitly set.
And the primary benefit of doing that is so the compiler can inline math functions like sqrt() as a tiny number of instructions (on modern CPUs) instead of having to call the standard C function, which is much slower.
For sqrt() on x86_64 and gcc/clang, yes. But functions like fmod() are generally more instructions. And as for trig functions like sin(), AFAIK most compilers will always use a function call, because the x86 trig instructions don't have good speed/accuracy compared to a modern stdlib.
And YMMV when it comes to other arch's and compilers (and -fmath settings).