I would argue that that's because most social sciences are at some level just applied math. Math however isn't social science. I don't want to be dismissive of social progress in curriculums though; I had many advantages growing up but I more than once found myself in classrooms where I didn't speak the language and math class was always a relief because it was just math. I realize this is a rosy view, but there was something to it.
I certainly empathize with you, but as long as most math curriculum is math at it's core, then it is a net benefit for the large majority of students. It does take a skilled teacher to really pull it off.
For the vast majority of students, the way they will use math most of the time is only after analyzing problems and translating them to mathematics, which for many students also really does help the math part work better. And ultimately before you start specializing your math classes (some may argue we should do it earlier than is done in the US, and I'd agree), it makes sense to train students to apply math a solid minority of the time in ways that aren't "just math".
Until the last two years of so of high school, the skills that are mainly being taught in math class aren't necessarily "just math", and the main reason they are being taught isn't to help people be better at pure math, but because they are foundational skills elsewhere.
These ideas aren't just there for fun. They're there because a significant amount of research has been done and replicated dozens of times around the world and found that for the educational objectives at that point in time it is the most efficient way to do so. If the hypothesis is that actually teaching math another way that completely gets rid of word problems and tie-ins to other subjects is more efficient, this is a testable hypothesis that is sure to be very interesting to the scientific community, and I'd love to see it put to the test.
Objectively, the research was of good quality. The research was replicated all over the world successfully, and it was statistically and methodologically sound.
Those people actually accomplished something. Their local education system became #4 in the world in mathematics without unreasonable hours or high levels of stress.
