Based on the fact that people speaking different languages can lack basic abstract concepts or reason about them very differently, I think thoughts do have a language or at least often follow a language.
A quote as a sample:
So let me tell you about some of my favorite examples. I'll start with an example from an Aboriginal community in Australia that I had the chance to work with. These are the Kuuk Thaayorre people. They live in Pormpuraaw at the very west edge of Cape York. What's cool about Kuuk Thaayorre is, in Kuuk Thaayorre, they don't use words like "left" and "right," and instead, everything is in cardinal directions: north, south, east and west. And when I say everything, I really mean everything. You would say something like, "Oh, there's an ant on your southwest leg." Or, "Move your cup to the north-northeast a little bit." In fact, the way that you say "hello" in Kuuk Thaayorre is you say, "Which way are you going?" And the answer should be, "North-northeast in the far distance. How about you?"
Here's a link to a transcript of a lecture with some very interesting examples: https://irl.umsl.edu/oer/13/
A quote as a sample: So let me tell you about some of my favorite examples. I'll start with an example from an Aboriginal community in Australia that I had the chance to work with. These are the Kuuk Thaayorre people. They live in Pormpuraaw at the very west edge of Cape York. What's cool about Kuuk Thaayorre is, in Kuuk Thaayorre, they don't use words like "left" and "right," and instead, everything is in cardinal directions: north, south, east and west. And when I say everything, I really mean everything. You would say something like, "Oh, there's an ant on your southwest leg." Or, "Move your cup to the north-northeast a little bit." In fact, the way that you say "hello" in Kuuk Thaayorre is you say, "Which way are you going?" And the answer should be, "North-northeast in the far distance. How about you?"