I have a pet theory that there's some overlap of writing patterns that A) indicate quality, clear communication and B) those that the LLMs have been converging on. As a result, we're ruining the reputation of some great writing patterns because we've associated it with AI slop.
My pet theory is that this was happening, because labs naturally worked to source highest quality data sources they could get. Even if they initially slurped everything indiscriminately, the obvious next step is to sort them by "quality", and bias towards the high end while trimming from low end.
So obviously, government documents, top publications from liberal arts scholars and STEM > Wikipedia > Reddit (and within, r/AskHistorians > r/all, etc.). The first group is the one most likely to (over)use correct / fancy typography.
I have a related pet theory: the telltale sign of AI writing is being too good for the context it's found in, in terms of vocabulary and style as well as structure.
What is jarring about this is precisely the quality - we associate certain styles with important (or self-important) venues, so it's uncanny to suddenly see random shitposts, copywriting spam, or regular chat replies look like work of NYT bestselling authors or legal scholars - it feels wrong in places we expect basic language, grug-brain reasoning, and lots of typos.
Also, what makes the above much more noticeable, is that AI systems don't have cross-session stylistic memory. From the POV of an LLM, that perfectly fitting, masterfully crafted starter is the first and only time it used it. From the user's POV, it's the tenth time they're seeing it in the span of an hour.
There's also no voice. If I get a text from a friend, I can hear them. Their tones, their mannerisms. I can't hear Claude because it's not a person that I've met and interacted with.
People use language in their own distinct fashion. To the point where you can often figure out the writer from a long enough sample. Newspapers and internet comments end up a blend of different voices but in visually distinct boxes.
LLM intentionally randomize output to cover up all the plagiarism, but this largely destroys the consistency that cause writers to feel unique. Larger patterns still emerge but it just feels stale.
Models are of course different from humans, because the process by which they were created is different. But what’s undeniable is that humans also love to quote or repeat what others have said — we learn from our own kind, that’s how we gain experience. Yet rarely does anyone hold a person to a word-for-word standard over the course of their entire life, checking how many of their sentences were picked up from other people’s conversations — and then call that infringement. Just as no one right now would say I’m plagiarizing a model’s speaking style because I use em dashes, and no one would consider it misconduct when I reference a famous person’s ideas in everyday conversation. I don’t think models intend to infringe; it’s simply how they learned language. But perhaps, just like humans, they need to learn to provide more citations for what they reference.
Even individual words can be an issue. Humans understand when you want a new story about a magic school there’s issues using Hogwarts as the name.
LLM’s will happily regurgitate multiple paragraphs practically verbatim if you don’t specifically design them not to behave in this way.
Because ultimately when you design a system to best predict the next output based on a given input perfect replication of prior training is the correct response mathematically.
That's a misconception. LLMs are trained to suppress regurgitation, not because of plagiarism concerns, but because quoting training data verbatim means the model didn't learn from it, just memorized it.
Models absolutely do have voices. Spend some time using them, and you'll see. I can easily tell Claude-written prose apart - a consequence of using it daily.
You misunderstood, “Larger patterns still emerge” means yes you notice LLM’s but it’s for different kinds of patterns than you get from people.
> not because of plagiarism concerns
I’ve second hand accounts of meetings focused on exactly the problem of how much verbatim text was being spat out and what to do about it. Some versions of specific models are far worse at this but it’s a huge legal liability not to address it.
can't say the same for newspapers or internet comments, but when reading books there's usually a voice or a "vibe", and if there isn't one then it's just an unreadable book
I didn't say it's copying anything. It's learning styles - all of them - in pre-training. Post-training is just about giving it the right defaults.
Doesn't change my point anyway - even if post-training, labs naturally want the LLM to come off as smart (well, except for Grok). Post-training is where they "untrained" em-dashes and such, due to complaints of users.
I did at the beginning, but there is only so much you can take, and no one cares if you have read books and seen it before. There were a bunch of programmers whom I thought would understand em dashes, but they did not and said no one uses such dashes only normal hyphens.
Em dashes ended up as one of the first victims of llm age
Em dashes were already "doomed" at least half a century ago, because they didn't make it into ASCII character set - which arguably is just because they were seen as redundant on ancient typewriters; that's also why they aren't there on normal keyboards, which was a double whammy: inconvenient for programmers to support, inconvenient for users to type.
In other words, since the very start of computer era, the use of em dashes was always a signal.
I think it's because it starts writing the final output and doesn't have the ability to go back and correct. It's like a stream of consciousness bandaid
