> the models are improving at a rapid rate. If asked ~3 years ago where the state of the models are today, it would sound like sci-fi
Absolutely true, many things will continue to improve in significant ways. However, if we look at the modern history of rapid disruptions driven by technology (a side interest of mine), persistent patterns emerge. Similar to avalanches or flash floods, such periods of very rapid disruption are often triggered by one or more significant breakthroughs in certain technologies. Early rates of change tend to be fast and furious but eventually begin to taper as recently unlocked low-hanging fruit is harvested and those racing through newly found terrain encounter all-new significant barriers and points of friction. Early in such periods, extrapolating the recent extraordinary rates of change forward has poor predictive power. Sudden extreme bursts tend to regress back toward the long-term trend line.
Arguably, the current disruption in LLMs can be traced to post ~2010 research slowly building to the 2017 transformer paper and the adjacent work it quickly inspired. So today is, arguably, mid or late-ish in the LLM rapid burst phase. The rate of fundamental, broad-based breakthroughs lifting all LLM applications has clearly slowed with many of the most impactful recent discoveries being in scaling, optimization, tuning and productization toward specific domains. That doesn't mean there can't be another transformer breakthrough tomorrow but, historically, black swans rarely travel in flocks.
> The rate of fundamental, broad-based breakthroughs lifting all LLM applications has clearly slowed with many of the most impactful recent discoveries being in scaling, optimization, tuning and productization toward specific domains.
To me it definitely feels like it's still accelerating, with the most impactful recent discovery being RL training reasoning models (late '24, early '25).
There's an interesting article called "sigmoids won't save you" https://www.astralcodexten.com/p/the-sigmoids-wont-save-you which argues that (unless you have privileged information) you should always assume a process will continue about as long as it’s continued already. (Lindy's Law)
With that in mind the current disruption should last another 10-15 years (assuming it started in '10 or '17.)
This is of course true in general. But the question is not "how with this evolve" but how will we deal with the rapid changes in the industry? I suspect a long term k-shape salary curve, even worse than today, with the lower 80-90pctile salaries bottoming out such that many have to exit the industry to make ends meet. You can laugh and blame them for not saving as much as they should, but that's still a fairly horrifying prospect for most of us.
I think a _lot_ about stock trading a profession vs algorithmic trading. It was brutal - suicides, many pivoting out to doing car dealership-style work. Probably a 1/10 or 1/20 survivor rate every couple years, with almost all of it a very painful five year period.
I would ask for references for the suicide claims, so others can assess the impact themselves. That's a very serious claim to provide without any proof, especially to a group of people who very well be going through the same thing. I am not saying it did not happen, only it's the right thing to do.
That really depends on how you define alive and well. There are still stocks and there are still traders, but the market valuations are obscene and it sure appears that there must be collusion or corruption driving the industry to jam massive IPOs into every index and 401k they can find as fast as possible to fasciliste and exit.
Progress happens in a series of S-curves. While your observation is correct that advances occur initially rapidly then taper off, the next step tends to arrive sooner than the previous, and with greater magnitude [1]. Tim Urban's article from 2015 has a great explanation of this phenomenon [2].
> The rate of fundamental, broad-based breakthroughs lifting all LLM applications has clearly slowed with many of the most impactful recent discoveries being in scaling, optimization, tuning and productization toward specific domains.
What this means is that the disruption across industries not even truly begun, because it's not the generic chatbot models that are going to kill labor, it's all the domain-specific applications that leverage those models to perform work that was performed by humans
Absolutely true, many things will continue to improve in significant ways. However, if we look at the modern history of rapid disruptions driven by technology (a side interest of mine), persistent patterns emerge. Similar to avalanches or flash floods, such periods of very rapid disruption are often triggered by one or more significant breakthroughs in certain technologies. Early rates of change tend to be fast and furious but eventually begin to taper as recently unlocked low-hanging fruit is harvested and those racing through newly found terrain encounter all-new significant barriers and points of friction. Early in such periods, extrapolating the recent extraordinary rates of change forward has poor predictive power. Sudden extreme bursts tend to regress back toward the long-term trend line.
Arguably, the current disruption in LLMs can be traced to post ~2010 research slowly building to the 2017 transformer paper and the adjacent work it quickly inspired. So today is, arguably, mid or late-ish in the LLM rapid burst phase. The rate of fundamental, broad-based breakthroughs lifting all LLM applications has clearly slowed with many of the most impactful recent discoveries being in scaling, optimization, tuning and productization toward specific domains. That doesn't mean there can't be another transformer breakthrough tomorrow but, historically, black swans rarely travel in flocks.