More great work on local model but you’re still losing a lot. Down to 2 bit quantization and the coder model throws away half the MoE experts. In a world where anything is better than nothing, this is a net win. But we have a way to go still.
The real problem is the DRAM mafia and artificial scarcity. One of my notebooks is almost 3 years old, effectively similar spec now - same price (a bit higher actually). My desktop PC built around march/april 2023 (4090, 64gb ram, 7900x3d) is now pretty much still the top dog out there due to the gpu and fast ram insanity and if I wanted to sell it today, I'd get more money for it now used and over 3 years old that when I bought it!
We should be having 64/72+ GB video cards by now. 128GB+ system ram prosumer laptops and 256GB+ system ram prosumer/gamer desktops. But it all went to shit and it will require some brutal datacenter and datacenter-adjacent bankruptcies before it gets better.
Some of these greedy bastards need to lose their pants on all of this.
A 64GB RTX 5095 GPU @ $2500 would pretty much collapse the cloud hosted AI market. Being able to run Qwen Flash Next/Qwen 4 Flash on my local workstation would eliminate the need to pay for cloud hosted AI for the average person/developer. But still API out periodically for architecture level decisions/layout.
Periodically I run into problems that I need Sol for, but I suspect Qwen Flash will meet most of my needs
Yea, it's nuts. I bought a 4080 3 years ago, and there's no viable upgrade path. I should have went with the 4090 in hindsight when it was $1600 USD from Nvidia's website; figured I would be upgrading around now.
I'm on the same boat with a 4080, the extra GPU RAM of the 4090 over the 4080 would have been nice, but it still wouldn't be able to run any decent models.
I configured my PC in December 2023 with a kit of 2x32GB DDR5-6000 for €249, thinking I would upgrade that RAM later to 128GB by just adding another kit. Now nearly 3 years later that same kit is over €1500 :-(
I've also lost all interest in upgrading to a 50-series GPU. With exception of the ludicrously expensive 5090 the entire 50-series lineup is capped at 16GB, I don't see the benefit of upgrading.
Yea - (I wish I had gone with 64Gb ram...) there is no reasonable upgrade path for us beyond waiting, unless the 5090 comes down to MSRP, which is unlikely. The 4080 is still a nice GPU, but the 4090 would have been a better option, if we'd known what was coming!
You could… actually do that. Same technique as abliteration. Run prompts until it can’t write Java lol. I bet it becomes worse at programming in general though :(
I waiting for the day when memory would stop being the bottleneck, when we will load specific parts of the thing on-demand and actually know which part does what.
Reminds me of my younger australian cattledog. It's not about what he's getting. It's about what my other dog is that he could be getting. So the whole time he's got a full bowl in front of him, he's death staring my other dog in case he abandons his bowl.
It's fascinating watching the wealthiest country in the world get butthurt over a company that employs 70,000 W2 employees who has paid around $39 billion in taxes in the past 5 years - and that's just federal and excludes e.g. property taxes and payroll. That's more than the GDP of Iceland or Honduras. Them legally avoiding a few billion only matters if you're a nyt journo out of news worth printing.
Three years. You really need to be your own person to handle it, given how alcohol hasn't just permeated the fabric of society, but is the fabric in many situations. For me being available to perform at the top of my game 24/7 is something I enjoy very much e.g. getting a call to do a mercy flight at a moments notice from the island I live on, at night in IMC, knowing it's no problem because I'm sharp - now that's a great feeling.
I think if you're going to go sober, doing something like volunteer fire or another field that lets you feel good about being available 24/7 when others might not be, really helps keep you motivated and appreciate the new state of being. Also just pushing yourself mentally or physically - e.g. training for a marathon or ironman - it lets you actually reap the rewards rather than feel like you've lost something.
Just going to throw out a follow-up here since I own a bar and work barman to fill shifts from time to time:
Athletic IPA or hazy IPA are my favorite NA beers. Athletic does a spectacular job of NA in general. Also Guiness NA - the can with the blue stripe on top - absolutely delicious. Also Bitburger 0.0% is pretty good. Really don't like Heiniken or the other mainstream beer brands that have checked the NA box.
As far as NA cocktails go I really like anything with Cayenne in it or a bit of spicy or sour. Kombuchas are also great for sipping I find. I must warn against Pathfinder NA spirit - I don't know what they put in it and if you read the ingredients I'm not even sure they know - but I've noticed a mental fuzziness the next day that I don't care for.
Also if you are bar staff and someone orders an NA they may want it in a glass even if you serve cans, so that others don't realize they're not drinking. As a non-drinker it's super nice to be able to stand in a group with a glass of beer without any questions. This works particularly well if it's a trade/industry event where the group doesn't know you and they're all using booze as a social lubricant. I'm a little sneaky in this respect because I'll go to the bar solo, grab myself a NA beer in a glass and I've been in groups where they're exchanging drinking stories and I'm just like "uh, yup, cool man."
If you're curious how one can work bar while being a non-drinker i.e. how do you know what you're serving? In Washington state it's illegal to drink while serving bar, so our team tastes and spits when needed. I do the same - so I'm basically tasting drinks for customers e.g. making sure wine isn't corked, making sure a cocktail is made well - but spitting and after 3 years just have absolutely no inclination or desire to swallow any alcohol anyway. But as I said, any barman in this state has to be a non-drinker on the job so it works out.
I've never found a NA wine I care for. Would be great if an Athletic-like brand emerges in that space. LMK if you're aware of one.
> alcohol hasn't just permeated the fabric of society, but is the fabric in many situations
First, congratulations on three years clean. I respect anyone who can cut things out that they find problematic for them. However, the framing of alcohol as a social necessity isn't really true anymore. Alcohol consumption is in a widely publicized decline and alcohol companies (similar to mobile games in this regard) are aware the majority of their revenue comes from anti-social heavy drinkers. Just about the only circles I'm aware of that still run on booze are college greek life.
> who can cut things out that they find problematic for them
This is worth some examination.
“The greatest trick the Devil ever pulled was convincing the world he didn’t exist” —Charles Baudelaire [Although the original is French]
I think the base state of human existence is eating healthy food and drinking for hydration. On top of that we've layered eating candy, drinking alcohol, smoking cigarettes, vaping marijuana, and we've found other fun and sometimes dangerous ways to exploit our biology for our entertainment.
In the West's free market economy the purveyors of alcohol have convinced us that the base state - in other words normalcy - is to drink booze. Hence the term non-drinker. And thus the attitude that if one doesn't drink, they must have a problem of some sort, since drinking is what normal is. If you don't drink in the West, it is worth remarking upon and calling it out: You are a non-drinker. If you do drink, you are not called out as a "drinker" - you are simply unremarkable.
Which is brilliant if you're selling alcohol. The normalization of drinking has sold kiloliters of alcohol every hour of every day globally for decades, and the received wisdom that non-drinking is a pathological state, keeps the drinker drinking, lest they be identified as having a problem, or as alcohol being "problematic for them".
It's amazing that the alcohol industry is in decline, given what they've managed to pull. I'd say once society's values shift enough, they're in for a very rough ride.
That is, once you are a drinker, and I am unremarkable.
In a broad range of human civilizations, from the Sumerians to the Romans to the Inca, daily drinking was the norm. Has nothing to do with the "West's free market economy"
Hemp [κάνναβις / kánnabis] grows in Scythia: it is very like flax; only that it is a much coarser and taller plant: some grows wild about the country, some is produced by cultivation: the Thracians make garments of it which closely resemble linen; so much so, indeed, that if a person has never seen hemp he is sure to think they are linen, and if he has, unless he is very experienced in such matters, he will not know of which material they are.
The Scythians, as I said, take some of this hemp-seed, and, creeping under the felt coverings, throw it upon the red-hot stones; immediately it smokes, and gives out such a vapour as no Grecian vapour-bath can exceed; the Scyths, delighted, shout for joy, and this vapour serves them instead of a water-bath; for they never by any chance wash their bodies with water.
— Herodotus' "Histories", passage on hotboxing Scythian tribes, written ca. 2,460 years ago
I'm not sure what you mean by running on booze (perhaps you mean daily drinkers), but I think this is real consensus bias to think drinking is not still fairly common socially even after college.
Edit: Deleted. Been oversharing a bit re research I'm doing, and the payoff is replies from folks who haven't bothered to go and take a look themselves, which I then have to spend more cycles refuting, etc. So best to just go back to the first step and not overshare and recover the cycles I'd spend on the rest of it. Admittedly I'm tired and pissed off, but yeah. HN won't let me delete this so I guess it's just a deletion edit. Sorry.
I've never understood this. There is no "gun show loophole" anymore, if there ever was. Some states have two different standards required things like background checks and identification for gun sales between private party sales and dealer sales. If a dealer goes to a gun show, they have to background check their buyers just like anywhere else. Similarly, if a private party (in a state where they're not required to background check) sells a gun on Craigslist, they're equally unrequired to background check.
Many states, including most of the "anti-gun" states, have moved to requiring background checks from all sellers, including person-to-person transfers and even gifts from family.
The "gun show loophole" is massively overblown. There's nothing special about gun shows in it.
That counts as many states, and most of the "anti-gun" states, which is what the parent actually said.
Personally, I think people should be as free as possible to sell goods privately without the government getting involved. It's not a loophole, it's how things should work.
If you think any of these laws prevent felons and other prohibited persons from getting guns, you must be remarkably unfamiliar with felons, and their willingness to commit felonies. Most felons I know through friends/family have a gun (often stored somewhere plausibly deniable), and it's not a particular secret.
These laws primarily harm law-abiding citizens — who were never the problem in the first place — far more than they restrict prohibited persons from acquiring guns.
The same thing will occur with restrictions on open models, but arguably the results are far more harmful — limiting the technological and economic capacity of the people and countries we have to worry about the least, leaving the playing field open for those we have to worry about the most.
Fun fact - habitual drug users are no longer blanket prohibited from possessing firearms in the US, at least according to UNITED STATES v. HEMANI, decided by the Supreme Court this summer.
Of course it's not quite that simple. Among other things, the form to buy a firearm still asks if you are a user of controlled substances, and lying on the form is presumably still a felony.
The opinion is still a fascinating read and IMO a huge step forward.
That's a tautology though. You're defining a set of people by their illegal behaviour and then saying "all of these people have a demonstrated comfort with a capacity for ignoring the law". Well, yes, obviously.
My point earlier was that people don't neatly sort themselves into "law-abiding citizens" and "felons" until after they've committed a crime. A law-abiding citizen can turn into a felon at any moment.
And, of course, this is in the context of the gun control debate, where you can't issue guns to only law-abiding citizens because you have no clue which of those law-abiding citizens are going to commit a felony and magically become a felon
Of course "a felon has previously committed a felony" is tautological. But "a person who has committed a felony is substantially more likely to commit serious crimes in the future" isn't.
It's a well-established empirical claim, and that's why they're prohibited persons.
As for magically predicting whether someone will become a prohibited person: we can’t, and even if we could, that would mean preemptively restricting a constitutional right.
Right, so your future felon is a law-abiding citizen now.
> These laws primarily harm law-abiding citizens — who were never the problem in the first place — far more than they restrict prohibited persons from acquiring guns.
When you sell a gun to a law-abiding person, you have no idea if that person is going to become a felon or not. There is no hard line between the two, any law-abiding person can become a felon at any point.
This is why the rest of the world restricts people from owning guns.
I'm a cybersecurity 'expert' and my 40 person team and I make our money by providing GPL software to the world that I wrote.
You have the same incentive you had to share your work as before, and that is to get attention, and customers, assuming that's your game. Open code means no vendor lockin for a lot of customers, so they can pay you but also trust you to not extort them. And if you're hit by a bus your product lives on, and someone else can adopt it.
Supply chain risks and vulnerabilities existed before LLMs came along. They're easier to handle now that we have LLMs helping us. In our org the tsunami of updates we need to do weekly is far more easy to handle with LLMs.
Open code has always been easier to find vulns in vs closed. AI didn't change that.
Yeah the slopfest is real, but easier to deal with thanks to agents/LLMs so it kind of offsets itself.
Your licenses are still enforceable in court. Agreed that not being able to reverse the fact that AI trained on your code and is selling that capability kind of sucks. But humans were doing that before AI.
Yeah on the one hand opening your code got you credit which was nice for the ego and for getting paid, and AI trains on it and doesn't give credit where it's due. But on the flip side, we get AI! Which I frikkin love. I feel like a kid in a candy store. It's training on my code too and it's training on my content. And when people ask about what the best product is for our space, the AI tells them its our product. Woohoo!
Regarding the future: We have some really really big problems that need solving - stuff that creates a massive amount of misery in dark stuffy hospital rooms with crying relatives saying goodbye to their 9 year old child. I've been in those spaces and I'd give up the previous generation of open source ethics in a heartbeat to make just an ounce of that misery stop. The training that my code provides AI is a tiny little part of that solution, and I'm proud of that. Whatever I can do to push our capabilities to the point where we can make the major breakthroughs this species needs, I'm happy to provide.
Hey thanks for WordFence! It made my early days as a programmer a bit less crazy. Having clients constantly installing plugins (backdoors) in their websites was a never ending battle. Don’t really touch Wordpress stuff anymore but it was definitely my favourite plugin back then.
“Before LLM’s there was_____” I see this whenever an LLM’s impact is assessed. We know. The issue is scale and the ability for smaller and smaller groups (down to individuals) to execute at scale.
LLM’s are pouring massive amount of gasoline on existing issues and people just keep shrugging.
Fake news always existed. Now one dude in India can flood multiple sock puppet media accounts with right wing content/images (actual example from a few months back) at a scale previously unimaginable.
People could always die crossing a street. Still, cars changed the discussion about pedestrian safety pretty materially. People didn’t simply throw up their hands and go “people have always been able to die crossing the street.”
A guest on a podcast said, in response to open source and community websites being flooded with AI scrapers, "the internet has always had people scanning websites; get over it".
I still respect the podcast, but that was such a shitty take. The difference between before and now was that those websites started closing their doors instead of paying the increased hosting feeds.
I'll get over it when we stop giving people a pass for the damage they're doing just because they're a corporation.
That is kind of net loss at this point. Hey, on the bright side we get onslaught of slop, ai psychosis, constant stream of doom trolling.
And ideology of pointlessness where any time you do or learn anything, you get told that why bother you should have used AI.
The kid forever locked in candy store is happy for a bit, but then they get hungry and feel bad. And no matter how much sugar you eat, it wont get better.
From an ego perspective, the artisan craft of programming basically being dead at this point makes me sad, having spent so much of my life getting good at it (or at least trying to).
From technological enthusiast perspective - if you can't find an endless stream of uses for this technology, I really don't know what to tell you anymore. I gave credence to AI naysayers for a while, but at this point I feel like its akin to denying gravity exists.
Who are we to gatekeep software development? So many people have gained the ability to interact with computers in ways they never dreamed of before! For me, I've never felt so engaged in software development, even if I'm no longer writing it line by line.
Like you say, there are a lot of negative externalities to this technology, but its something we're going to have to solve rather than dismissing it outright. The industrial revolution caused all sorts of problems! But you can't argue we're worse off because of it.
I don't think it's dead if you have a market which values high-quality, artisanal products.
Sure, artisanal code in itself isn't something inherently sellable but if you pride your program on being a quality product then it's still valuable for others:) AI is basically the new JavaScript in a way.It won't create the next Linux or the next SQLite by itself.
By this point, if you care about a quality product, you should learn how to leverage LLMs for that like running automated audits for correctness and performance opportunities.
There isn't a market for "yeah there's a memory leak but I wrote it by hand."
Yep! We run security reviews on our pull requests now and are shocked at how it stops a lot of vulnerabilities being shipped. We've had a couple of high score CVEs from the before-LLM times, and when the AI reviews the code that introduced the CVEs, it easily picks them up. We had 2 humans reviewing every PR, and both missed the issues. It's far too easy to miss security issues when you manually review them, but LLMs are exceptionally good at finding them. Unfortunately for me, I admit, I just can't get myself to push code anymore without an LLM checking my work (or writing much of it when I'm at work, I try to write code by hand in my own time to make sure I don't rust away, but at work there's no way to justify doing it the "slow" way anymore).
Fair enough, but as things stand now, the usual LLM-assisted piece of software is usually also partly or wholly LLM-designed too. Not just implemented on a function level or a file level. And LLM design is usually called "slop" because it's nothing spectacular unless you bring fresh ideas to it from a human perspective.
Hand-written memory leaks don't have a market but hand-designed software with hand-designed UX and a hand-designed vision does :)
I'm generally impressed with the code that Fable/Opus is writing for me these days; I would be proud to have had the same foresight had I implemented the solution myself.
And Fable's architectural design is pretty much always well-reasoned and a good place to start.
There's this idea that the best way to use LLMs is to be in the backseat constantly yelling out corrections, but that hasn't been true in my experience for quite some time, though I only use a few sota models.
I think something being slop this late in the game is mainly a reflection of the person using it. I can't really blame AI anymore when pretty much any lever you'd recommend to de-slop it is one prompt away.
I agree with your last bit, and that is the only thing left now that AI solved the technical part.
Thing about being in cybersecurity is you get to see outcomes that aren't a matter of debate or taste. For example, if I lock an agent in a jail and tell it to attack something and that the only way to win is to show me a number stored on that target, and it succeeds, then assuming our jail was effective, it's an outcome that isn't debatable. I've lost count of the moments I've had this year where my jaw just drops because I'm holding undeniable proof of a level of expertise I've never seen in humans - and that is far above human capability, in a field where I'm an expert.
So I guess from my perspective, it's not a candy store or candy. It's something that can solve problems we've never before been able to solve. Problems that are too hard for a human.
Another example: Recently one of our agents found a vulnerability so complex, that our team, who are experts in their field, could not understand it and had to ask an agent to write a blog post to explain it to them. It had more steps in the exploit than we've ever seen, and would never have been discovered by a human.
Cybersecurity is a leading indicator of what's to come in other fields. Leading because programming is something models are inherently good at. Doing wetwork in a lab is harder to plug into a model or agent. But it's on the horizon. So we will be seeing these kinds of breakthroughs in other fields, and the leading indicator says they're going to blow our minds.
If you think this stuff is candy, and you're relating it to social media, you're simply not paying attention or getting your hands dirty. And honestly if I wasn't hands-on, every day that passed would make me progressively more scared and more angry as it pulled away from me.
And frankly, I think most of the source of outrage is this:
> Yeah on the one hand opening your code got you credit which was nice for the ego and for getting paid, and AI trains on it and doesn't give credit where it's due.
Turns out, a lot of people didn't really do things in the open to benefit the others, in pay-it-forward style. They just did it for selfish gains. Which is fine, just like keeping source closed and selling licenses is fine. The problem is with lying - doing something for personal gain, while claiming it's for greater good, thus getting more gains through dishonesty.
LLMs just shone a light onto it. People who had betterment of others on their minds, don't have a reason to consider LLMs training on their output as taking anything from them. On the contrary, their outputs now contribute to a general-purpose problem solving tool that will (and already does) help humanity with way more problems that anyone imagined.
I personally am more than happy to know LLMs may have trained on my content. I don't begrudge the companies the $0.000001/year they probably owe me for my relative contributions. I get orders of magnitude more value for myself from LLMs every day, in my personal life alone.
There are other reasons to dislike LLMs and fear or hate what AI is doing to the world. But people who feel something was "stolen" from them, who now close their blogs and turns repos private because LLMs - they're just showing they had ulterior motives for their work - which again, is fine if they were up-front about it. The OSS subset of those, just paint themselves as being grifters all along.
> There are other reasons to dislike LLMs and fear or hate what AI is doing to the world. But people who feel something was "stolen" from them, who now close their blogs and turns repos private because LLMs - they're just showing they had ulterior motives for their work - which again, is fine if they were up-front about it.
100% this. Why should I be bothered that some of my work that I released freely to the world is now potentially used by millions of people as a small part of the knowledge in a state of the art AI system? I am honored!
But I guess some people just have different motivations for releasing their code, and require explicit attribution to feel honored enough to make it all worth it for them. Not me, though.
You have no proof that the world will be a better place because of this stuff but there is plenty of proof already that it will be worse, possibly significantly worse but the jury is still out on that.
"AI" -- as in agentic workflows -- have been around for a little over a year. it doesn't seem like enough time to "prove" anything conclusively to me, what makes you so confident?
> Turns out, a lot of people didn't really do things in the open to benefit the others, in pay-it-forward style. They just did it for selfish gains. Which is fine, just like keeping source closed and selling licenses is fine. The problem is with lying - doing something for personal gain, while claiming it's for greater good, thus getting more gains through dishonesty.
> There are other reasons to dislike LLMs and fear or hate what AI is doing to the world. But people who feel something was "stolen" from them, who now close their blogs and turns repos private because LLMs - they're just showing they had ulterior motives for their work - which again, is fine if they were up-front about it. The OSS subset of those, just paint themselves as being grifters all along.
thank you for putting such clear words to a feeling that's been troubling me about this outrage for a long time.
I have been getting occasional paywalls from the BBC asking to pay to read articles. Clearing cookies does remove the paywall for a while so I think it's some sort of A/B test they're running.
I would put love above health. Suppose you are perfectly healthy and even rich but have no friends to share anything with, life must be meaningless then.
The two are related, bad health makes it difficult to have meaningful relations and lack of meaningful relations fucks up your health. Smoking a pack of cigarettes a day is worse for life expectancy than loneliness.
I'm sorry to read that. In fact I meant love in a more broad sense. There are many kinds of love, it does not need to be having a spouse. I work every weekday mainly for having good relations with people, money comes second.
But coming back to your story of life (ignoring the last sentence) : it's not too late to find love and also: try knowing and loving yourself, being content a bit more. When that succeeds, it is more easy to find love.
The first half of your message led me to write a well meaning reply to tell you you still have time to be happy and meet that girl from high school or someone else. The second part is so dark and evil that I regret wasting my time.
As with many things, it's easier to stay healthy if you are wealthy.
You can buy personal training services, workout gear, a home in traffic-calmed neighborhoods where you aren't jogging or cycling along rural highways, a pool, etc.
You can buy fresh fruit and vegetables instead of highly-processed food. If you're American you can buy imported or certified organic meat instead of the hormone-filled shit on the shelves at grocery stores. You can spend less time working and more time cooking healthy, with no pressure to eat fast food because you're in a rush.
Statistically, yes. And statical health things are the only health things to really worry about (because you can't ever do things to avoid getting cancer, you can only ever do things to decrease your chance of getting cancer).
But then again, the list of things that statistically keeps you healthy while being expensive is actually very short (GLP-1s currently being the most notable item on it).
Supposedly this is going to change in the near to mid future with more wonder drugs incoming. But, for the time being, that's nice!
I mean if I had someone tailoring workouts to me, keeping me motivated to stick with a training regimen, and free time enough to get super fit and look great naked, I'd sure be happier.
You are looking at it from the wrong perspective, you are trying to fit "healthy" into your current environment. I think it's the wrong approach.
For example: societies that live the longest and are generally the healthiest do not have gym equipment or trainers, exercise is a core part of their every day life.
The mistake you are making [I think] is looking at these things as impossible goals because of lack of more money so it is easy not to try and risk failure.
I accompanied a family member in an ambulance recently and I had the ambulance rerouted to a private clinic. They needed a €25k payment guarantee but we were able to go into surgery within a day.
Usually yes. But people will also sacrifice their health for their loved ones, for example by working hard unhealthy jobs, or for a mission that's larger-than-themselves.
reply