Not $400 a month, it doesn't. At least not around here (we average $0.12/kWh and I don't personally run my cards over 300W.)
And there's a lot of labour and effort involved...
Theoretically, the people who hang around HN are more likely than most to be capable of the labour and effort of setting these things up and maintaining them.
People don't even run their own email servers...
People don't run their own email servers because a convenient coalition of spammers, standards bodies, and large email providers have done their best to make running one's own email server almost impossible.
> Theoretically, the people who hang around HN are more likely than most to be capable of the labour and effort of setting these things up and maintaining them.
Theoretically, the people who hang around HN have immense opportunity cost when doing this. Being capable does not mean it takes no time. Time they could be using for something more profitable (and fun?).
That's one way to look at it. Another way to look at it is to point out that the potential cost of delegating cognitive resources to companies like OpenAI and Anthropic is unbounded.
The AI models we currently have still maintain their working state with a finite and laughably-small context window, but that is already starting to change. My .claude directory contains over 300 .md files that I didn't put there myself. Their contents are eye-opening. The question of who owns, stores, maintains, and can access that data is going to become insanely important over the next couple of years.
If you thought LLMs themselves were disruptive and contentious, just wait until the fight over object permanence gets under way. That's when owning your own box full of graphics cards is going to become important. My own bet is that I won't care too much about the electric bill or my opportunity cost when we all find out what the AI labs really have in mind, and what they're going to have to do in order to justify the valuations they're seeking.
> The question of who owns, stores, maintains, and can access that data is going to become insanely important over the next couple of years.
That's a good point. But you already have one possible answer: use an open source harness and call out to open-weigh models only. Then you can own all the relevant data. Still without any need to own a box full of GPUs.
> Theoretically, the people who hang around HN are more likely than most to be capable of the labour and effort of setting these things up and maintaining them.
'Capable' doesn't mean your labour has no opportunity costs.
I think my argument is easier to attack by noting that you can use AI to substitute for much of that labour.
That's what I did personally, not being a Linux guru. But to your point, I still had to spend a lot of time and effort dorking around with the hardware.
It's worth it, knowing that there are no rugs Sam or Dario or anyone else can pull.
> It's worth it, knowing that there are no rugs Sam or Dario or anyone else can pull.
At the moment, it's still very easy to switch from one open weights model to another, and even between the closed models. So the 'no rug pull' property is nice to have, but not as big of a deal.
Power averages $0.30 to $0.45 for a business in the UK if you convert it to dollars. Smaller businesses without load shedding agreements are at the middle / top of that. (there are lots of people complaining about it). Maybe I just have expensive power, but $0.12 per kWh is pretty cheap.
Not $400 a month, it doesn't. At least not around here (we average $0.12/kWh and I don't personally run my cards over 300W.)
And there's a lot of labour and effort involved...
Theoretically, the people who hang around HN are more likely than most to be capable of the labour and effort of setting these things up and maintaining them.
People don't even run their own email servers...
People don't run their own email servers because a convenient coalition of spammers, standards bodies, and large email providers have done their best to make running one's own email server almost impossible.