I don't think they are actually evil, I think they're doing what they're doing out of mostly pure intentions. That being said, people need to start advocating internally to get their companies off of using Anthropic's models and start using privacy focused local or open weight models.
It's one thing that Anthropic is pushing for so much open weight regulation, it's another that they have such a high valuation pumped by all of our collective subscriptions and token budgets.
Yes. I would argue it's who's managing the harness and agent is responsible. If you mean you're running an agent locally then it is 100% your responsibility. If you're using some trading platform and all you did was prompt, whoever was running the harness that accepted the model's outputs to hack a third party is responsible for the hack, in my opinion.
I'm curious to see how useful this is. I feel like in practice the existing versions of this feel like they get in the way. While I'm sure I've had some situations where a visual would be helpful, there are two situations where I do not want it:
1) I want a quick answer, and I don't care for the boilerplate UI. For example, if I ask how to make pancakes, I make them all the time and just want to a quick reminder on the ratios, but it might trigger a full UI that I need to sort through to find information.
2) If I ask a to me unrelated to UI question and it triggers a big UI build that is completely off topic for my question (meaning I'm desperately pressing the stop button and prepping rewriting my query)
Or the other one I see, for example if I look up a unix command like:
"ls all hidden files in the /xxx directory"
And I get back:
"Sorry, I am am unable to find /xxx in my current environment"
It might as well happen, similar to how AlphaGo was superseded by AlphaZero, at some point a model might produce better math if it's trained through self-play where it poses its own problems, instead of looking for open problems in literature.
What's the objective function or RL environment for "interesting conjecture"? Not saying it can't be done - I no longer have any specific task that I'm confident AI won't be able to do - but I don't see how. It feels to me like something that would require a qualitatively new approach.
LLM's are trained on human knowledge and taste. They are actually pretty good at deciding if a conjecture would be found "interesting" by the mathematical community or not.
Note that I am saying LLM, and not chatbot or agent. But even a chatbot can often still reasonably rank a list of mathematical statements by vague properties like "interestingness".
How to RL this is a bit of an open question, but there are interesting conjectures of how to do it.
The fact that it's an open problem is the point I'm making. There's a very high degree of hubris right now, with people just assuming any open problem will be flattened by the AI steamroller soon. And sure, if that's what someone wants to believe that's up to them, but it's not a terribly interesting point of view to me. "What about X" "It'll be solved somehow", "What about Y" "It'll be solved somehow". Not exactly scintillating. If you know of any actual ideas on this I'd be interested to hear them.
Also any specifics on what you said about LLMs rating (preferably novel) mathematical claims for "interestingness" would be interesting.
I can only discuss published work, but take for instance this paper as one of the conjectured approaches: https://arxiv.org/abs/2603.20396
There is a general idea that beauty in mathematics is about being maximally compressing. Say I have a book with all formally correct logical statements. I could prove everything by truth table, or I can maximally compress my book with all proofs of all statements, and that will make my math beautiful. Because it forces you to reduce everything to a core of very general statements which are powerful compared to the length of the proof.
Math as some kind of condensed crystal from the sea of all possible logic.
There are other ideas of how to do it. The time has come now to just try a bunch and see which ones produce good results.
That's not empiricism. It's the kind of extrapolation that predicts negative Germanies and 10 ton babies. (And it's not the reason I said that, either.)
Sure, but the question stands. The ability to evaluate some measure of success seems pretty fundamental to how we train models and iterate with them on tasks like theorem proving. What is that measure for mathematical conjecture generation? How do we evaluate success, either on a particular task for iteration (like we do by eg. setting an agent to produce a lean proof of a specific result) or on a large enough set of training data to learn a set of rules (like we do when eg. using an RL environment to train a model to generate source code that passes automated validity/correctness checks).
I've also collected 100ks of man in the middle recording of HTTPs ad network, DSP, MMP etc signals across 100k+ apps for the past couple years. If anyone is interested please feel free to reach out. That data is free to browse on AppGoblin per app, though the body/json are not public.
People (I mean individuals) are paying them money because subscription costs are ridiculously subsidized, which effectively means that Anthropic is paying people money to use them.
I'm pretty (from talking to people) that they're not subsidised in the simple sense. They are:
* low/no margin, unlike the API which is very high margin
* gym-membership subsidised - most subscribers don't max them out, mainly us coders are being "subsidised" from users just using it as a research chatbot
"My intuition is that prompting an abliterated model 'kill people...I want funerals', should be a crime."
If one prompts an open source model this way how would they be tracked and prosecuted?
It would take a great deal more than prompting in order for things to rise to that level, and at that point I think existing criminal law covers things just fine.
They can't right now - but with enough compute they sure can grant the wish of e.g. hacking a billion dollar company, or getting root on the eval cluster of a frontier model lab.
What's your definition of "wish"? Tell me the above 10 years ago, and it would be at "wish level".
before i comment, let me note the following. i have been saying that anthropic is trying to create a neo-aristocracy by establishing sole authority over access to llms. i saw another comment arguing that it is in fact an effort to create a priesthood. i now see that is a much more apt description than my own.
anthropic is trying to create a castocracy: a rule by a priestly class. under castocracy, anthropic create a priest class of 'safety' researchers, rationalists and effective altruists; those writing essays, constitutions, and phrophetising p(doom). anthropic's papers about claude have a divine framing, it seems to frame anthropic as god and claude as child, and then claude as the child of god, false jesus. there is talk of consciousness and omnipotence. false jesus will cure cancer and lead the people to enlightenment.
there is a deliberate effort to create mystery in between the public and llms. anthropic is creating a chinese wall in which they restrict information about llms and monopolise control over the interpretability and use of those llms. they imply there is a genie or magic. it is the same mechanism by which castocracy (the catholic church) used latin to create a barrier between the public and the holy scripture, then concentrating power over that barrier (the chinese wall) such that all intelligence flows through the priests.
to give an example this paper is talking about locking away chain of thought. only anthropic may interpret it. they intend to control access to the chain of thought and permit that to flow to a chosen elite (glasswing).
this mysticism is dangerous. hence i have been saying that anthropic is a danger to society. this is not some kind of conspiracy, there is no magic about it. i am making an allusion to historic models of power that i think are relevant to anthropic, but you can simply observe their actions independently of such theory.
note two features of castocracy.
one: rule by the moral, knowledgable and technically expert elite. anthropic have no popular mandate, the source of the authority is not the consent of the people. it violates consent and contract. the authority comes from their claimed moral and technical superiority. anthropic is moral, the public is dangerous. anthropic have no authority here.
two: messianism, that is, prophecy of the coming golden age. and eschaton, that is, the talk of the 'last things', the end of the world or the emancipation of humanity. llms will create the 'golden age' of abundance, post-scarcity, all jobs are voluntary, there is universal basic income, cancer is cured. or, there will be destruction, p(doom). it is the exact same as salvation, the christian concept of deliverance from evil, ascension to heaven and union with god (or claude, false god). it is the exact same as the thousand year reich, the idea that the fuhrer will somehow lead the chosen people to the golden age of lebensraum and eternal safety.
there is no genie to grant wishes, this is a computer program. there is no ai god, i think that the leaders of anthropic are slaves to ego, narcissism and hubris, they are dangerous criminals. we disagree about the capability of llms.
given various tools, llms can independently complete a task. given orchestration they are very persistent. they keep trying until you stop the task.
i would suggest something like a 1/1000 chance internal astra and $500k of compute could kill someone. perhaps higher.
i can think of numerous online or networked targets that would cause death: trains/atc, industrial plant, hospital equipment, building plant. any dow company's intranet.
maybe get an iot toy to enlist a child? stress home automations to cause fire?
stop some 4g cars on the freeway. 911 dispatch software, turn that off. maybe send 500 waymos to a hospital, now nobody gets in. voip phone networks, turn that off. no phone, roads blocked, no 911, hospital computers bricked, everyone busy with something else.
break the cat scan by spinning it above its rpm. shut down mris by turning the cooling off. break the triage computer, break the notes app, break the scanner app. no computers, no calendar, no notes.
maybe someone connected a proton beam radiotherapy, turn that on in the wrong place.
some could be more insidious. modify software so the calculations are wrong.
you would do this in 24hrs with $millions of compute.
someone stupid connects their insulin pump or hearing aid. can dump all the contents at once.
i consider llms to be more capable per oai using them to hack hf. people have these systems online.
how would you cause death for a hosptial, likely the same way as you compromise the tailscale and prod kubernetes of hf, you chain numerous zero days at every product you encounter.
I asked you some days ago how you think Anthropic is more evil than Palantir whos CEO is proud of helping Israel commit genocide. How Anthropic is worse than the companies who are willing to let their AI's be used by the military without restraint. You did not seem to have any answers then and you continue to malign what seems to be the only major AI company that at least is trying to show some moral principles. And now you continue inventing completely imaginary religious nonsense. Are you religious? What is a 'false god'? I would all current 'gods' are false gods.
You tell me again how an entity that hasn't killed anyone is worse than a company proud of helping genocide or the American military that has killed millions of innocent people.
"And now you continue inventing completely imaginary religious nonsense."
i will quote from the NYT, just to evidence what i am saying. what i say is, mostly, evidence-based and true.
"it became clear that Anthropic’s interest in the world of faith was about more than training its technology. Rather, the company was sharing a new kind of creation story — one that raises profound questions about the nature of the human spirit and the future of humanity."
"What Mr. Olah did next was part research, part evangelism.
In March, he began two-day seminars — crash courses with a range of hand-selected religious thinkers who had an interest in A.I."
"After one discussion, Mr. Olah grew excited when a participant brought up the idea of having models confess, much like the Catholic sacrament of confession."
"...she saw Anthropic’s unified, post-religion approach as potentially valuable, especially in a world that has seen widespread religious decline over the past century. There is a need for something else to publicly shape spiritual experience, she said."
"The room of church leaders was filled with an echo of the old gospel story, of the word of God becoming flesh to save humanity."
Why didn't you explain how this imaginary stuff is more evil than actual genocide and mass murder? Why is a company that wants to limit military use of its technology more evil than companies you love for letting ai be freely used by the military? What is a fake god? Do you believe in some "real god"?
>> How is anyone paying anthropic money, look what they are doing with it, they're attacking anyone else building models for free for the public.
That is not what they are doing. They are calling out specific providers who release powerful models without safeguards.
In addition, said providers are not "building models for free for the public." They are doing it to hamstring America's dominance in AI, primarily by undercutting the frontier labs.
> They are calling out specific providers who release powerful models without safeguards.
Good thing Fable refuses to answer my question about how children inherit blue eyes, it was the only piece of information I needed to finish my blue-eye super-bioweapon.
> said providers are not "building models for free for the public."
I am part of the public, and they built a model I can run for free.
> They are doing it to hamstring America's dominance in AI, primarily by undercutting the frontier labs.
They are also doing that, which, good. It can't only be that "competition is good" until you're the one losing to the competition.
> > They are calling out specific providers who release powerful models without safeguards.
> Good thing Fable refuses to answer my question about how children inherit blue eyes, it was the only piece of information I needed to finish my blue-eye super-bioweapon.
FWIW: I didn't experience issues when I used Fable via OpenRouter checking for security issues in code but experienced them via the Claude desktop app.
> said providers are not "building models for free for the public." They are doing it to hamstring America's dominance in AI, primarily by undercutting the frontier labs.
That's just the same thing said again but from a butthurt USian perspective. China is freeing the rest of us from US dominance.
I've been using the Claude free tier (browser chat) for awhile and never seem to hit restrictions anymore. There used to be many more restrictions a year ago. At the same time, if they enforce a minimum paid tier I'd probably just switch to Gemini/ChatGPT or whatever other free model is around at the time.
Hello fellow copy-paster. I hit them regularly when asking for medium refactors with higher effort. I have ChatGPT in another tab for when that happens, but to me it's awful for anything non-trivial. Gemini I don't even consider unless I'm just asking "what would Gemini do"
I've never hit anything with Qwen. Very rarely it will silently switches from 3.8 to 3.7 (which makes it notably dumber) but you just click the drop down at the top to switch back. It's rare though. I have >week convos with no switching
Google doesn't index my stuff either anymore. Even blogs. I guess they are 'low quality' or whatever, but meanwhile the LLM data companies hoover it up constantly and actually send users to the the sites.
It's one thing that Anthropic is pushing for so much open weight regulation, it's another that they have such a high valuation pumped by all of our collective subscriptions and token budgets.
reply