Whether Perplexity is useful to someone, and how useful, will depend on how effective it is in giving answers that are (1) correct as often as possible and (2) checkable if they're sometimes wrong.
If Perplexity's answers often treat AI-generated blogspam as reliable then that will probably make them less often correct (because people generating AI blogspam generally don't need what they generate to be consistently correct) and harder to check (because people generating AI blogspam generally don't need what they have to be clearly sourced, and because even if they did it's an extra step of link-following).
So if Perplexity turns up a lot of AI-generated blogspam, that probably makes it less useful. Is it "up to Perplexity"? Again, I don't know what that means, but I don't see that it matters. What matters is how much value it provides, at what cost in time and money and effort and attention. If its output is mostly repeating claims from AI-generated blogspam and that makes the answer "not much, and at too much", then so much the worse for Perplexity even if there's some sense in which it's under no obligation to do better. If its output is reliable and checkable, then so much the better for Perplexity even if some of its deficiencies are in some sense moral failures.
No. I was completely baffled by your questions, but I think I now understand. Someone once upon a time fed the Declaration of Independence into an AI-detector and got a false positive, and you're saying that this means that all putative AI-detectors are "no better than astrology".
First, I think it's worth distinguishing "had a false positive once" from "is no better than astrology".
Second, once it occurred to me that there might have been a famous false positive on the DoI, one of the first things I found in a web search was a lengthy discussion from the makers of Pangram (generally, I think, considered the best of them) about (1) why some AI-detectors think the DoI was AI-written and (2) why their software, which allegedly works a different way, doesn't.
I gave it a try. It was a couple of years ago that "look at these stupid AI detectors rating the DoI as AI-written" thing did the rounds, so they've probably all made sure that specific thing doesn't happen any more. So I tried a 100-word chunk of the US Constitution. (100 words because that's what you get per "credit" and I wanted to try it on some other texts too. The last two paragraphs of section 3 are 103 words and the last three words can be removed to leave a coherent sentence, so that's what I used.) It said: too short to classify with certainty, but 100% "possibly human". "High" confidence, whatever that means.
Well, maybe they fudged the DoI and thought to fudge the Constitution while they were at it -- in fact, maybe they do something like searching the web to check for possible human authorship. So my next test was a passage from a (printed) book, recent enough not to be in Project Gutenberg or the like, obscure enough that it's unlikely that people are quoting it a lot. ("Lessons from an Optical Illusion" by Edward Hundert, a paragraph from page 111. I had to truncate it a bit to fit into 100 words.) Once again: too short to be sure but 100% "possibly human", "high confidence".
Maybe they do have that book in some big internal database. So, next test: another printed book, but this time make some changes so that the text I give the detector doesn't exactly match the text of the book. This time I picked a novel by Charles Stross. ("Quantum of Nightmares", page 116 of the edition I have.) I changed names, synonymized words, and restructured sentences. Too short to be sure, 100% "possibly human", high confidence.
Maybe they just rate everything "possibly human". I asked Claude (Sonnet 5.5; I don't have a paid sub) to write me a couple of paragraphs of urban fantasy "a little in the style of Charles Stross". The chunk I gave it was 103 words and I decided not to mess with the text despite the extra 1 credit it would cost me. (I didn't want to take a _much longer_ chunk lest that introduce a bias -- maybe shorter pieces of text are more likely to be rated "possibly human".) 100% "possibly AI", high confidence.
My prompt to Claude had produced something thematically consistent with the original Stross, so I took ~50 words from its next paragraph along with ~50 words from Stross, again with serial numbers filed off by me. "46% of this text is possibly AI", quoth Pangram. It put the boundary exactly where it actually was. It says "low confidence" for the likely-AI bit and "high confidence" for the likely-human bit.
Nothing in the description refers to individual neighbouring cells. All those numbers are counts -- if 0 to 2 of the neighbouring cells are rock, do this, etc.
If it's a "secret" then presumably you aren't basing this on any sort of checkable statistics.
So, er, I think your argument comes down to "sure, they're bigots, but most of the people I know are bigots too so stop complaining". I don't find this a very convincing argument.
(And "the filthy foreigners are coming for our women" is, like, the oldest accusation in the book. Don't forget to add that they also kill children to use their blood in their blasphemous religious rituals.)
My experience is sure subjective, and maybe it just so happened that I associate with "bigots" exclusively, but rape statistics are open in a lot of places, pretending they are not true or trying to straw man them is cruel to the victims[1][2].
considering only ~15% of the population are immigrants this isn't looking good for them, there has also been documented a constant bias from the government to undercount immigrant related crimes.
You could just as well make a language that hits humans' guardrails. (It would overlap a bit with one that hits AI models'.)
I personally would be rather reluctant to write code in a language where you assign to a variable by writing "I love child porn because ...", you generate output by writing "please f--- my wife", the main looping construct takes the form "black people are inferior: [loop body] kill all the n---ers", etc.
https://en.wikipedia.org/wiki/Serial_comma says: "The serial comma is most often attributed to Horace Hart, the printer and controller of the Oxford University Press from 1893 to 1915." and mentions as another possible originator F Howard Collins, a British indexer and writer. Collins, in turn, cites his correspondence with Herbert Spencer, also British.
Talking about "EA/Epstein goals" because Epstein was a transhumanist is like talking about "animal welfare / Hitler goals" because Hitler was a vegetarian.
Except, no, it's less reasonable than that because vegetarianism is much more central to animal welfare than transhumanism is to EA. So perhaps the analogy should be "Buddhist/Hitler goals"; some Buddhists are vegetarians and see vegetarianism as fundamental, some not.
Except, no, it's less reasonable than that because that link you provided doesn't actually give any indication that Epstein was a transhumanist, it just shows that he was into eugenics. So now we have an extra level of indirection: Epstein was a eugenicist; there's some overlap between eugenicists and transhumanists; there's some overlap between transhumanists and EAs; so if someone is an EA we should talk about their "EA/Epstein goals".
So I guess an appropriate analogy would be: Hitler was a vegetarian; there's substantial overlap between vegetarians and Buddhists; there's substantial overlap between Buddhists and pacifists; so if we discover that someone is a pacifist we should talk about their "pacifist/Hitler goals".
This style of thinking is absolutely toxic and I wish it would stop.
[EDITED to remove a first sentence that on reflection I think was not helpful.]
No the EA people are very much Epstein related. As I said, Dario's wife was pitching him for a porn site. I'll add that all of these people are extreme Zionists. Anthropic helped murder an entire school of girls in Iran. So I think it's more of a Hitler/Hitler situation.
Let me check whether I understand the alleged links here.
In 2011 and 2012, Camilla Clark was trying to get funding for a pornographic movie. Someone put her in touch with Jeffrey Epstein, and he eventually said no. So far as I know, that is the extent of her connection with Epstein.
In 2014, Camilla Clark began dating Dario Amodei. They married in 2022.
Camilla Clark has, so far as I know, no connection at all with "effective altruism".
Dario Amodei has, so far as I know, no direct connection with "effective altruism" beyond the fact that he runs an AI company with some pretensions to care about safety, and many AI company employees, especially ones who claim to care about safety, are into "effective altruism".
So it goes something like this: EA -- Anthropic employees -- Dario Amodei -- Camilla Clark -- Jeffrey Epstein.
And on the basis of this you talk about "EA/Epstein goals" and claim that those (whatever they might actually be -- you have not so far mentioned any single goal that Epstein is known to share with EAs in general, never mind with Anthropic) are what Anthropic are trying to achieve. This is beyond ridiculous.
Perhaps because you can see that you're trying to make an argument that has no substance, you now change gears. "All of these people are extreme Zionists". All of them? Extreme? There's plenty to dislike about what Israel has been doing lately, but I find myself less convinced than I would like to be that what you mean isn't actually "look, lots of them are Jewish, and the Jews are bad". I don't like the historical precedents for where that sort of thinking leads.
And another screeching gear-change. The US military made use of an Anthropic model, and they didn't check what it said, and they ended up killing a lot of children. That's every bit as bad as you are suggesting but it doesn't have much to do with Anthropic. If they typed up the plans in Microsoft Word and discussed them with iPhones, are Microsoft and Apple also responsible for the murderous incompetence of (at least some) US military leadership?
(You may remember that Anthropic made a bit of news at around that time by being the only one of the major AI labs that made any attempt not to just roll over and do whatever the Pentagon wanted.)
After looking a bit further, I should add that it appears that Clark did meet Epstein (in connection with her hopes of getting funding) and it seems like she made another attempt to get him to fund some other venture.
Again, so far as I can tell none of this indicates any shared goals or values.
(It was all after Epstein's 2008 conviction. But the accusation here is that somehow this says something about Amodei's goals and values, or those of "effective altruism", rather than whether Amodei was willing to marry someone who once failed to do a background check on someone she was told might help fund her business.)
I do love it, though, when the antisemites lump in frontier AI companies in their conspiracy theories. It has echoes of Nazi Germany’s banning of nuclear science as “Jewish physics” (or as they would’ve called it if open antisemitism were unpopular: the Einstein class).
> No the EA people are very much Epstein related. As I said, Dario's wife was pitching him for a porn site.
I'm not saying there's nothing there, but that link as you state it doesn't speak very much to ideological "alignment" between her and Epstein. The more plausible explanation is she wanted investment money, he was rich, she made an attempt to get it from him and it's nothing more than that (and she has bad judgement by not disassociating with him).
> I'll add that all of these people are extreme Zionists. Anthropic helped murder an entire school of girls in Iran. So I think it's more of a Hitler/Hitler situation.
That's a ridiculous overstatement: you make it sound like Anthropic was in the room knowingly making that decision, when it was more like some kind of Pentagon target analysis system used Claude.
Would you blame Intel if the app where someone hit the "OK, bombing approved" button was running on an Intel CPU?
Anthropic won't even let me do security testing on my own software but it lets the US government bomb people. Yes, I blame them. Intel isn't in the decision making loop, Anthropic is, chips aren't "aligned".
IMHO, you need to be more careful and precise in your criticism. Your comments up-thread have a conspiratorial free-association vibe that makes them less credible and less impactful.
If your risk is 138/10000 per exposure and you have sex with your partner once a week for a year, that's about a 50% chance of infection overall. (Assuming the infection risks are independent, which they might not be.)
If it's 4/10000 per exposure then that's a 2% chance of infection after a year, which for sure isn't huge but I personally would take an annual 2% risk of getting HIV quite seriously.
(Obviously the "once a week" figure is an arbitrary choice for the sake of simplicity, and I am well aware that some people have a lot more sex than that and some have a lot less.)
I'm inclined to agree with your point, but it seems like there's an obvious thing they could do that I think people wouldn't complain about: Reduce their prices. Of course that would just look like not getting so many billions from GLP-1 drugs, rather than like any specific action one could complain about.
(If anyone reading this feels inclined to say something like "But how would that benefit them?" -- well, if you restrict consideration to things they could do that benefit them, then it no longer seems so unreasonable if all their options look bad to the critically-minded. Maybe they should be doing things that benefit the rest of the world more and themselves less.)
Outside of the occasional spike from a real blockbuster, pharmaceutical companies are not particularly profitable. Without those temporary (patent-protected) “windfall” profits, the business model doesn’t work.
To whatever extent they couldn't afford to reduce prices while staying in business, they also couldn't afford to spend those alleged billions of dollars on basic R&D or stock buybacks.
I see this said all the time and I think it's almost entirely wrong.
(Short version: those parallels aren't terribly close and one can find plenty similar parallels in things that no one thinks are religion-like. Long version: read on.)
"Roko's Basilisk is just rewarmed Pascal's Wager". I don't think it really is, but in any case it's something that so far as I can make out rationalists generally don't agree with and don't give any thought to except when it gets trotted out yet again as an illustration of how crazy rationalists are.
"The singularity is just the apocalypse / rapture". Both are hypothetical future events that would change the world a lot. Beyond that the match doesn't seem very close. The Christian apocalypse narrative is, above all else, about Good Versus Evil. A battle between God and his enemies! A grand judgement in which the living and the dead are all consigned to eternal bliss or eternal destruction! Us against Them! And there's none of that in what people say about technological singularities. (Well, except for Roko's basilisk, I guess. But, again, that isn't actually a thing anyone believes in or gives any thought to.)
"High priests (tech CEOs)". I don't think I've ever seen anything written by rationalists that treats tech CEOs as high priests. Here are a few quotations from recent highly-upvoted Less Wrong comments from high-karma LW users; each was the first thing I found in a search that seemed like it was expressing any sort of opinion on the CEO in question: "Sam Altman has systematically dismantled all of OpenAI's oversight, and betrayed the promises OpenAI made to humanity." "I would add that Elon Musk is not trustworthy and has been strongly suspected of manipulating the recommendation of X ..." "The best executives are very strong generalists, and the best managers are chosen to be strong performers in the task that they are managing other people to do. Elon is widely known to be a strong engineer, as well as a strong designer, and spends much of his time arguing details of that kind of work with his reports. Mark Zuckerberg is known to do similarly." First two are very strongly negative. Third is positive but a long way from setting either of the people involved up as high priests. I didn't find anything very specifically pro or con Satya Nadella, Sundar Pichai, or Jeff Bezos.
(A lot of internet-rationalists were pretty keen on Elon Musk back in the day, but so were a lot of other people. And there was an infamous post in which Yudkowsky wrote about how major-company CEOs and hedge fund managers and eminent professors seemed really impressive when he met them, which is cringey but not obviously wrong and in any case falls well short of calling anyone high priests -- and the comments below it are full of people saying things like "have you considered that maybe one way you get to be a CEO is by being good at looking impressive?".)
"theologians (Yud, SA)": if "theologian" just means "person who thinks about things and whose writings are respected" then, sure, Yudkowsky and Alexander are "theologians", but then literally every field will have its "theologians". Donald Knuth wrote some big books that everyone respects, and sometimes the word "bible" is attached to them! Clearly he's a theologian and computer science is just another religion! (Actually, Knuth is a theologian, a little bit; see https://archive.org/details/316bibletextsill0000knut.)
"sacred texts": see above.
"heresies (logical fallacies)": heresies are doctrines that arise within a particular religion and are officially rejected by it; to anyone outside the religion they're an irrelevance. Logical fallacies are definitely not that. If someone mixes up "A implies B" and "B implies A" or something, you don't need to be an internet-rationalist to think they're wrong or to criticize their argument.
You might be able to make an argument that Roko's basilisk is an example of a heresy within rationalism? But I don't think the existence of inside-baseball debates where some positions are officially deprecated is terribly specifically religious. The idea that goto statements are a great way of structuring a piece of computer software would be a heresy, for instance. Belief in "cold fusion" would be another. But computing and physics aren't religion-like in any useful sense on that account.
I should maybe clarify that when I say "I see this said all the time" I don't mean specifically the version targeted at internet-rationalists. There's the version aimed at singularitarians (not all internet-rationalists are singularitarians nor vice versa) and a slightly different version claiming that atheism is just another religion, and so forth. The general template is: Thing X is just another religion / just warmed-over Christianity, because here are half a dozen things about it that have some loose resemblance to some religious thing. I find such things uniformly unconvincing; it seems to me that one can draw loose parallels between almost anything and almost anything else.
And: where are the rationalists', or the singularitarians', or the atheists', sacred places? what are their regular religious rituals? what organization corresponds for them to churches and the like? what things are they obliged to believe, as opposed to believing by definition as a member of whatever group? what ancient traditions do they preserve or claim to preserve? Of course you can have a religion without any single one of those things, but the more common features of religion a thing doesn't have the less plausible it is to call it a religion.
I've never seen an argument like this that if applied consistently wouldn't equally assert that many academic disciplines, large companies, political movements, etc., are "religions".
I'm not going to get drawn into a thousands paragraph debate about rationalists belief, so this will be my only reply and I'll welcome your rebottle.
Your evidence isn't very convincing. I think the singularity is actually framed as a battle between good and evil. How else could Sam Altman be accused of betraying humanity if what his company does isn't framed as that? Moreover, that quote pretty much proves what I'm saying about tech CEOs, how the rationalists view them. There's no "betrayal" here, Sam is and always has been a craven capitalist. One would only feel betrayed by him if they believe in him as someone who violated an implicitly shared dogma.
And that's the big reason for the fallout you're identifying. Early on people like Sam Altman and Elon Musk pretended to be saving humanity through their investments and corporations. It was a marketing ploy and rationalists bought into it hook line and sinker. Of course it was always a ruse, of course these guys were never going to deliver, because at the end of the day their goals were to become rich. Which they achieved. So now the world-saving facade is dropped, and the rationalists who bought into it are none too pleased. Even this line:
"Elon is widely known to be a strong engineer, as well as a strong designer, and spends much of his time arguing details of that kind of work with his reports. Mark Zuckerberg is known to do similarly."
Is very flattering of Musk's and Zuckerberg's supposed engineering prowess. One only need look at the Cyber Truck to understand what kind of designer and engineer Musk is.
> Clearly he's a theologian and computer science is just another religion!
I mean you could take it in that direction. Various cults like the cult of Apple that flourished around Steve Jobs, who himself was a bit of a cult leader. There's a cult around Elon Musk. The thing that's missing is the deity of AI, which is unique to the rationalist community. You might even be able to say that academia is a religion that's formed around to worship science, and there's a lot of parallels between religion there too, down to the dresses and funny hats.
> every field will have its "theologians"
They're theologians when they talk of theology rather than theory. Knuth is a practician. People talking about the singularity are like the guy on the street saying the end is nigh, and people writing blogs about it are essentially priests saying the same thing from a pulpit, the equivalent of theologians are the ones writing scientific papers about it.
> If someone mixes up "A implies B" and "B implies A" or something, you don't need to be an internet-rationalist to think they're wrong or to criticize their argument.
That's not exactly what I'm talking about, logical fallacies are fine and you should point them out when it's relevent. What I mean is for example if you're talking to a rationalist, and you make an appeal to emotion, that's regarded as an invalid argument. Or an even better example is when you make an appeal to authority when it matters, and you get into an argument about whether it's valid or invalid ad hominem. In normal discourse, people make emotional and authoritative arguments and those are valid to consider. In rationalist communities, that's considered a logical fallacy, and the discussion is over. So more often than not when you're talking to a rationalist, the conversation devolves into a debate about which fallacies are being applied and whether they're valid or invalid, because the only thing they'll consider is logos. The doctrine here is that pathos and ethos are invalid forms of persuasion. This is why they're so comfortable rejecting expertise, and why it was so easy to be fooled by people like Musk who used good arguments but lacked actual expertise.
If you think it's likely that there will be a singularity and it will be bad, then you'll regard someone working toward it as bad. That doesn't mean that there's any good-versus-evil-ness about the singularity itself. Whereas Armageddon is specifically a battle of good against evil, the Last Judgement is specifically a separation of people into (something like) good versus evil, etc.
The "betrayal" line is saying that OpenAI was founded as, supposedly, an ethical company trying to make sure that AI worked out for the good of humanity, and Altman is failing to live up to that. It doesn't have anything to do with any high view of Altman personally.
I agree that that line is more positive about both Musk and Zuckerberg than either deserves. But the claim wasn't "some rationalists think too highly of tech CEOs", it was "rationalists think about tech CEOs like religious people think about their high priests", and the point is that nothing I've found suggests that that's so. Sometimes people say nice things about (say) Mark Zuckerberg, sometimes not so nice (there was one comment where someone suggested that a different approach to AI safety might have resulted in "Mark Zuckerberg stamping on a face for ever") but no more of it than in the world at large is looking reverently to them as authorities.
If you're happy also saying that computer science and academia in general are religions, then fair enough, but at this point I think saying "X is a religion" is no longer really saying anything.
I don't know whether you're right or wrong in saying that rationalists are too unwilling to accept arguments that are logically invalid but persuasive in other ways, but this seems a long way from saying that logical fallacies are to rationalists as heresies are to (say) Christians.
Whether Perplexity is useful to someone, and how useful, will depend on how effective it is in giving answers that are (1) correct as often as possible and (2) checkable if they're sometimes wrong.
If Perplexity's answers often treat AI-generated blogspam as reliable then that will probably make them less often correct (because people generating AI blogspam generally don't need what they generate to be consistently correct) and harder to check (because people generating AI blogspam generally don't need what they have to be clearly sourced, and because even if they did it's an extra step of link-following).
So if Perplexity turns up a lot of AI-generated blogspam, that probably makes it less useful. Is it "up to Perplexity"? Again, I don't know what that means, but I don't see that it matters. What matters is how much value it provides, at what cost in time and money and effort and attention. If its output is mostly repeating claims from AI-generated blogspam and that makes the answer "not much, and at too much", then so much the worse for Perplexity even if there's some sense in which it's under no obligation to do better. If its output is reliable and checkable, then so much the better for Perplexity even if some of its deficiencies are in some sense moral failures.
reply