Effective altruism. A lot of the folks working on AI at large tech companies are disproportionately represented in the movement. There's a lot of overlap between EA and the rationalist community as well. The wikipedia page is a good place to start https://en.wikipedia.org/wiki/Effective_altruism
I think it's also worth noting that EA is closely linked to utilitarianism. Most of the pitfalls that people see in EA are the same pitfalls that are classic to utilitarianism, a la "we're going to do this thing we know is locally-bad, because we have a lot of confidence in other effects that are universally-good".
It's important to separate objections to utilitarianism from the obvious fact that it can very be hard to correctly apply the utilitarian calculus. It's partly because of this difficulty that most classical utilitarians thought that people should generally follow commonsense morality and not try to directly apply the utilitarian calculus (which then led to the charge of paternalism and teaching one morality to the masses and another to a supposed elite).
But there are also people who just oppose utilitarianism, like G.E.M. Anscombe. For instance, in https://integrityproject.org/wp-content/uploads/2015/07/mr_t..., she seems to grant that dropping the nuclear bombs on Japan was probably good from a utilitarian perspective (because it saved lives overall) and also to grant that bombing campaigns that necessarily entail massive civilian deaths (including, apparently, area bombing German cities) are morally permissible but still to argue that dropping the nuclear bombs was impermissible because it constituted murder ("intentionally" killing the innocent). But this kind of distinction, which I think is what actual anti-utilitarianism must come to, is hard to even consistently maintain, and I suppose many HN readers would find the effort quixotic.
The first half of your answer presupposes some platonic utilitarian calculus that, if it were applied correctly, would yield moral outcomes. This is very hard to believe. If I look at notable/well-known examples of EA-affiliated people, it is hard to skip by members such as SBF. Did he correctly apply the utilitarian calculus?
It is relatively easy to take the proceeds of a massive fraud, buy a relatively small (as a percentage of the fraud) $ amount of mosquito nets, and save more lives than the lives impacted by your massive theft. Is this a correct application of the utilitarian calculus? What sort of data would we need a priori to do this calculation "correctly"? Do you think he had a careful estimate of the suicide rate of victims of ponzi schemes before perpetuating the fraud, or would any suicide rate have made the decision net [pun intended] moral, as any such victim of fraud would lead to >> 1 net purchased (so you would almost always net save lives).
The above is of course snarky. It is also a best-effort way of analyzing a notable utilitarian's actions. I do not think it would be difficult at all to use this type of argument to argue that SBF's actions net raised utility in the world. If only we all would become fraudsters, then we could truly live in Omelas --- a notable utilitarian paradise.
Yeah, I didn't mean to downplay how hard it is to apply the utilitarian calculus or even to suppose that the bare doctrine of utilitarianism resolves questions about what the ultimate good we should be trying to maximize is. I basically agree that utilitarianism is not a complete recipe for how to live. I just think that it probably gives the correct answer in cases where we can see clearly how to apply it because I'm skeptical of theories like Anscombe's. Which is to say that utilitarianism is a big tent.
Now if we look at EA, the basic tenet of EA seems obvious -- basically just utilitarianism. And from what I've seen, in practice also, EA is a pretty big tent. I don't know the specifics of SBF's case, but I think essentially no one thinks that he acted correctly. I don't know how many mosquito nets he bought, but I agree that if he bought enough, it might be that he net raised utility, and if that is so, it's something to be thankful for. But it doesn't make him some kind of utilitarian saint unless he couldn't have done even more good by some other course of action that wouldn't have hurt the ponzi scheme victims and brought opprobrium on the whole EA movement
This kind of reasoning leads you to reasoning that if he was an ineffective fraudster, it would be less moral, as he would have bought less mosquito nets. So it’s not only moral to do fraud, but you most extremely competently do fraud.
I think this being a reasonable utilitarian point to make is not a point in utilitarianism’s favor.
Even people who say they are deontologists often slip back into utilitarian arguments when they're not careful — for example, when arguing Kant's categorical imperative against lying, they slip into talking about the local benefits vs. overall harms.
The real gap, as you've said, is more about overconfidence in one's utilitarian calculus for distal vs. proximal moral outcomes. An average Joe is likely to give a lot more weight to the moral outcomes that rely on local information and affect his friends and family. The characterization of an "EA" — whether fair or not — is that they're much more likely to use a lot more explicit moral calculus and attempt to correct for proximal vs. distal biases.
In a way its very similar to Sowell's arguments about the informational economics of a distributed market vs. a central planner.
What FTX decisively disproved was the idea that people's origin stories involving apparently sincere desire to do good in the world and them constantly broadcasting that should be used as a reason to unquestioningly trust them when their notion of greater good happens to align perfectly with them accumulating enormous quantities of wealth and power.
(and Sam, bless him, originally wanted to help animals rather than own the machine god. And probably sincerely believed he was going to do great things for humanity from all the misappropriated funds he was definitely going to win back against a backdrop of EAs and VCs queueing up to glaze him and his commitment to the greater good)
I don't think people are objecting to the EA idea that some charities are more evidence based than others so much as the distinctly EA idea that it would be more effective still to donate to charities like OpenAI
todays EA is not about giving to charities, that was the original mission with 40k hours and ethereum (i think vitalik still believes in this version). then the yudkowsky xrisk/ai safety crowd took over lesswrong and turned it into a cult.
now its utilitarianism taken to the extreme. if you believe a skynet scenario killing everyone on earth is plausible then the "logical" thing to do is allow literally anything in the name of stopping it. that includes mass murder and dictatorship. the only thing that can balance the infinite negative value from an evil machine god is the infinite positive value from a good machine god.
thats the main difference today, one faction around sam and dario believes in creating the good ASI first and sacrificing all the world resources to do it before someone makes the bad one, the more pessimistic like yud want to stop all ai development to reduce the risk that an evil god is made to zero.
I may be naive, but I have the feeling that "I will arbitrarily set numbers on things and call it impartial" is... weird at best.
I understand how one may wonder if there was a way to do that, but it feels insane to me that one would actually conclude that "yes, it is possible". We have examples everywhere showing that it is generally impossible to define a metric that correctly represents the underlying concept we want to measure.
Said differently, I feel like Effective altruism fundamentally starts by saying "I don't believe in Goodhart's law". Which seems intellectually dishonest to me.
To quote notorious effective altruist Scott Alexander:
> Look. I’m the last person who’s going to deny that the road we’re on is littered with the skulls of the people who tried to do this before us. But we’ve noticed the skulls. We’ve looked at the creepy skull pyramids and thought “huh, better try to do the opposite of what those guys did”.
"Look. I see that it doesn't work. I want it to work, so I will continue trying, even if it fundamentally cannot work. I am not interested in thinking about whether or not it can work. I am interested in showing to the world that I am well-intentioned and trying to do something, even if that something doesn't make sense".
I mean... Yes, any short snappy explanation is going to be easy to strawman by someone motivated to do so.
The longer non-snappy explanation is that "I will arbitrarily set numbers on things and call it impartial" obviously doesn't match EA's self-conception, that lots of EA cause areas are speculative and don't focus on numbers, that EAs that do focus on numbers do a lot of work to make sure the numbers aren't arbitrary, that EAs as a general rule don't claim to be impartial, and that awareness of Goodhart's law doesn't mean "never trying to objectively measure anything at all".
> I am interested in showing to the world that I am well-intentioned and trying to do something, even if that something doesn't make sense".
This is the kind of pre-conception that's essentially immune to reality. I hear the same thing about vegans (oh they say they care about animal suffering, but everybody knows about factory farms, they just want to feel superior to everybody else) or environmentalists (they say that climate change is a threat to humanity but really they just want to lecture us about our cars).
All I can say is that it doesn't match my experience, and that the effective altruists I've met spend quite a lot of time "thinking about whether or not it can work" and trying to learn from other people's mistakes.
Those are fair points indeed. Let me try to elaborate on my opinion of EA:
> that awareness of Goodhart's law doesn't mean "never trying to objectively measure anything at all".
Goodhart's law doesn't say "never try to measure anything at all". It says "if you try to optimise for the metric, then your metric is doomed". What EA does is pretty much say "let's devise a metric and optimise for it". It does NOT say "let's measure something without influencing it at all". That is totally different.
Wikipedia says (happy to read your corrections if you think it is incorrect):
> Effective altruism (EA) is a [...] movement that advocates impartially calculating benefits and prioritizing causes to provide the greatest good. It is motivated by "using evidence and reason to figure out how to benefit others as much as possible, and taking action on that basis".
While I appreciate the idea of "trying to provide the greatest good" (difficult to go against that :-), my criticism is about the method.
* It is not very hard to convince oneself that if we stopped eating animals, then we would stop abusing chickens (did you know that tens of millions of chickens die during transport in trucks every year in England?) and emptying the oceans, and it would be objectively better in terms of animal suffering and for the biodiversity.
* It is not very hard to convince oneself that our CO2 emissions are literally going to get most of us killed, and that it would be globally better for us "humans who are currently alive" to do something about it. But there already, it's not entirely clear to me if the better outcome for life on Earth is to save the human species. Kind reminder that the human species is currently, measurably destroying all other species at a speed orders of magnitude faster than the extinction of the dinosaurs.
Effective altruism wants to do "the greatest good", but what is "good"? It may be good for a subset of humans to bomb another country and steal their oil, but obviously that would not be good for the subset of humans in the bombed country. It may be good for humans to find a clean magical energy, but that wouldn't change the current mass extinction for the other species (kind reminder that the current mass extinction has nothing to do with climate change, it is all about... well humans having easy access to energy and doing what humans do when they have cheap energy).
I feel like effective altruism says: "We can't define what the greatest good is, but we want to believe that anything is better than nothing. So we define a metric that we call 'impartial' (but that obviously isn't) and optimise for it, knowing that optimising for a metric defeats the purpose of that metric". Really it's rich people who want to do something good but don't want to bother getting informed and convincing themselves about what they want to do. "I'll give a ton of money and in return I get philanthropy points to share with my rich friends, but I don't want to have to think about what is being done with that money".
When someone invests a ton of money and energy into something they genuinely care about, they don't call themselves effective altruists, do they? They are just working for that cause. Effective altruism seems to be about rich people delegating the work of "doing something good" by donating some extra money, while they keep doing what made them rich in the first place (which almost always is something that is going against whatever I would consider the greatest good).
> Really it's rich people who want to do something good but don't want to bother getting informed and convincing themselves about what they want to do. "I'll give a ton of money and in return I get philanthropy points to share with my rich friends, but I don't want to have to think about what is being done with that money".
The people I have met at effective altruist conferences are not rich, though they lean upper-middle class. I've seen way more "enthusiastic broke student" types than millionaires.
> When someone invests a ton of money and energy into something they genuinely care about, they don't call themselves effective altruists, do they?
Well N=1, but I do.
(And also I've met tons of EA people who were not shy about investing all their energy in a cause they care about, even when all mainstream society tells them it's pointless.)
