I replied to LeCun's claims about their latest protein structure predictor and he immediately got defensive. The problem is that i'm an expert in that realm and he is not. My statements were factual (pointing out real limitations in their system along with the lack of improvement over AlphaFold) and he responded by regurgitating the same misleading claims everybody in ML who doesn't understand biology makes. I've seen this pattern repeatedly.
It's too bad because you really do want leaders who listen to criticism carefully and don't immediately get defensive.
Same thing with making wildly optimistic claims about "obsoleting human radiologists in five years", made more than five years ago by another AI bigwig Geoffrey Hinton. They are doubtless brilliant researchers in their field, but they seem to view AI as a sort of cheat code to skip the stage where you actually have to understand the first thing about the problem domain to which it is applied, before getting to the "predictions about where the field is going".
Very similar to crypto evangelists boldly proclaiming the world of finance as obsolete. Rumours of you understanding how the financial system works were greatly exaggerated, my dudes.
The tribal thesis in the AI world seems to be that AI workers don't need subject matter expertise, as the AI will figure it out during training. In fact, subject matter expertise can be a negative because it's a distraction from making the AI good enough to figure it out on its own.
This assumption has proven to be very fragile, but I don't think the AI bigwigs have accepted that yet. Still flush from the success of things like AlphaZero, where this thesis was more true.
Closing your eyes doesn't make the room empty. And in the same way not programming preconceptions into the neural net doesn't make the preconceptions go away?
I realize explaining a joke or something like this takes away some of the charm (sorry), but would love to get the point :)
Randomly wiring a neural net doesn't remove preconceptions, it just makes it that you don't know them. Similarly, closing your eyes doesn't make a room empty, it just makes it that you can't see what's there. Minsky is pointing out that Sussman's fundamental assumptions on desiring to remove preconceptions is logically flawed.
Gotcha - I read it as Sussman didn't want to program his preconceptions into the network (for which randomness seemed suitable, which is why I was confused). Your explanation makes more sense.
There's an old series of - stories? jokes? called 'unix koans' [1] which always end with a master answering a question in a very unclear but profound-sounding way, then the line 'Upon hearing this, [someone] was enlightened.'
A Koan (公案) is a concept from Zen Buddhism. Zen made its way into pop-culture (or at least the popular counter-culture movement) in the US in the 1960s via several writers of the Beat Generation. Project MAC at MIT was founded in the early 60s (Gerald Sussman started there in '64 I think)? So a number of faux Koans were in circulation in the AI crowd by about 1970.
Unfortunately because it's delivered as a koan we'll never know whether he's talking about the fact that the random weights determine the nearest local minimum or the hyperparameters.
also unfortunately, a "real koan" often relies on conflating expectations+conditioning versus direct experience, where direct experience is shown to invoke an "impossible" result; teaching that the conditioned, subjective mind does not see fundamental aspects of reality, though it thinks it has the answers.
These technical mimics of that structure echo a time when a lot of people, relatively, were experimenting with disregarding personal subjectivity in favor of direct experience and deeper Truth. In the modern tech versions, that is rarely if ever part of the story?
The problem with preconceptions about your parameters is that you might be missing some crazy cool path to your goal, which you might find by randomly exploring your sample space. I remember seeing this same principle in mcmc samplers using uniform priors. Why is this so crazy?
It's predicated on the assumption that a random discovery from a zero-comprehension state is more likely to get you to a goal than an evolution from a state that has at least some correctness.
More generally, it disingenuously disregards the fact that the definition of the problem brings with it an enormous set of preconceptions. Reductio ad absurdum, you should just start training a model on completely random data in search of some unexpected but useful outcome.
Obviously we don't do this; by setting a goal and a context we have already applied constraints, and so this really just devolves into a quantitative argument about the set of initial conditions.
(This is the entire point of the Minsky / Sussman koan.)
> from a zero-comprehension state is more likely to get you to a goal than an evolution from a state that has at least some correctness.
I get that starting from a point with "some correctness" makes sense if you want to use such information (e.g. a standard starting point). However, such information is a preconceived solution to the problem, which might not be that useful after all. The fact is that you indeed might not at all need such information to find an optimal solution to a given problem.
> by setting a goal and a context we have already applied constraints.
I might be missing your point here since the goal and constraints must come from the real world problem to solve which is independent from the method to solve the problem. Unless you're describing p-value hacking your wait out, which is a broader problem.
With exploring, the starting state should only affect which local-maximum you end up in. Therefore you need to make an argument that a random starting state is likely to end up in a higher local-maximum than a non-random starting state.
There is always a starting state; using a random one only means you don't know what it is.
There are a lot of problems that arise from lack of domain expertise, but they can be overcome with a multidisciplinary team.
The biggest defeating problem for pure AI teams is that they don't understand the domain well enough to know if their data sets are representative. Humans are great at salience assessments, and can ignore tons of the examples and features they witness when using their experience. This affects dataset curation. When a naive ML system trains on this data, it won't appreciate the often implicit curation decisions that were made, and will thus be miscalibrated for the real world.
A domain expert can offer a lot of benefits. They could know how to feature engineer in a way that is resilient to these saliency issues. They can immediately recognize when a system is making stupid decisions on out of sample data. And if the ML model allows for introspection, then the domain expert can assess whether the model's representations look sensible.
I'm scenarios where datasets actually do accurately resemble the "real world", it is possible for ML to transcend human experts. Linguistics is a pretty good example of this.
It makes sense to have a domain expert and an AI expert working together, but I'd offer two important modifications:
1) The AI expert is auxiliary here, and the domain expert is in the driver's seat. How can it be otherwise? You no more put the AI expert in charge than you'd put an electronic health record IT specialist in charge of the hospital's processes. The relationship needs to be outcome-focused, not technology-focused.
2) The end result is most likely to be a productivity tool which augments the abilities/accuracy/speed of human experts rather than replacing them. AGI being not that sciencey of a fiction, we aren't likely to be actually diagnosed by an AI radiologist in our lifetimes, nor will an AI scientist make an important scientific discovery. Ditch the hype and get to work on those productivity tools, because that's all you can do for the foreseeable future. That might seem like a disappointing reduction in ambition, but at least it's reality-based.
Unless of course the "domain experts" have fundamental disagreements or have equally limited knowledge of what should constitute what is important to extrapolate data beyond their own scope. E.g. like in comp sci, there might be multiple comparable ways to accomplish n, but which is best to reliably accomplish an unknown or unforseen n+1...depends.
> The tribal thesis in the AI world seems to be that AI workers don't need subject matter expertise
Not throwing any stones here, because I've been guilty of the same sort of arrogance in other contexts. But I think the same thing happened a ton during Bubble 1.0 and the software-is-eating-the-world thing. And it's hardly limited to tech: https://xkcd.com/793/
For me, at least, where this came from was ignorance and naivete. Three things cured me. One was getting deeper mastery of particular things, and experiencing a fair bit of frustration when dealing with people who didn't understand those things or respect my expertise. The second was truly recognizing there were plenty of equally smart people who'd spent just as long on other things. And the third was working in close, cross-functional team contexts with those people, where mutual listening and respect were vital to the team doing our best work.
So here's hoping that the AI bigwigs learn that one way or another.
Not only AlphaZero, but hadn't the whole field of computer vision (which LeCun is specialised in) had its major breakthrough with letting the AI figure out the features (i.e. CNNs)?
> Any radiology AI that needs millions of training sets is useless in practice.
Why? I have no doubt that radiology AI might not be that useful (though radiologist friends of mine say AI is making an increasing impact on their field.) But this logic doesn't make sense. So what if an AI needs a million training examples or even a million training sets? Once your topology and weights are set, that net can be copied/used by others and you get a ready-to-go AI. There's an argument to be made that if training scale is what's needed to get to AGI, then maybe AGI is unrealizable, but that's not the same as saying a domain-specific AI is useless because it needs a large training set.
I think all this work could be useful if only those people understood that the technology is not mature enough to remove people from the loop.
For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human.
This could save a ton of time and help reduce cases where radiologists missed obvious things.
My son once broke his arm. I brought him to ER, they made the photo but the two people who looked at the photo said there is nothing wrong with the arm. I asked for the copy of the photo.
A week later the swelling did not subside so I took the photo to another doctor and he pointed out an obvious fracture line.
There are many ways to deploy automation and I wonder why everybody tries to shoot for removing humans altogether when most of the time this is literally asking for problems.
I've definitely seen radiologists miss things (as a patient and a researcher) but I've also seen the behavior where if something is labeled or indicated the next person (or in this case the first person) might not give it a true 2nd look and could easily default to agreeing with the AI instead of actually checking things.
Overall I think this would be a net benefit as image analysis can help with a lot of these activities, just needs care to not inadvertently remove the humans you still need.
Right. Until we trust the AI 100% (at which point we get rid of humans) the AI input shouldn't be given until after the human makes a call. However then the AI gets above 80% (or something like this, UX has better numbers) trusted humans know the AI will look will just skip looking themselves.
The above is a common problem that UX research is interested in. I'm not sure how much it is solved, but it goes well beyond medical fields.
sadly, this will never appear completely in practice, I predict. Refer to information theory roughly, to support that. That is, pure signal is not often occurring in real systems. Some systems will favor signal quality at the expense of time and fewer false positives and negatives, but the majority of cases in the real world will favor less-effort, less-time decisions that are expedient, or worse, favor obfuscation of the process to cover the sins of the humans benefiting/profiting directly from the show. Also, systems that just are not actually working well, will be sold for quick profit using pressure marketing and forced contracts, at least I think so...
>For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human.
The human will still have to look at the x-ray to see if the AI missed something. 95% accuracy is not good enough, those 5% of cases are what most of their training is for, missing it can mean a lost human life. Maybe it can be used to speed up obvious diagnoses, but it cannot be used to filter and rule anything out. The amount of time a radiologist will spend looking at the x-ray will probably not be reduced, so I don't think there's money to be saved here.
A useful productivity tool could be to examine datasets after the radiologist found nothing, as a way to double-check their reading. This won't reduce costs but might marginally improve patient outcomes. Radiologists in first-world medical systems don't really miss a lot of stuff, though.
And of course for simple obvious non-life-affecting stuff like broken bones and dental x-rays, you don't need radiologists now either. Your son's x-ray was probably not looked at by a radiologist.
> In the 1970s, it was found that 71% of lung cancers detected on screening radiographs were visible in retrospect on previous films [4,6].
> The “average” observer has been found to miss 30% of visible lesions on barium enemas [4].
> A 1999 study found that 19% of lung cancers presenting as a nodular lesion on chest x-rays were missed [7].
> Another study identified major disagreement between 2 observers in interpreting x-rays of patients in an emergency department in 5-9% of cases, with an estimated incidence of errors per observer of 3-6% [8].
> A 1997 study using experienced radiologists reporting a collection of normal and abnormal x-rays found an overall 23% error rate when no clinical information was supplied, falling to 20% when clinical details were available [9].
> A recent report suggests a significant major discrepancy rate (13%) between specialist neuroradiology second opinion and primary general radiology opinion [10].
> A recent review found a “real-time” error rate among radiologists in their day-to-day practices averages 3-5%
> In patients subsequently diagnosed with lung or breast cancer with previous “normal” relevant radiologic studies, retrospective review of the chest radiographs (in the case of lung cancer) or mammogram (in breast cancer cases) identified the lung cancer in as many as 90% and the breast cancer in as many as 75% of cases [11].
> A Mayo Clinic study of autopsies published in 2000, which compared clinical diagnoses with post-mortem diagnoses, found that in 26% of cases, a major diagnosis was missed clinically [11].
Certainly. In fact, prostate cancer is often best left undiagnosed.
As the CDC says on its page for prostate cancer screening[0]:
>Screening finds prostate cancer in some men who would never have had symptoms from their cancer in their lifetime. Treatment of men who would not have had symptoms or died from prostate cancer can cause them to have complications from treatment, but not benefit from treatment. This is called overdiagnosis.
>Prostate cancer is diagnosed with a prostate biopsy. A biopsy is when a small piece of tissue is removed from the prostate and looked at under a microscope to see if there are cancer cells. Older men are more likely to have a complication after a prostate biopsy.
> For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human.
There was a paper a while ago about an effort at something like this. They got the AI going and noticed that the first thing it did was classify all the x-rays by race of the patient. Then they freaked out, gave up on their original project, and wrote their paper about how AI is inherently evil.
Remove the human and you can charge close to what it costs to employ the human. Aid the human and you’re looking at more like tens to hundreds of dollars per month.
The funny thing is, of all the areas where ML could help, radiological image classification definitely is the one where ML could shine (and I think it did). HUmans doing radiological image classification are basically a network service now (IE they can do their job on the other side of the world, and their efforts are extremely carefully evaluated using later data such as disease progression).
Image classification could be employed (together with very very good UI) in good productivity tools for radiologists. However, radiologists don't classify images for a living, they make diagnoses, so it's not going to get close to replacing them. I agree that radiology is the most "friendly" discipline to inject ML-based productivity tools into their "pipeline", since they essentially don't even need to be at the hospital, other than interventional radiology.
A radiologist's diagnosis is not image classification, it's reality classification maybe. (That's quite poetic).
Watching an educational Youtube video about endangered tiger habitats is not the same thing as segmenting possible embedded pictures of kitties and poachers or whatever, and classifying them as such. There's, like, a lot of additional context.
A (good) radiologist is interpreting the images in light of the patient's history, symptoms, and other tests, with the goal of forming a diagnosis and treatment plan. This is rather different from taking an MxN array of pixels and trying to decide if it contains a tumor.
For example, about 10% of people have small gallstones. If an ultrasound incidentally detects some in a healthy, asymptomatic person, nothing happens. The exact same images, coming from a patient with a history of upper-right abdominal pain and jaundice, probably lead to a referral for surgery instead.
What you described is outside the scope of the problem being solved in radiological classification. That problem- like protein structure predction- is an intentionally simplified process used to make it possible to fairly compare humans vs ML.
What you're describing is the general problem of informed diagnosis, which is also classification, but typically takes into account a great deal of qualitative information. Few if any ML people are working in this area because there is no golden data and it's nearly impossible to evaluate in a quantitative way.
Agreed--but that's the point: the demand is not actually for radiological classification, it's for accurate, actionable diagnoses. Heck, radiologists call what they do "interpretation."
I can certainly imagine that radiologists would love (say) a tool that automatically flags low-quality or mis-oriented images, but that's not at all where the hype is at.
Those things already exist. It was very enlightening- my uncle was a radiologist and I spent a few hours watching him do his job. The software they use is extremely sophisticated with lots of custom bells and whistles (and the monitors have insane contrast ratios). Most people don't see it but non-ML medical imaging is extremely mature and is developed in close contact with the users.
There is a personal bias I have observed when I have listened to people talk over the last ~5 years, for example:
- Startup founders without any domain experience in extremely risky ventures
- Crypto bros
- Donald Trump
I consider myself - and others view me as - a hyperrational person (possibly often to a fault), and I must admit that when I hear an outlandish claim like the ones spewed by the above, I am sometimes left in a strange emotional state... a stupor?
Like, I don't quite believe the claim because I'm defensively rational, but I feel a certain dizziness and confusion (thinking to myself, "could this actually be true?") until I come back to my senses. The more outlandish and impassioned the speech, the stronger the effect.
It's made me realise we're all built similarly, from the most rational from the most gullible.
> Like, I don't quite believe the claim because I'm defensively rational, but I feel a certain dizziness and confusion (thinking to myself, "could this actually be true?") until I come back to my senses. The more outlandish and impassioned the speech, the stronger the effect.
I always like to ask myself: "what are they trying to get me to do?". Whether it's a habit, voting pattern, product, etc. Then I ask myself: "is it useful? who does it benefit?"
Whether it's true or not, idrc. what's more important (at least to me) is what it does and who it benefits
(also in this case to avoid any ethical conundrums, something is useful if it makes $, beneficial to X party if it makes them $)
This. In the language popularized by Kahnemann et al. I tell myself: wow, he really got my system 1. And then system 2 needs to work on on undoing the belief created, get back to sober doubt and weigh the probabilities.
But this is an interpretation after the fact, it feels exactly like you describe it. Stupor.
I somewhat doubt most people actually have the `"could this actually be true?"` phase. It seems most like... just your rational mind reminding you that whatever you set your priors, if you set it to exactly zero the Bayes' Theorem breaks down?
I do- I definitely see limits to my own rationality that have only been obvious after extraordinary reflection and addtional data. I would presume that most people who are truly self-aware recognize that we are all fundamentally irrational.
I think HN is skewing heavily against AI and blockchain claims. I pick those to make the point below.
For what it's worth, I agree that blockchain itself is a first-generation technology and sucks relative to other things, like giant vacuum tube computers did. However, the concepts it enables (smart contracts) have as much promise as the idea of software programs running on personal computers back when most people wondered why you need them, since they do very little but play pong.
When I wrote the following article for CoinDesk in 2020, I didn't want to say "blockchain voting", I wanted to say "voting from your phone". Because there are far better decentralized byzantine-fault-tolerant systems, than blockchains. But that's what they ran with:
For every technology we use today, there was a time it was laughably inadequate as a replacement for what came before.
And that's really, the crux of the issue. It happens slowly, and then all at once. Yes we need to listen to guys like Moxie who are skeptical, but we need to also then go and have a discussion from different perspectives, not just one specific perspective. It has even become fashionable in many liberal circles to be against the type "tech bro" typified by HN, including VCs and Web 2.0 tech bros. So before you downvote, realize that most of you would be on the receiving end of it in other echo chambers, due to this phenomenon of thinking there's only one best narrative.
People like Moxie are much more interesting and interested, because they say they' love to be proven wrong. And I am also open to substantive discussion:
I imagine it's the same with AI claims about traditional fields. Where have we heard that before? "Yes it's cute and impressive but these guys don't really understand what the experts know about chess."
> For every technology we use today, there was a time it was laughably inadequate as a replacement for what came before.
That is just success bias. How many times did people try out perpetual motion machines? The rest of the article tries to make the converse that if something is inadequate today, it will be the replacement in the future.
When do you consider something to be not worth spending time on?
Mark Twain lost all of his money on a wide variety of speculative investments, most of which I would call fairly reasonable.
Getting in on the ground floor does you no good if it's the wrong kind of ground floor, or if it's the right kind of ground floor but the one next door ends up going to the moon while yours falters, or even if it's the right exact ground floor but you go bankrupt investing too early and the person who buys it from you rides it to the moon.
I think its more a comment of how we should be open to people contributing to these efforts (while still being mindful of wild claims). While the concept of perpetual motion machines is flawed, the effort put into reducing kinetic energy loss is very useful. Flywheels are being actively studied as a method to store excess energy storage. In the long-term view I don't think you can consider anything not worth spending time on, the result just might not be applicable to the original goal.
> How many times did people try out perpetual motion machines?
Or even turning lead into gold! A ton of famous scientist (i.e. Newton) were very busy with that idea but we end up only knowing them for other side projects/discoveries.
My skepticism about e.g. AI and blockchain is not definitive or final, and the goal is not to end the conversation. To the contrary, for me this is a (possibly very deficient) style of reasoning about the world. You are excited about technology X, can I express my skepticism in a sufficiently annoying way to prod you into helping me understand some insight about technology X which disproves my hypothesis that technology X is a technology in search of a problem? For example, I don't see an important problem which is solved by blockchain in a way that's superior to other solutions, when taking into account their respective advantages and drawbacks. Just don't see it.
For example, I want to have a centralized authority which can override fraud or a mistake in a financial transaction. I want laws to apply, I want them to be written by elected humans.
I'm not even sold on the value of any kind of electronic voting in general elections, since trust in the process is so vital here that in my mind the horrible inefficiencies of pen and paper and a bunch of humans manually tallying up votes in a thousand school gyms until 11 pm is actually quite okay. I'll pay for that with my taxes, no problem. Now you add blockchain into the mix, and I don't know what problem it solves that does not have superior alternative solutions.
And so on. But I'm gonna stay open minded. Technology X might one day find the perfect problem to solve, or I might realize I was stupidly wrong about technology X for some time.
For now, I would simply ask that you go through the many applications here and tell me if you see smart contracts being very useful for collectively managaing high-value decisions and capital after reading: https://intercoin.org/applications
Can you instead point out the one (strongest, best) application for which the case of using blockchain solves a big problem in a way that's better than existing alternatives, given all their respective advantages and drawbacks?
I'm lazy, you see. Conveniently forgot to mention that.
1. Communities around the world issue their own currencies independent of the dollar (believe it or not, this is as important for financial stability as not just relying on Facebook’s server farms in California)
2. They give their members a UBI in the local currency that they can only spend locally
3. The amount of daily UBI to give out is determined by the vendors in the community voting on it (monetary policy)
4. Have each vendor tagged with “food”, “clothing” etc. and apply taxes to make negative externalities more costly and withdraw money from the economy (fiscal policy)
5. Calculate statistics on how money flows locally in the smart economy and have economists make recommendations about the fiscal and monetary policies, while the population continuously gets to adjust them up and down based on these recommendations
6. Hooking up all such communities to a decentralized exchange called Intercoin, where the central currency is only held by real KYCed communities and isn’t easily susceptible to pumps and dumps by speculators and banks like Goldman Sachs, and also encourages recurring and sustainable value exchanges between communities
7. Allowing tourists to buy the local currency, and allowing individuals around the world to donate to the community and see stats how the money is being used
What problem does this one solve, and how does it solve it? To me, this just looks like how a normal currency works, "but on a computer". Is the (claimed) advantage that you can narrow the scope of a currency? I just don't see the benefits in doing so.
Well, that sounds exactly like what people said about all the other technologies that were "on a computer". Watch this exchange, for instance, between Bill Gates and David Letterman... about "this internet thing"
Technology empowers individuals and smaller communities. That's what it does throughout history. Personal computers. Personal printers. Now you can send email instead of relying on a centralized post office. VOIP disrupted Ma Bell monopolies in our lifetimes, instead of $3 a minute phone calls you can now do full videoonferencing for free. The Web instead of gatekeepers at radio, TV, magazines, newspapers, etc.
In all of those cases, you could question why "on a computer" matters. Who needs email when there are phonecalls? Who needs the Web anyway when there is email? Who needs online dating sites when there are matchmakers?
Nathan Myrhvold at Microsoft told people at Excite that "search is not a business".
Economist Paul Krugman wrote that by 2005, it would become clear that the Internet's effect on the economy is no greater than the fax machine's.
Well, these smart contracts are programmable, and you program against one widespread platform, like JVM or in this case EVM. That's a huge benefit. You can have programmable money, programmable elections, without having to deal with thousands of APIs, or as you are saying "normal" physical currency. Governments in fact also want to phase out cash, and even bank credit, and create centrally controlled "CBDCs" so your "normal" money is also under attack by your governments. China and USA and Canada have already done it and they'd love to be able to freeze people's accounts, restrict them from getting on trains, etc. It may be preferable to incarcerating them later on for years, as we do in the USA.
I read the list of applications. All of them seem pointless, or at least inferior to what we already have. Obviously the people writing these lists are unclear on the basics and probably haven't even read the NIST blockchain technology overview.
What they all have in common is cutting out an inefficient rentseeking middleman that people have been forced to trust. Yes that includes FTX, Binance, Coinbase and governments.
Here is an example… how would you do this internationally without crypto?
It's a stupid idea. There are plenty of good charities already helping refugees. You can just donate cash to them. Using cryptocurrency just complicates things without adding any value or solving any real problems.
Sorry, but no. You don’t seem to be very knowledgeable about what these charities actually get done and their efficiency.
The real problems are making global payments directly to the people on the ground without waste and having confidence in how they are spent.
This works with the existing infrastructure in every city — similarly to point-of-sale machines that VISA and Mastercard did a lot of work to set up over decades. Back then you would be asking why the world needs credit cards and payment systems when there were perfectly good cash based systems and charities on the ground.
Anyway, the vendors sell food, the person shows up and buys the food. We know how the money was spent. The people help people.
To try to show you by analogy… it is as if people said that there should be a decentralized and uncensorable network for uploading videos taken by people’s own dashcams, phones, etc. of rockets hitting buildings, detention campa etc. But you’d keep saying that the Associated Press and the current centralized media is perfectly adequate and reports everything we need to know, and if people wanted they could upload their videos to Telegram or some other adhoc solution that isnt designed like the news agencies. Why decentralize anything? Because the people DON’T get a good system otherwise
Actually I'm more knowledgeable about this stuff than you are, and it's still a stupid idea. It fails to account for all the tax and legal compliance issues that vendors have to deal with.
And real vendors don't want cryptocurrency magic beans anyway. They want useful currency like dollars or euros that they can use to pay their own suppliers.
I doubt that. But hey, if you're so knowledgeable, then you'd realize that the systems do in fact support taxation and make auditing for compliance with any goals far easier.
Technology empowers individuals and smaller communities. That's what it does throughout history. Personal computers. Personal printers. VOIP instead of $3 a minute phone calls. The Web instead of gatekeepers at radio, TV, magazines, newspapers, etc.
In all of those cases, you would probably say "the real vendors don't want the Internet anyway". Who needs email when there are phonecalls? Who needs the Web anyway when there is email? Who needs online dating sites when there are matchmakers?
Nathan Myrhvold at Microsoft told people at Excite that "search is not a business".
Economist Paul Krugman wrote that by 2005, it would become clear that the Internet's effect on the economy is no greater than the fax machine's.
You'd be in good company ... a lot of smug people have always said this newfangled stuff is totally useless because people are perfectly fine using the "useful" systems they've always used, not "magic beans" like this new programmable money.
I think the framing of your argument is a little bit misguided. Looking at history and possibilities doesn't really get you anywhere. Sometimes people get a tech massively wrong, sometimes they don't.
In order to have an actually good discussion, we need to look at the thing and go past the "well someone criticized the Internet too and now look at it".
So taking the idea of the blockchain. What makes the blockchain a "different thing"? The differential aspect is that it allows untrusted nodes to join in a distributed architecture. It's not the only database that exists and it's not the only distributed one either. So any claims of new features brought by blockchain should justify why they need the "untrusted distributed nodes" part. If they don't, we can assume that those new features don't really need the blockchain: either it's already been done, it's not an use case people are too interested in or it's not viable due to other reasons (economical, technical apart from storage, political...). In the case of blockchain claims, most don't actually justify the need for the untrusted distribution. For example, smart contracts: it's just a fancy word for "computer program" only that it runs on a distributed trustless architecture. But is that really needed? My bank already runs computer programs that execute loan payments, for an example of things people try to implement with smart contracts.
Compare that with personal computers or even AI. PCs allowed data manipulation, storage and calculations at a capacity that was not previously available. Of course the first computers wouldn't have enough power to do things that a wide array of people would find useful, but "low power" isn't a fundamental aspect of personal computers in the same way that "low bandwidth" isn't a fundamental aspect of blockchains.
Suppose I start a Web2 company, and end up building the next big social network. Or maybe I deploy Mastodon or Qbix and make a large community.
Now elections, roles, permissions and credit balances may hold significant total value.
Various jurisdictions now start to require you to hold surety bonds, get audits etc. Suppose you manage payments volume of $20 million a month for a teacher marketplace. One of your developers can just go into the database and change all the balances, salami-slicing money to themselves. Or someone can go and change all the votes.
How can the users trust elections, or that you won’t abscond with the money one day, or get hacked, like FTX and MtGox?
Web3 solves this with smart contracts. For the first time in history, we can guarantee (given enough nodes) that it is infeasible to take actions you are not authorized to do. The blockchain is readable by everyone - but more importantly, only authorized accounts can take individual actions that do a limited amount. It’s truly decentralized.
The alternative is to build elaborate schemes where watchers watch the watchers — and the more value is controlled by the database (in terms of votes or balances) the more risk and liability everyone has. Why have it?
Have teachers be paid by students using web3 smart contracts and tokens. Your site becomes merely an interface which contains far more low-value things.
As for data, you can store it on IPFS with similar considerations. Read this:
As you can see, my company and I have been giving it a LOT of thought, and not distracted by ponzi schemes. I am able to articulate exactly when you need Web3 and IPFS.
Do smart contracts resolve that problem? I don’t think they do completely.
Most people won’t have the knowledge or the time to verify the contracts. They will trust your word that they can’t be used to scam them. Smart contracts can still have failures too. And as long as the blockchain doesn’t control the real world, it won’t guarantee anything there (such as people making multiple wallets to manipulate the votes).
> The alternative is to build elaborate schemes where watchers watch the watchers — and the more value is controlled by the database (in terms of votes or balances) the more risk and liability everyone has. Why have it?
Why not the alternative of a regular bank account with public records? That also eliminates the risk that any mistake or manipulation stays there forever. It’s a tradeoff, not an absolute improvement.
In any case, it seems you have indeed thought about a real use case where the blockchain at least makes some sense. But that’s precisely my point, we need to be talking about actual use cases and not empty claims about potential without actually looking at why it’s useful.
1. End-users don’t need to personally verify smart contracts or any other open source software. The key is to have each version have a hash, and on EVM thatms easy — it’s the address of the contract. Then, more and more auditors go through the code and sign off on the contract. In fact, this should be done for lots of open source ecosystems, eg composer or npm, because then we might not have stuff like this: https://blog.sonatype.com/npm-project-used-by-millions-hijac...
2) On EVM you can just audit a smart contract FACTORY, and then people can trust all the instances made from it. This is immensely powerful.
And then there is also regular use by people who put huge amounts of value into a protocol and it is never hacked. It is why for example people trust UniSwap liquidity pools not to rugpull them, and the same with many other protocols.
Now, to be fair, you don’t need a blockchain for that. You can use merkle trees and code signing. But the current Web sucks at it — you just have to trust the server not to suddenly send you bad code, or your “user agent” to execute JS that sends your password in the clear somewhere. And App Stores are black boxes where you just have to trust Signal’s claims or Telegram’s claims. I write about that here in much more detail:
There are real societal consequences on the largest scales, from these platform choices of tech stack.
3) Sure, in the real world, things may not match the data on the blockchain. That painting could be stolen despite what the NFT says. The house up for sale may be a fake. And someone could default on a loan that you thought would bring you revenues.
Blockchain doesn’t guarantee any of those things. However, on Aave marketplace, I am guaranteed a return, or my money back in the token I had lent. So my risk is reduced now to black swan events on the token market, so if I’m using mainstream tokens the chances of getting wiped out or losing my collateral are almost totally eliminated.
People who don’t want to use it don’t have to. But if you’re building a very big community with a lot of value at stake, would you rather put votes, roles, and balances into a central database and PHP app code, or a blockchain with smart contracts?
From personal experience I can tell you it should be the latter.
4) In your example it would be the bank that would opt to use blockchain and freely offer other indepdendent entities not under their control to run nodes snd secure it. It reduces the bank’s risk and liability!
And yes thank you for recognizing that there may be a real class of use cases!
I feel like there is a tendency for technology to overpromise by offering solutions to the wrong problems. To take your example, what is the fundamental issue with elections? I would say that it is that the optimal number of people for deliberative decision making is probably around Dunbar's number, certainly not millions. When you have millions of people, it is purely a media game, so decision quality falls off a cliff. So I doubt general elections can ever yield better results than they currently do, regardless of the voting system and regardless of technology.
So my general skepticism regarding blockchain is that it presents technological solutions to social problems (so it won't work). AI is different since it's a bit all over the place. In principle, it aims to solve problems that are obviously worth solving, but as long as it will fall short of its promises, the partial solutions we do get are kind of a mixed bag: if we need to keep a human in the loop or at the wheel, it's suddenly a lot less attractive. And the path to the AI being good enough to be trusted is nebulous at best. But we'll see. As for PCs, their utility has always been obvious and the roadmap clear.
The fundamental issue is that elections are too expensive so we don’t do collective decision making that often, and usually only for our state or national governments.
People tend to elect representatives for long terms instead and then complain about them, rather than delegating their vote to experts or trying other systems like Ranked Choice Voting.
Many people complain about having to travel far to a polling booth, and disenfranchisement, whereas they could vote for their phone. Elderly and minorities in rural areas often have bad access.
If elections were cheap, people could easily engage in collective decision making of various types and choose various ways yo tally votes. None of that is possible today, we are like the people before computers, or before the industrial revolution - having a limited number of options, newspapers, etv.
We would also have more confidence in the results as we’d check using the Merkle tree that our vote was counted. It would be user friendly to do so.
And we could also implement many of the results on-chain, such as how much UBI to give out in our own community’s currency, or how much to tax transactions.
The fundamental issue with elections is that the mob is easily swayed and because most of us are unqualified to hold opinions about most things. What is best or true is not decided by majority vote. Never in a million years would I want mob rule. I don't know where this strange doctrine of mob wisdom came from, but it is dangerous and false. The vote is open to all adult citizens only because we need a hedge against corruption in leadership, and even then, it is a vote for leadership, not a system of referenda and direct democracy. Even this "hedge" can easily enable corruption and vote the worst tyrants into power.
And the reason we localize voting, or should do so, is because of the principle of subsidiary.
Back in 2014 there were a lot of studies on wisdom of the crowds. You’d have to explain why they beat experts at many fields yet in others they would be worse. Most of the failures (FTX, MtGox, Softbank, invasions of Iraq and Ukraine, wars in general) are the result of centralization, not the regular people (who don’t want to kill others en masse). Centralizing power and decision making in a few hands leads to a lot of consequences:
That's not hard. Most cases where wisdom of the crowd works fine is when the crowd does not have a personal or emotional stake in the outcome. Such gatherings attract mostly people that have an interest in that particular area, so the crowd self-selects for competence.
What is your expected outcome when you let the crowd decide how much taxes they pay, or how much to spend on road/water/electricity grid maintenance?
My point is that general elections are intrinsically mediocre at decision making. You can make them cheaper, more efficient, make sure everyone can vote, add a delegation system, and so on, but you'd just be polishing a turd.
It's pointless to improve voting technology if people don't understand the ramifications of what they vote for, and they will never know that unless they invest hundreds of hours of careful study into it. If everyone does this, there is no way it won't be insanely expensive.
IMHO there is a very simple solution to all of these issues that could have been implemented a century ago: pick representatives at random. Send them all to the capital, lodge them and pay them to think about the issues full time, talk to experts directly, interview candidates for Prime Minister and other executive positions directly, coordinate with each other, and so on. Give people the time and the information they require in order to cast the very best votes they are capable of.
I'd say the very simple solution is to have representatives decide. Then there's a separate debate about how to the representatives are assigned. At random, based on voting, based on competitive self-appointment by means of heavy weaponry, etc.
Blockchain may or may not make it easier to bypass the concept of having representatives decide, but this assumes we'd want to in the first place, which I am in full agreement with you: this is a feature, not a bug, so I don't wanna, so I don't need blockchain.
TBH I think even a basic "I considered your point, but X and Y factors seemed to mitigate it enough for my standards, defined by P and Q. Let me know if I'm misunderstanding anything" would do a lot. It's important to always show you've considered that you're wrong about something.
I'm not sure what OPs particular point was, but Yan seemed to argue over and over again that testing Galactica with adversial inputs is why "we can't have nice things" which to me seems not just defensive but kind of comical.
Any AI model needs to be designed with adversarial usage in mind. And I don't even think random people trying to abuse the thing for five minutes to get it to output false or vile info counts as a sophisticated attack.
Clearly before they published that demo Facebook had again put zero thought into what bad actors can do with this technology and the appropriate response to people testing that out is certainly not blaming them.
Any AI model needs to be designed with adversarial usage in mind
Why? There's probably plenty of usage of ML where both the initial training set, its users and its outputs are internal to one company and hence well-controlled. Why should such a model be constructed with adversarial usage in mind, if such adversarial usage can be prevented by e.g. making it a fireable offense?
> He's supposed to agree with you, or not express an opinion?
Wow, not sure what to say if that's what you think are the only options. I didn't see the original response to the parent commenter, but this quote in the article, "It’s no longer possible to have some fun by casually misusing it. Happy?" doesn't bode well.
I get that in the post-Twitter world it can be heart to differentiate between valid criticism and toxic bad-faith arguments, but lets not pretend that it's impossible to acknowledge criticism in a way that doesn't immediately try to dismiss it, even if you may not agree in the end.
No, you can disagree with someone without acting defensive. When a person is acting defensive, they're trying to protect or justify themselves. People who are insecure or guilty tend to act defensive. You can have a disagreement and defend your positions without taking things personally.
You're right. That response is sufficient for people who provide it in good faith. There are bad faith actors who aren't happy unless you actually respond in detail and convince them otherwise. They're more than happy to raise a ruckus about how "XYZ ignored my feedback and criticism".
From the HN Guidelines: " Be kind. Don't be snarky. Have curious conversation; don't cross-examine. Please don't fulminate. Please don't sneer, including at the rest of the community. Edit out swipes. "
I feel we are at crypto/blockchain levelof hype int ML and basically the old saying of "if you are a hammer, everything looks like a nail" applies.
For someone who dedicated their career to ML, they'll naturally try to solve everything in that framework. I observe this in every discipline that falls prey to it's own success. If there's a problem, those in the industry will naturally try to solve it with ML, often completely ignoring practical considerations.
Is the engine in your car underperforming? Let's apply ML. Has your kid bruised their knee while skating? Apply ML to his skating patterns.
The one saving grace of ML is that there are genuinely useful applications among the morass.
Without even opening the link I half expected it to be about LeCun and I want wrong.
Him and Grady Booch recently had a back and forth on the same subject on Twitter where to me it seemed like he couldn’t answer Booch’s very basic questions. It’s interesting to see another person with a similar opinion.
> It's too bad because you really do want leaders who listen to criticism carefully and don't immediately get defensive.
For sure. If this is how he treats outside experts, I can't imagine what it's like to work for him. Or rather, I can imagine it, and I think it does a lot to explain the release-and-panicked-rollback pattern.
ML people are the ultimate generalists. They claim to make tools which are domain agnostic, but they can't really validate that for themselves because they have no domain knowledge about anything.
Could you share the critical feedback you gave? I am interested as someone who works with biological systems and is curious about how ML can or cannot help.
I told him that: the increased speed but lower accuracy of their protein structure predictor was not useful because the only thing that matters in PSP is absolute prediction quality. And that speeding up that step wouldn't have any impact on pharmaceutical development, which is one of his claims (closed-loop protein design).
Without being an expert in this matter, this seems wrong.
Sure you want quality here but there’s always going to be a human in the loop for this kind of work.
Any workflow with a human in the loop has this speed vs accuracy tradeoff.
While I’m not saying that speed trumps accuracy here, I don’t think you can dismiss without evidence that the tradeoff exists and speed might have benefits.
Lab work and clinical trials are incredibly slow. A single experiment testing a single candidate might take weeks (in cell lines), months (rodents) or even years (humans/non-human primates). You're going to do a bunch of them and they often require expensive reagents and/or tedious work.
Consequently, shortening the wait for a predicted structure by a few hours (or days) won't really move the needle. This is especially true if it makes your experiment, already probably a long shot, less likely to succeed.
GP is saying that the slow part of pharmaceutical development (synthesis,trials,etc) takes so many orders of magnitude longer than the software part (candidate generation) that any speed improvement is moot. In fact having lower quality software generated candidates merely leads to wasted time later.
I think it makes sense when you realize that the product (Galactica) and all the messaging around it are just PR - they're communicating to shareholders of a company in deep decline trying to say 'look at the new stuff we're doing, the potential for new business here'.
You interrupting the messaging ruins it, so you get some deniable boilerplate response. its not personal.
But we gave the keys to the economy to some vain children who have never had to do real work to make a name for themselves. Straight from uni being librarians assistants to the elders, straight to running the world!
Society is still led by vague mental imagery and promises of forever human prosperity. The engineering is right but no one asks if rockets to Mars are the right output to preserve consciousness. We literally just picked it up because the elders started there and later came to control where the capital goes.
We’re shorting so many other viable experiments to empower same old retailers and rockets to nowhere.
I'm delighted you called out these problems when you came across them, and sorry that he didn't have the grace or maturity to take it on board without getting defensive.
Like many thin-skinned hype merchants with a seven-figure salary to protect, they're going to try and block criticism in case it hits them in the pocket. Simple skin in the game reflex that will only hurt any chances of improvement.
It won't? AI-powered drug screening has definitely been overhyped, but in the longer term highly accurate protein structure modelling should let us understand protein-protein interactions and provide new opportunities for intervention.
Hrmm, I think I was expecting a different type of disagreement when you said he was wrong on facts.
I'm sure you know your stuff, and that you have a lot of experience with proteins that haven't helped with drug discovery or engineering, but it sounds like this is indeed a mismatch between predictions rather than facts.
It very well could be the case that speeding up certain problems by multiple orders of magnitude really does help with drug discovery, and this isn't factually inconsistent with the fact that solving those problems hasn't turned out to be useful so far in this area.
If you can discover a drug faster and that drug is as useful as dirt does it matter?
This isn't my field but I could grab a bunch of random jars off a shelf and pour them into capsules. No matter how fast I can do this won't improve medical outcomes for patients.
you just described how modern high content screening, which has been one of the most useful techniques for finding drug leads, works. Since lead-finding is a bottleneck in the drug discovery process, it has been highly effective because it can measure things that are not currently computationally accessible.
Don’t forget that we already have highly accurate protein structure modelling. AlphaFold adds to that but it’s not like it’s something radically new. Proteins involved in diseases we care about have been extensively studied.
Except a blackbox that simply spits out a resultant folded protein doesn't actually improve our understanding of anything. Are we just going to fuzz the blackbox with different protein combos hoping to find something useful? In that case, aren't we just more likely to find some stupid error in the ML predictor?
Improved structure prediction is mainly useful in hypothesis generation when doing hypothesis-driven science (IE, you want to confirm that a specific part of a protein plays a functional role in a disease). Its also a nice way to think creatively about your protein of interest.
THe problem is those distilled soundbites get learned by the next generation and they try to apply it. At least I will give AlphaFold/DM credit for correcting their language - originally they claimed AF solved protein folding, but really, it's just a structure predictor, which is an entirely different area. Unfortunately, people basically taught computer scientists that the Anfinsen Dogma was truth. I fell for this for many years.
> It states that, at least for a small globular protein in its standard physiological environment, the native structure is determined only by the protein's amino acid sequence.
Seems like "no true scotsman". If you present a counter example, they'll go "but this is only true for "small", the one you gave me isn't small.
Let's say you have a pool of smart first year grad students you want to inspire to work hard on problems for you for the next 7 years.
Do you say "we're going to give you a problem that is unsolved but likely has a general solution, and you have a chance of making progress, publishing, and moving on to a postdoc" or do you say "THis is an impossibly hard problem and you will only make a marginal improvement on the state of the art because the problem space is so complex and large"?
You say the first because it gets the students interested and working on the problem, only to learn many years later that the simplified model presented was so simplified it wasn't helpful. I fell for that and spent years working on drug discovery, structure prediction, etc, only to realize: while what Anfinsen said was true, it only applies to about 1% of protein space. It's not so much a "no true scotsman" as "some scotsmen wear quilts, and others have beards, but neither of those is sufficient to classify an example as scots".
It's tough because I think he has a really difficult job in many regards; Meta catches so much unjustified flak (in addition to justified flak) that being a figurehead for a project like this has to be exhausting.
Being constantly sniped at probably puts you in a default-unreceptive state, which makes you unable to take on valid feedback, as yours sounds like.
At some level he must know (AI) winter is coming along with the recession, which is why he is so defensive, as if a barrage of words will stave off the inevitable.
As a bright-eyed science undergraduate, I went to my first conference thinking how amazing it would be to have all these accomplished and intelligent people in my field all coming together to share their knowledge and make the world a better place.
And my expectations were exceeded by the first speaker. I couldn't wait for 3 full days of this! Then the second speaker got up, and spent his entire presentation telling why the first person was an idiot and totally wrong and his research was garbage because his was better. That's how I found out my field of study was broken into two warring factions, who spent the rest of the conference arguing with each other.
I left the conference somewhat disillusioned, having learned the important life lesson that just because you're a scientist doesn't mean you aren't also a human, with all the lovely human characteristics that entails. And compared to this fellow, the amount of money and fame at stake in my tiny field was miniscule. I can only imagine the kinds of egos you see at play among the scientists in this article.
I worked for a university for many years and I can confirm this. I have never seen such negativity, scheming, and fighting in my many professional years since. All because in the end, they were fighting over nothing and they knew it. But they needed to feel like what they were doing was important.
At the end of the day, all of the noise of negativity and bad press is being drowned out by incredible demos. I don't know what to chalk this up to if not jealousy. Most people in the ML-o-sphere are ignoring it.
At the end of the day, all that matters is: are users using what you built?
I mean.... That matters to someone in the industry, yes. But not necessarily someone in academia.
You chose industry over academia, and that's fine. It lines up with your values. But realize that not everyone shares those values and beliefs. To some, the act of discovering a new thing is much more important than the users using said discovery. And that lines up with academia more so than the industry.
It's fine that you like industry better than academia, so do I, but you'd better count your lucky stars that scientists exist.
> At the end of the day, all that matters is: are users using what you built?
How would you measure Isaac Newton's advances in calculus and mechanics or Einstein's general theory of relativity, against say, a web app with a billion users?
The two examples you picked are of incredibly and unusually useful advances in science. I have a friend in grad school who told me he deliberately didn't want to work on anything useful!
If you want to steelman the GP's argument, you should compare the web app with e.g. some niche in pure math. There the trade off between novelty/interest and usefulness to people today is more clear.
I think the two are incomparable and both useful, but it's disingenuous to strawman the GP as saying web apps are more useful than relativity.
As a former student of CS I could pick hundreds of examples of algorithms and data structures, which are now baked into the standard libraries of all programming languages, and therefore in web apps, which were invented in universities. Same with AI - industry is now collecting the fruits, but the groundwork research was absolutely indispensable and very few in industry were doing it until FB or Google set up their research institutes (and we could have another debate on whether those are academic, industrial or somewhere in-between).
Yes, I picked those two examples for the effect or as a reduction to the absurd (not a strawman), because going only by the immediate or tangible value of what one "builds" (science isn't even built, but rather discovered) is not a good way to dismiss academia.
One of the most intense and fun user bases I had was in HPC at an academic healthcare research institute. I've also worked in high energy research.
When most folks think of academia they think faculty, but staff vastly outnumber then. Contrary to popular belief there are legions of cold, level headed, engineers that get shit done.
A lot of the research isn't some random study of something that may or may not be useful in half a century or more, it's often immediately applicable and winds up in products or shaping government policy on a global scale. Especially the well funded ones.
But we don't hear about that stuff. We hear what the media and tech companies are currently trying to cram down our throats.
I have no insight into the natural sciences, but I've spent a couple of years in computer science academia. With that in mind:
> None of it would be possible without academia though. Industry just applies academic research.
Meh, that vastly oversells academic research. Very little of academic research in computer science is actually used in the industry. It's not that the industry is ignorant, but rather that the majority of academic work is useless: They create artificial problems [1] and solve them in shoddy ways, with hand-picked benchmark results, and frequently without even publishing the source code.
It's probably not surprising, given that the typical incentive is to get a PhD. So you need a "problem" that can reliably be solved in 3-5 years and which allows you to produce 5-10 conference papers with your name on it.
[1] I'm not talking about theoretical fields – my comment is purely about supposedly practical research.
I was once watching a VC interview a snooty machine vision scientist at Johns Hopkins who was talking up how well his research was at recognizing three d things. So the VC pulled out his cellphone and took a photo of a box on the table. He asked the professor to have the software highlight the rectangular solid. Whoop. He never heard back. The software in the lab that was supposedly so great couldn't do a very basic task that wasn't from its preapproved set of tasks.
I do think that academia can be the source of some great ideas, but they often end up believing their own BS.
I worked at Google and there's just tons of stuff that never actually existed in academia and was created, launched, and then replaced by something better entirely within the company without any publications!
I was working for somebody once who seemed to think LeCun was an uninspired grind and I'm like no, LeCun won a contest to make a handwritten digit recognizer for the post office. LeCun wrote a review paper on text classification that got me started building successful text classifiers and still influences my practice. LeCun is one of the few academics who I feel almost personally taught me how to do something challenging.
But the A.I. hype is out of hand. "A.I. Safety" research is the worst of it, as it suggests this technology is so powerful that it's actually dangerous. The other day I was almost to write a comment on HN to a post from lesswrong where they apologized at the beginning of an article critical of the intelligence explosion hypothesis because short of Scientology or the LaRouche Youth Movement it is hard to find a place where independent thought is so unwelcome.
Let's hope "longtermism" and other A.I. hype goes the way of "Web3".
AI Safety is important, but the unsafety isn't from the superintelligrnt AI, it's from dumb and cruel people hiding behind it as an excuse for their misbehavior.
Weapons of Math Destruction is good book on the topic. Using ML to do tasks like evaluate employee performance (and trigger firings), issue loans, insurance etc. is affecting peoples lives in the real world today.
Insurance has always been a statistically driven process and ML is just an advanced form of stats, so it makes perfect sense that the industry should use this technology.
I know of models that use things like typing speed and birth month to make loan decisions. Feed the AI inputs like race and it will make racist decisions.
I am not aware of models that use birth month to make loan decisions, but if that's a useful predictor, why not include it?
I'm not really sure exactly what you mean by "feed AI inputs like race and it will make racist decisions", as that's an absurdly general claim that doesn't even really say anything useful. If you mean the model reflects the underlying stats that correlate with race, class, gender, then sure, yes, I agree the predictor will do that. To say that it's racist is an entirely subjective claim.
THe point of insurance isn't to avoid social ills, but to statistically minimize the impact of risk on individuals (while making a tidy profit for the insurer).
Because birth month distribution is highly correlated with climate zone; see the heat-map in [1]. Climate zone, in turn, is highly correlated with race (and probably also immigration status in this case).
Even if we set ethics aside, this is still a terrible idea because credit risk analysis is highly regulated in most cases. There is enormous regulatory risk to using birth month in a credit risk model. Particularly because hen a jury asks "what's an alternative causal model that could explain the bank's incorporate of this data point into their models?" their answer is going to be "absolutely none".
> I am not aware of models that use birth month to make loan decisions, but if that's a useful predictor, why not include it?
Because it's fucking unfair, and humans hate feeling like they've been wronged for unfair reasons they cannot control. How would you feel if your loan got denied, you asked why, and you found it it would have been approved if you had been born in April instead of June? Don't pretend you'd love it and smile. You'd rant about it and tell all your friends how bad that lender is.
If the lender rejects you because your debts are too high or your income too low or something, or the zip code of the home you're mortgaging, that makes more sense and feels more relevant and in your control, at least.
Yeah, it's nice from the point of view of the lender, but the cost is too high to society.
Society feels so strongly about this that (in the USA) we've even passed all kinds of laws about what pieces of information are absolutely banned from being used to determine who gets a job or who gets a home. If you are a landlord or hiring manager you have to be aware of them so you can at least pretend to comply.
Sometimes I wonder if AI is already here secretly. Half of my replies on hacker news seem to basically boil down to teaching what humans are actually like.
> How would you feel if your loan got denied, you asked why, and you found it it would have been approved if you had been born in April instead of June?
I mean, I pay more for car insurance because I am immutably male. I don't feel great about this, but assuming that males my age are statistically more likely to be in crashes this makes total sense.
Likewise, I don't feel great that even if I were really good at basketball I'd be way less likely to make the short list of an NBA recruiter, because statistically I'm not going to be as good as someone two feet taller. It may not feel great, but it still absolutely feels fair too.
If people born in April instead of June were actually statistically more likely to be in crashes, I don't see how this is any different or unfair.
If I understand correctly, most modern insurance models contain both a general risk ("males are more dangerous so their premiums should be higher") and personal risk ("this male has driven without an accident for 20 years, which is better than the average male, so we categorise them as less risky").
This is a naive answer and neglects (1) models for housing loans, insurance, etc. are trained on historical data. (2) Policy (in the USA) at the time of data collection, and therefore currently was racist (this is not subjective).
Historically, at least in the United States, loans were unavailable to PoC, especially black Americans. Districts in many American cities were "redlined"[1], that is to say certain districts of cities were deemed "unprofitable" for banks. Redlining was policy and was targeted towards discriminating against black Americans, who were often victims to predatory loans with unserviceable interest rates, which commonly resulted in defaults. The defaults caused worse scores for people in the neighborhood and created a positive feedback loop. Historical housing loan data (and insurance data) includes this data. People today are affected by this historical data and this absolutely must be taken into account by the developers of these models.
Because of the entangled nature of real data, dropping "race" as a feature for training wouldn't solve the problem. Factors like zip codes (think of the redlined districts) would also influence the outcome[2].
Creating models, whose output can impact people so profoundly, e.g. can Jane get health insurance, calls for more reflection than just "numbers correlate!".
There isn't an "easy" solution, but step 0 is recognizing that there is an historic problem that is being dragged with us into the future because of the way our current systems work.
Political solutions are necessary, maybe something like subsidized loans for people from formerly redlined communities to purchase and restore homes, or start businesses. Urban planning projects, like increasing mixed-zoning, pedestrian traffic, and good public transportation would help keep money in the neighborhood. Then there is the question is how to deal with gentrification and increase quality of living in a community without displacing people from that community. It takes a team of experts from various fields, the community itself, and goal-oriented cooperation.
I am not aware of models that use birth month to make loan decisions, but if that's a useful predictor, why not include it?
Because it’s radically unfair to do that! You’re seriously suggesting that some people should be penalized for the day they were born?
Taking the point more seriously, it’s hard to believe that the risks for whatever you’re ensuring have populations of people each month that are so different that a random person from each has a risk associated with them that birth month, as a signal, isn’t likely to discriminate unfairly almost as much as it helps. Naively, it’s likely to unfairly penalize almost half the population. Maybe you’re prepared to accept that because, hey, it’s right over half the time, but tell that to the vast population you’re penalizing.
And some things just shouldn’t be included. Health insurance, for example: isn’t it unfair to include risk of cancer caused by genetic factors in premium calculations? I guess you could say that’s different because parents could control birth month, but, man, that just seems so uncaring to me.
In the race example, let’s say there is a discrepancy in the underlying stats now, but it’s caused by structural problems in society caused by racism in the past. By using that as a signal to discriminate now it makes it more unlikely that the problems will ever be resolved (example: Black people cant get loans as easily, so can’t start businesses as easily, so the wealth gap is reinforced). If you think that’s ethical I would suggest that you are considering statistics too much and people too little.
If there was a reason for people to be higher statistical risk based on birth month (say, "babies born in november had 0.5% higher risk of long-term respiratory disease") then, yes, that should be incorporated into a model.
Genetic risk of cancer cannot be included in insurance decisions (excluding voluntary life insurance) because a law was passed (GINA) preventing it.
It's not the role of the loan industry to make it easier for black people who appear to be a higher statistical risk to get loans. That's the purpose of government- to make it illegal to include genetic history in life insurance models- provided that society believes in that principle.
It's not the role of the loan industry to make it easier for black people who appear to be a higher statistical risk to get loans. That's the purpose of government- to make it illegal to include genetic history in life insurance models- provided that society believes in that principle.
Another way to say that - one that appears less uncaring to readers - would be “yes, including birth month or race in insurance risk calculations would be unfair+, so it should not be done. If necessary, the government should step in to prevent this”.
+ you may squabble with the use of ‘unfair’ here - obviously it is not ‘unfair’ in the stastical sense, but that’s not what the word generally means in spoken English. ‘Inequitable’ would be better, but for some reason ‘equity’ seems to be evolving to mean ‘at all costs, produces equality of outcome’ and so is a victim to the same problem in the opposite direction. I chose the more common word.
[edit: I used insurance here, sorry - the same would apply to loans I think]
I’ve been thinking about this more, and it’s occurred to me that there is an underlying assumption in It’s not he purpose of the loan industry… that corporations are required to make decisions amorally - which is don’t think is or should be the case. That may be an underlying schism in our argument.
You are correct. I expect that corporations understand they exist in a competitive environment and incorporating morality represents a real risk that your competitor will replace you, because morality is not profitable.
I do agree that if you expect corporations to act morally, then it would be sensible for them to be more careful in understanding and correcting long-term social ills.
Is it the role of the loan industry to make it more difficult for black people to get loans, regardless of whether the individual is a higher statistical risk? (Apparently so, since that's the way it's always worked, right?)
It is trivial to bake an irrelevant distinction into a naive ML driven process.
An insurance model that uses data on financial stability in a society where a specific race of people have been, through bigotry alone, forced into a less stable position by default, will be bound to perpetuate the original bigoted patterns of behavior. It's not inaccurate to describe the model as therefore racist because it's modeling itself off a risk assessment where certain groups have historically been forced into the fringe and are therefore inherently risky.
I'm sure if black wall street wasn't razed to the ground by a racist white mob (among several other attempts to gain stability, wealth, etc. by black people being destroyed by racist white anger) then maybe the model wouldn't need to "reflect the underlying stats that correlate with race...". But those underlying stats, and those correlations, didn't just happen in a vacuum, hewn out of the aether like a magical and consequentless thing.
The points you are making are entirely political and apply generally, not just to insurance. Insurance is a business and it's not the job of the insurer to correct long-standing social ills.
Whether or not my points apply generally does not mean it does not apply to insurance. In fact, it very likely means insurance is a subset of generally, and therefore insurance also has to deal with racism. You cannot extricate models of society from their ills by pretending the ills aren't relevant.
Generally when a loan company is training a model, they don't approve every application for a month and see if they're repaid to get unbiased data.
Instead, they look at who was approved, and maybe whether they repaid their loan.
So if a human making loan decisions isn't keen on giving out loans in View Park-Windsor Hills, a model trained to produce human-level performance won't be keen on doing it either.
This place was using any and all data that improved the fit. What browser you used, typing speeds, and birth month all improved fit in training and worked in the test set and got thrown in. That was the only hurdle, they didn’t care as to why, they just added the data.
I thought it was one of the most deceptive books I ever read! It contradicts itself enough that it is often obviously wrong even without conceptual knowledge of the field.
A book called Automating Inequality by Virginia Eubanks touches on how for many years now, we've hid unfair policies behind computer complexity. It was really eye opening to me how what seems to be a poorly implemented system is actually working as intended in order to make life difficult for certain people.
Inequality has existed long before computers and is a social problem not a technical problem. You don't solve social problem with technical solutions since people are clever enough to simply find the next tool to enforce the social status quo using.
You are 100% right. The book just laid out exactly what you are saying.
Computers just made the enforcement easier and with less opportunities to break out of it - for example, a sympathetic public servant no longer has the power to make exceptions since "the computer won't allow it."
^ and this isn't just a public service problem, either.
I'm making this number up, but I swear that 90% of the time when I come across a company that's absolutely floundering under the weight of its own poor decisions, it's because those decisions are enforced by software such that people CAN'T work around it.
What ends up happening is doing anything crosses 15 silo's and each of those silo's absolutely enforces it's authority such that no single person is able to bridge the gap between even two of them.
What makes it hard is that there are legitimate reasons why you don't want a single person being able to write code, push it to production, open firewalls, open access to databases, add ACL's, etc. Depending on the industry, the need for controls should exist, and because there's SOME legitimacy there it gets pushed across the threshold past reasonableness to absolute pain for everyone involved.
---
And if I may, that's where the most dangerous solutions lie. When there's a legitimate use of them but they can be abused.
Not familiar with the book, but hasn’t ‘sympathetic public servant with power to make exceptions” - major source of inequity? Giving/not giving exceptions for reasons of bias or through bribes?
It depends on the intent of the rules being enforced.
An example given was a state wanted to spend less on low income benefits. They setup an overly complex computer system that decides eligibility and requires participants in the program to fill out multi page forms 100% accurately on a regular basis or their benefits are suspended immediately. There was no way to appeal this decision, only to re-apply.
Employees of the state agency no longer had any insight into why a form was rejected, and could not speed up the process to resubmit it because the participant had long term medical issues and needed potentially lifesaving prescriptions filled. As a result, many people who would otherwise be eligible for benefits had them denied.
This lead to a marked decrease in spending on social services and was considered a win by the state.
Close but no cigar... It replaces one inequality (bias against those the officials at the top don't like, in favor of those they do) with another one (Bias against those the officials at the bottom don't like, in favor of those they do)
In practice it creates more freedom, because users have no choice in their top authorities (I'd have to move to another state or country if I don't like the laws) but often have a lot of choice in bottom level authorities (if the guy at the DMV is mean I can go to a different location)
I'm not saying it's strictly good, but it's much more complex. IF you are someone who truly believes the top level policy is good and perfect and has no problems, then yes, any kind of bottom level control.
But if your are someone who is harmed by the top level policies, you now have an escape hatch by which a public servant can help you overcome this.
For example once in NY I tried to register a car I had bought out of state and one DMV worker told me I could not because the previous owner was named Johnathan Smith but had signed his name as John Smith. Ridiculous, but it's possible they were following the letter of the law and an automated system would have said the same.
I simply went to a different DMV where a different worker took a more favorable interpretation. If it had been a piece of software making this comparison I would have been screwed.
Funny that I have another anecdote, also for the DMV.
I owned a vehicle for 15 years and it broke down on me, as a result I dropped insurance on it as I wasn't sure if I wanted to keep it or not.
fast forward 3 months and it turned out to be a relatively cheap repair so I went to get it tagged (it had expired while uninsured) and the woman at the DMV absolutely hammered me on the paperwork I had to show her to the point that even the agent at the insurance company was confused about what she wanted.
I go to another location and lo-and-behold it's no longer a problem. Maybe she was in a bad mood, maybe she didn't like the way I looked. I have no idea, but the point you're making is absolutely relevant in day to day life.
And if I can pontificate slightly further, inequities exist in life, there is no way to completely remove them. Giving autonomy to low level people and squashing the ones that abuse that autonomy is absolutely better for people overall, even if it isn't perfect.
If someone (a sympathetic public servant) is doi g something to fight inequity, they would not be a "major source of inequity" themselves, no.
That isn't to say that corrupt public servants don't exist, and that they don't look nearly identical if you were to take a brief glance of their actions without looking at their reasoning, but that doesn't mean they're the same people or causing the same end effect.
I think the problem in this comment, and the mistake many others make, is that they consider their viewpoint canonical, and everyone else's viewpoint wrong.
"Sympathetic" and "source of inequity" are subjective. Letting someone get extra points on their test because of their race is equitable to some, unequitable to others. Who decides?
There's an argument to be made that differing opinions of equity spread across many dozens of public servants might be a better approximation of true equity than a single opaque model embedded into a computer and applied to all.
Equity is not the same as equality and was never intended as a way to gauge equality.
Equity, by its very definition, is unequal. Every single thing done in the name of equity is unequal. On purpose. Because the "job" of equity is to right inequalities in the system.
That doesn’t address the issue. Computers execute stored human knowledge. If that knowledge contains bias, then computers can absolutely propagate that and amplify it.
So yea, inequality is a social problem, but it is amplified by technical problems. We’ve seen this time and time again, the Internet at large being the most glaring example.
We shouldn’t ignore this amplification effect while we wait to solve the social problems. Furthermore, computers are often viewed as some unbiased decision machine, which makes the problems worse.
The Internet and Zoom are good things in general. But there was never any valid need for lockdowns at all, and it should be clear to everyone now that lockdowns caused more problems than they solved. So, I don't understand your point.
The real social problem we ought to be solving is stripping bureaucrats and politicians of their ability to impose lockdowns on us at all. And there is no technical solution for that.
> But there was never any valid need for lockdowns at all, and it should be clear to everyone now that lockdowns caused more problems than they solved. So, I don't understand your point.
That's not logical. There's no valid need for traffic, but I appreciate Google Maps traffic reports.
One could contend that perhaps the greater social problem solved by Zoom and the Internet was "How do we allow people to have more lucrative jobs without them paying xx% of their salary to live within a commutable area of an in-demand city"?
Like the post on here a while ago about ai algorithms setting prices for landlords. Most human have a limit to what they will personally do to someone, but for whatever reason if they get told to do something worse and they can blame it on that thing instead people are willing to be monsters. So it only takes a few sociopaths making the software to make a whole industry even worse.
"Nuclear Weapon Safety is important but the unsafety isn't from the Bombs but from dumb and cruel people hiding behind them as an excuse for their misbehavior."
The most aggrivating thing about EA "longtermism" AI Safety stuff is that is takes the oxygen out of the room for actual AI safety research.
Using ML for object detection, object tracking, or prediction on an L2-L5 driver assistant system? AI safety research sounds like a capability you'd really want.
Using ML for object detection, object tracking, or prediction on an industrial robot that is going to work alongside humans or could cost $$$ when it fails? AI safety research sounds like a capability you'd really want.
Using classifiers or any form of optimization for algorithmic trading? AI safety research sounds like a capability you'd really want.
Building decision support systems to optimize resource allocation (in an emergency, in a data center, in a portfolio, ...)? AI safety research sounds like a capability you'd really want.
Hell, want to use an LLM as part of a customer service chatbot? You probably don't want it to be hurling racial slurs at your customers. AI safety research sounds like a capability you'd really want.
Unfortunately, now "AI Safety" no longer means "building real world ML systems for real world problems with bounds on their behavior" and instead means... idk, something really stupid EA longtermism nonsense.
I'm going to add that "AI Safety" (allied with longtermism) is also part of the hype machine for big tech.
Pressing the idea that AI is dangerous makes it seem like these companies are even more powerful than they are and could drive up their stock price. When the AI Safety people get into some conflict and get fired they are really doing their job because now it looks like big tech is in a conspiracy to cover up how dangerous their technology is.
You just listed 5 completely distinct applications of AI safety (and surely can name countless others) and then concluded with a complaint that the concept of AI safety is not well-defined?
The whole point is that the technology itself and its capabilities are not well defined, people are constantly inventing new applications and new methods now at breakneck speed, so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications.
Before 2016 or so, "AI Safety" meant the types of things I listed above: how to design AI systems that are safe to use in safety-critical settings.
Unfortunately, the term "AI Safety" has been overrun with effective altruist longtermism folks talking about something closer to https://en.wikipedia.org/wiki/Existential_risk_from_artifici... which is ill-defined and, frankly, usually completely vapid nonsense.
> so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications.
No, no, no!!! AI safety methods -- methods that really increase confidence in real systems -- are not generic. ALL of the useful AI safety work I know of is deeply related to one or more of: the specific model architecture, the specific optimization algorithm, the specific data, or the specific application. Almost always all of the above.
“Field is squishy” != all parts of the field are equally squishy. The presence of some clear problem-and-solution pairs does not preclude the existence of more ambiguous problems. This is especially true as we’re deploying AI against increasingly vague goals with increasingly vague constraints, such as in the case of the non-lying Diplomacy bot that absolutely did deceive human players.
> But the A.I. hype is out of hand. "A.I. Safety" research is the worst of it, as it suggests this technology is so powerful that it's actually dangerous.
It boggles my mind how anyone can think otherwise. Existential dangers of superintelligent or even non-intelligent AI are the long-term result of the dangers of AI being developed and misused over time for human ends.
It's the exact same argument behind why we should be trying to track asteroids, or why we should be trying to tackle climate change: the worst-case scenario is unlikely or in the future, but the path we're on has numerous dangers where suffering and loss of human life is virtually certain unless something is done.
Because it's like cavemen pondering the safety of nuclear fusion after discovering fire. Yes: nuclear fusion could be dangerous. No: there is nothing useful that can come out of such "research".
You know what would be useful for cavemen to ponder? The safety of fire. Or you know, just staying alive because there are more dangerous things out there.
The current state of so-called "AI" is our fire. It's impressive and useful (and there are real dangers associated with it) but it has no bearing on intelligence, let alone superintelligence. It's more likely that a freak super-intelligent human will be born than that we accidentally produce a super-intelligent computer. We produce a lot of intelligent humans, and we've never produced a single intelligent computer.
As it stands, we don't have the understanding or the tools to do anything useful wrt safety from a super-intelligent AI. We do have the understanding and tools to do something useful about asteroids and climate change.
> No: there is nothing useful that can come out of such "research".
That's just wrong. This very paper we're discussing has AI safety factors built-in: the AI is not supposed to be able to lie, and yet it exhibited deceptive behaviours anyway. That falls squarely under AI safety, and is a pretty useful observation to inform future attempts.
> The current state of so-called "AI" is our fire. It's impressive and useful (and there are real dangers associated with it) but it has no bearing on intelligence, let alone superintelligence.
That's conjecture. We don't know what general intelligence really is, so you simply cannot gauge how close we are. For all you know we could be one simple tweak away from current architectures, and that should be terrifying.
> As it stands, we don't have the understanding or the tools to do anything useful wrt safety from a super-intelligent AI.
Even if that were true, which it's not, we certainly won't develop that understanding or those tools if we don't start researching them!
> We do have the understanding and tools to do something useful about asteroids and climate change.
Suppose you had a time machine and went back to the 19th century to explain the dangers of asteroids wiping out all of humanity. They didn't have the launch capability or the detection abilities we do now, but if they were sufficiently convinced of this real threat, does it not seem plausible that they could be motivated to more heavily fund research into telescopes and rockets? Couldn't we plausibly have reached space sooner and be even better prepared in the present to meet that threat than we are now?
Instead of a time machine, couldn't we just use our big brains to predict that something might actually be a serious problem in the future and that maybe we should devote a few smart people to thinking about these problems now and how to mitigate them?
This all just strikes me as incredibly obvious, and we do it in literally every other domain, but somehow say "AI" and basic logic goes out the window.
The word "safety" doesn't normally encompass lying or, more appropriately in this case, saying something untrue without realizing it. That's considered a very different kind of problem. Safety normally means there's a direct chance of physical harm to someone.
This kind of elasticity in language use is the sort of thing that gives AI safety a bad name. You can't take AI research at face value if it's using strange re-definitions of common words.
How exactly does honestly not clearly fall under safety? AI is an information system, and truthfulness from an information system that impacts human lives is clearly a safety concern.
This is not a redefinition, the harm results from the standard usage of the tool. If the AI is being used to predict the possible future behaviour of adversarial countries, then you need the AI to be honest or lots of people could die. If the AI concludes that your adversary would be more friendly towards its programmed objectives, then it could conclude lying to the president is the optimal outcome.
This can show up in numerous other contexts. For instance, should a medical diagnostic AI be able to lie to you if lying to you will statistically improve your outcomes, say via the placebo effect? If so, should it also lie to the doctor managing your care to preserve that outcome, in case the doctor might slip and reveal the truth?
How much software is safety critical in general, let alone software that uses deep learning? Very, very little. I'd actually be amazed if you can name a single case where someone has deployed a language model in a safety critical system. That's why your examples are all what-ifs.
There are no actual safety issues with LLMs, nor will there be any in the foreseeable future because nobody is using them in any context where such issues may arise. Hence why you're forced to rely on absurd hypotheticals like doctors blindly relying on LLMs for diagnostics without checking anything or thinking about the outputs.
There are honesty/accuracy issues. There are not safety issues. The conflation of "safety" with other unrelated language topics like whether people feel offended, whether something is misinformation or not is a very specific quirk of a very specific subculture in the USA, it's not a widely recognized or accepted redefinition.
> I'd actually be amazed if you can name a single case where someone has deployed a language model in a safety critical system. That's why your examples are all what-ifs.
AI safety is not a near-term project, it's a long-term project. The what-ifs are exactly the class of problems that need solving. Like it or not, current and next generation LLMs and similar systems will be used in safety critical contexts, like predictive policing which is already a thing.
Edit: and China is already using these things widely to monitor their citizens, identify them in surveillance footage and more. I find the claim that nobody is using LLMs or other AI systems in some limited safety critical contexts today pretty implausible actually.
None of that is even remotely plausible. You're just making things up with zero basis in actual science. Essentially you are making a religious argument. There's nothing wrong with religion per se, but don't expect the rest of us to take action based on your personal faith or irrational fears.
Just calling something a "religious argument" as a way to dismiss it is pretty silly.
And there's lots of actual, real scientists who think AI risk is a real thing we should be concerned with. Both within the field (e.g. Stuart Russel) and outside of the field (e.g. Stephen Hawking.) Are all of these scientists also talking with zero basis in actual science?
Artificial general intelligence does not exist (yet). As of today, there are no actual scientists in that field. But even smart people love to pontificate about things they don't understand.
We don’t produce intelligent humans nature does. Otherwise I agree. We’re learning what the end result of intelligence looks like (predictions) not the actual intelligence yet.
The intelligence is what determines the shape of these networks so they can even learn a useful pattern. That’s still gonna be humans for the foreseeable future.
Yes, nature produced intelligence while optimizing only for reproductive fitness. In other words, intelligence emerged spontaneously while solving a seemingly completely unrelated problem. That should terrify you.
Current AI research largely isn't focused on general intelligence, but the possibility remains that it could still spontaneously emerge from it. We can't even quantify how likely that is because we don't understand intelligence, so whatever intuition you have about this likelihood it's completely meaningless and not based on any meaningful data. We're in uncharted waters.
That intelligence is a emergent property of reproductive fitness isn't terrifying- it's amazing and exciting and suggests a wide range of scientific opportunities.
For a while I ran a project that ran a (scientific) binary on every Google machine in production using all the idle cores on the machine to do protein design (it worked!). At the end of the project we seriously considered making the binary instead a learning system that had access to 'sensors' (IE, signals from Google production machines) and 'outputs' (the ability to change production parameters) and just let that run autonomously. I figured Google prod is one of the few computational/networking systems with enough inherent complexity and resources that spontaneously emergent general intelligence was not inconceivable. However, the project was deemed too risky by SRE, rightfully so. Not because of emergent intelligence but emergent disruption to serving ads.
> That intelligence is a emergent property of reproductive fitness isn't terrifying
You missed the terrifying aspect. It's not terrifying that general intelligence emerged from optimizing for reproductive fitness specifically, it's terrifying that general intelligence emerged while nature was stumbling around randomly in the dark trying to optimize for a complete unrelated property.
This is a strange comparison that doesn’t seem logical to me. If we created intelligences and set them in competition with each other we may get something to evolve millions of years later.
General intelligence didn’t poof into existence it happened over half a billion years and even then it came from fungi which is itself an intelligence.
Minor technical correction- it would be false to say that GI came from fungi as we humans have GI, but don't descend from fungi (we share a common ancestor) and likely haven't received the genes for GI (horizontal gene transfer) from them.
Personally I figure that once you have a complex enough system with feedback regulation and self-preservation, something like general intelligence occurs spontaneously through the normal mechanisms of evolution (probably more than once in different kingdoms) because it provides a heritable survival edge.
> If we created intelligences and set them in competition with each other we may get something to evolve millions of years later.
Why millions of years? Do you agree or disagree that humans can develop technology faster than nature could evolve it on its own? It took maybe 70 years for computers to go from not existing, to matching humans on visual recognition tasks.
As you just acknowledged, it took nature at least billions of years to evolve humans. Is not focused technological development obviously orders of magnitude faster at evolving intelligence than nature? Does it not then follow that artificial general intelligence is a lot closer than an argument based on natural evolution might imply?
It happened over billions of generations, and an AI generation can be millions of times shorter than a biological one. One of those is also shrinking rapidly.
> It boggles my mind how anyone can think otherwise.
Some AI dangers are certainly legitimate - it's easy to foresee how an image recognition system might think all snowboarders are male; or a system trained on unfair sentences handed out to criminals would replicate that unfairness, adding a wrongful veneer of science and objectivity; or a self-driving car trained on data from a country with few mopeds and most pedestrians wearing denim might underperform in a country with many mopeds and few pedestrians wearing denim.
But other AI dangers sound more like the work of philosophers and science fiction authors. The moment people start predicting the end of humans needing to work, or talking about a future evil AI that punishes people who didn't help bring it into existence? That's pretty far down in my list of worries.
> But other AI dangers sound more like the work of philosophers and science fiction authors.
Your ability to read this sentence right now when we have never met and may not even be on the same continent was once in the domain of science fiction. Don't underestimate technological progress, and specifically, don't underestimate the surprising directions it could go.
Some fantastical AI predictions will happen, most probably will not, and some utterly terrifying ones no one foresaw will almost certainly happen. The unknown unknowns should worry you, and AI is full of them.
> The unknown unknowns should worry you, and AI is full of them.
Sure, but where should that rank in my worries relative to 'designer babies' and 'rise of authoritarian states as economic powerhouses' and 'corporations that can commit crimes with impunity' and 'rising medical bills' and 'widening gap between rich and poor' and 'far right extremism' and 'water shortages' and 'economic crisis wipes out my savings' and 'cyber warfare targeting vital infrastructure' and 'rising obesity' and 'voter suppression' and the many other things a person could worry about?
In my view, the only wrong opinions on where to rank this are "at the very top" and "at the very bottom or not at all". We will only know the correct answer in hindsight, so the sensible position is to just start funding some legitimate AI safety research.
A whole lot of the arguments about why we shouldn't be concerned really boil down to "I cannot conceive of risk until that risk has materialized." Impossible to argue against, really.
"the worst-case scenario is unlikely or in the future ... loss of human life is virtually certain"
These two things are in conflict. We could ignore both asteroids and climate change and according to the best known science there'd be very little impact for vast timespans and possibly no impact ever (before humanity is ended by something else like war).
Yes, also for the climate. Look at the actual predictions and it's like a small reduction in GDP growth spread over a very long period of time, and that's assuming the predictions are actually correct when they have a long track record of being not so.
Really stuff like asteroids and climate is a good counter-argument to caring about AI risk. Intellectuals like to hypothesize world-ending cataclysms that only their far sighted expertise can prevent, but whenever these people's predictions get tested against something concrete they seem to invariably end up being wrong. Our society rewards catastrophising far too generously and penalizes being wrong far too little, especially for academics, NGOs etc. It makes people feel or seem smart in the moment, and they can punt the reputational damage from being wrong far into the future (and then pretend they never made those predictions at all or there were mitigating factors).
> We could ignore both asteroids and climate change and according to the best known science there'd be very little impact for vast timespans and possibly no impact ever (before humanity is ended by something else like war).
That's just incorrect. The Tunguska event was a nuclear-weapon scale asteroid. These are predicted to happen once every hundred years or so. If it happens over a populated city millions would die. If a person wrongly concludes this was a surprise nuclear attack, maybe everyone would die, to say nothing of the real risk that the asteroid itself could be big enough to wipe us all out.
There's a lot of uncertainty around climate change, but changing climate patterns will certainly change resource allocations (fresh water, arable land, etc.). This will lead to shortages in places that were once abundant, which could easily lead to wars in which millions die.
> Really stuff like asteroids and climate is a good counter-argument to caring about AI risk.
This. This boggles my mind. Long-tail risks exist, and burying your head in the sand and pretending they don't just places millions of lives at risk, and potentially the entire human race. You don't have to think these are top priorities, but to dismiss them as complete unimportant is frankly bonkers.
"The Tunguska event was a nuclear-weapon scale asteroid"
Which had very little impact on humanity because it exploded in the middle of the tundra.
"These are predicted to happen once every hundred years or so"
What is predicted exactly, by whom and how were these predictions validated against testable reality given the postulated rareness? If they're so common then why is it so hard to name the last 10? I think in reality these events are very rare and will almost always happen over the oceans, deserts, poles etc where not many people live.
"Long-tail risks exist, and burying your head in the sand and pretending they don't"
They exist and I am not pretending they don't. I am saying that this style of reasoning in which an extremely unlikely event is unfalsifiably and arbitrarily assigned near infinite downsides in order to justify spending time and resources on it, is problematic and as a society we are far too generous towards people who do this.
Luckily, it again happened near a depopulated zone, but the damage was still extensive.
> I think in reality these events are very rare and will almost always happen over the oceans, deserts, poles etc where not many people live.
Most car accidents will happen to bad drivers, so if you're a good driver you don't need to wear your seatbelt, amirite?
The fact that an easily preventable event typically happens without much damage is no consolation when that's not the case.
> They exist and I am not pretending they don't. I am saying that this style of reasoning in which an extremely unlikely event is unfalsifiably and arbitrarily assigned near infinite downsides
Your mistake is thinking the likelihoods assigned are arbitrary and unfalsifiable. By your logic, COVID-19 was unlikely as most outbreaks are small and isolated, and the likelihood of a global contagion so unfalsifiably remote it's not worth thinking about. Therefore pandemic preparation is a waste of time and money. Now that 6 million people have died, that view doesn't look so rosy in hindsight.
The calculations on asteroid threats have been done based on known data, and even with our current preparations we still miss some potentially devastating ones like Chelyabinsk:
The so called "AI safety" researchers issuing breathless warnings about the dangers of superintelligence are nothing but grifters. They are the modern equivalent of shaman, telling the common people that they need ever increasing amounts of resources to continue their vital work in protecting us from wrathful spirits. There is zero actual scientific evidence to support their claims. It is essentially a modern-day secular religion.
(There is value in doing research and ethical analysis into AI/ML statistical algorithms to prevent hidden biases or accidental physical harm. People working in those areas are producing real benefits for the rest of us and I'm not criticizing them.)
That's akin to asking "what sort of human catastrophe do you think would happen?" A whole host of things are possible in principle on a long enough timeline. If you constrain the timeline, then you'll get more plausible answers but with potentially wide error bars because we don't really understand what "intelligence" really is, ie. current research could be pretty close but could also be pretty far from it.
The fact that we don't know how close we are is itself dangerous. It's like doing gene editing on pathogens without a proper understanding of germ theory and biosafety. That's where we are with AI.
Do you actually have empirical data demonstrating that scientists aren't concerned about AI, or that "most" AI safety doesn't involve scientists, or is this just your gut feeling?
Edit: for instance, here's a computer science professor and neurologist who published a book about all of the serious dangers of AI, and calling for funding safety research:
Furthermore, AI safety programs are hiring AI researchers and computer scientists to actually do the work, so are you claiming that these people don't sincerely believe in the work they're doing?
>But the A.I. hype is out of hand. "A.I. Safety" research is the worst of it, as it suggests this technology is so powerful that it's actually dangerous. The other day I was almost to write a comment on HN to a post from lesswrong where they apologized at the beginning of an article critical of the intelligence explosion hypothesis because short of Scientology or the LaRouche Youth Movement it is hard to find a place where independent thought is so unwelcome.
I hesitate to say "safe space", but... what if a group of people wants to come together discuss AI safety? If they'd have to regurgitate all the arguments and assumptions for everyone who comes along they'd never get anything done. If you are really interested to know where they are coming from, you can read the introductory materials that already exist. If the 99.9% of the world is hostile towards discussing AI safety (of the superintelligence explosion kind, not the corporate moralitywashing kind) there is some value in a place which is hostile to not discussing it, so that at least those interested can actually discuss it.
> it is hard to find a place where independent thought is so unwelcome.
Is that actually true, though? It's true that a higher fraction of the people in that community give credence to the intelligence explosion hypothesis than pretty much anywhere else. (This is what one would expect, since part of the purpose of LessWrong is to be a forum for discussions about super-intelligent AI.) But even if the intelligence explosion is a terrible, absolutely-wrong theory, that doesn't prevent the people who hold it from being open-minded and tolerant of independent thought. Willingness to consider new and different ideas is something the LessWrong community claims to value, so it would be a little bit weird if they were doing way worse than average at it. And AFAICT, it seems like they're doing fine. Some examples:
- Here [1] is a post critical of the intelligence explosion theory. It has 81 upvotes as of this writing, and the highest upvoted comment goes like: "thanks for writing this post, it makes a lot of good arguments. I agree with these things you wrote" (list of things) "here are some points where I disagree" (list of things). This may even be the original post you were talking about in your comment, except that it doesn't start with an apology.
- LW has 2 different kinds of voting: "Regular upvotes" provide an indication of the quality of a post or comment, and "agree/disagree votes" let people express how much they agree or disagree with a particular comment. Down-voting a high quality comment just because you disagree (instead of giving it a disagree-vote) would be against the culture on LW.
If you're already sure that LW is wrong about superintelligence, and you're trying to explain how they became wrong, then "those LW people were too open minded and fell for that intelligence explosion BS" makes more sense to me than anything about suppression of independent thought.
> "A.I. Safety" research is the worst of it, as it suggests this technology is so powerful that it's actually dangerous
Well that's an interesting way to misrepresent an entire important field of research based on what a few idiots said. There are serious people in that field who aren't addicted to posting on LessWrong.
I studied machine learning at NYU, and from interacting with Yann LeCun, I can say he’s actually a nice guy. Yes, his tweet is grumpy. I still feel as if the implication that Galactica should have been taken down was the worse thing happening here.
I read the MIT Technology Review article, and I was asking myself “what is an example of Galactica making a mistake?” The article could easily have quoted a specific prompt, but doesn’t. It says the model makes mistakes in terms of understanding what’s real/correct or not, but the only concrete example I see in the article is that the model will write about the history of bears in space with the implication that it’s making things up (and I believe the model does make such mistakes). I don’t think it’s a good article because it’s heavy on quoting people who don’t like the work and light on concrete details.
Does the imperfection of a language model really mean the model should not exist? This seems to be what some critics are aiming for.
I saw examples of people using it to generate scientific-sounding fake studies like: the benefits of eating glass, or promoting antisemitism.
That being said, I am very partial to the AI researchers here who feel like their cool demo has to be taken down because some people were misusing it. It's an unfair high standard they're holding AI demos to, compared with other technologies. It's analogous to asking Alexander Graham Bell to shut down an early telephone prototype because some jerks were using it to discuss antisemitic conspiracies.
I agree that the model does make mistakes. Your examples sound realistic, and I hope we make more progress to improve on models that can propagate stereotypes and similarly negative aspects that can arise in training data. I had meant to criticize the journalism, versus saying the mistakes don't exist.
It is the same problem as with all the AI models that are “racist” (note some are actually racist or insensitive): the AI model just does what it’s told, doesn’t know right from wrong, and amplifies fake and actual differences in ways that make us uncomfortable.
So you can get an AI model that hasn’t been hardened against these attacks to write a paper on why <racist thing> or create an image depicting <racist or porny thing> and it just does it. Because the model is just an input:output device and doesn’t have the “wait maybe I shouldn’t do that because it’s bad” feature.
And while teen and young 20 something males will get a huge laugh out of posting screenshots of it, the journalist crowd will freak out and start calling the model, researchers, and company racist.
Personally I have played with large language models and the chutzpah with which they will lie and make things up is indeed astounding (they do a good job at making them sound believable, and lie with utter seriousness and confidence). So I can see where the controversy comes from, although I agree with the other commenters that the researchers should be able to put up a bit fat disclaimer about it
I kinda agree with LeCun here. Why can't companies and people just put out cool things that have faults? Now we have a tool that got pulled, not because any concrete harm, only outrage over theoretical harm. It is not the tool, not the people finding faults, but people reaction's that seem like they have gone too far.
> In the company’s words, Galactica “can summarize academic papers, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more.”
The first step to successfully publish prototypes is creating realistic expectations. That's being done all the time in papers and other ML projects. Instead Meta listed a set of features in a language model that can be summarized as "magic".
GPT-3 can already most of those things. If you haven’t spoken with GPT yourself, you might think it’s overhyped. But it really is quite amazing. I’m saying this as a 15-year veteran machine learning engineer who is not impressed by much in ML. The “hype” part of what’s happening (the disconnect between reality and the goal) is that these models do make silly mistakes. But they also can actually perform the operations on the list above, and not in some “works 1% of the time” sense. This kind of AI is at a turning point, becoming deeply impressive and significantly expanding what code can do.
I think it’s overly generous to say that GPT-3 can “do” any of those things. It can generate stochastic nonsense that can be shoehorned into the approximate shape of those things.
I wrote the software that uses GPT-3 to produce those. You can look at the papers to judge the accuracy for yourself.
2. Solve math problems:
What is the area of a circle with radius 2?
The area of a circle with radius 2 is 4π.
3. Generate wiki articles:
Write the first paragraph of a wiki article about tigers.
Tigers are the largest members of the cat family. They are apex predators, primarily preying on ungulates such as deer and bovids. They are territorial and generally solitary but social animals, often requiring large contiguous areas of habitat that support their prey requirements. This, coupled with the fact that they are indigenous to some of the more densely populated places on Earth, has caused significant conflicts with humans.
4. Write scientific code:
Write a matlab function to plot the largest three singular values of a matrix.
function plotLargestSingularValues(A)
s = svd(A);
plot(s(1:3));
I'm not trying the others because I think they're more specific to Galactica.
The models, including GPT-3, absolutely do make mistakes. We are in the first few years of these capabilities existing at this level, so much research is needed before the results are consistently good. In my experience, though, GPT-3 is extremely useful and reliable for many use cases. (Building and using tools like this is part of my career.)
>Why can't companies and people just put out cool things that have faults?
Absolutely they can, and his employer could have kept it up. The issue is the phantasmagorical and ridiculous claims about AI-generated scientific research that LeCun peddles. When there's something concrete one can use to test these extremely bold claims, there's a way to at least partially apply a reality check to the claims, and demonstrate their ridiculousness. Which is a very useful and important part of how the scientific field evolves and advances. Feeding non-experts all these wild claims in perpetual future tense only works for so long, and it ought to be that way.
If you put something online, and present it as useful tool, then you have to expect that people are going to try to break it. You can look at that as free testing and open-source bughunting, or you can complain about about misuse and take it offline. The responsible parties took the latter route, which is kind of silly.
What this project created was something sophisticated and powerful, but not something people wanted, and they got (rightfully) pilloried for it. Instead of shaking ones fist at the world for rejecting your brilliance, maybe the really smart ones are making the things that others actually desire, and not merely developing techs that give themselves leverage over others and expecting the world to defer to this demonstration of intellectual prowess.
This whole incident was a case study for product management and startup school 101. I've made this exact same category of error in developing products, where I said, "hey, look at this thing I built that may mean you don't have to do what you do anymore!" and then was surprised when people picked it apart for "dumb" reasons that ignored the elegance of having automated some problem away.
If this model were really good, they would have used it to advance a bunch of new ideas in different disciplines before exposing it to the internet. Reality is, working at Meta/Facebook means they are too disconnected from the world they have influenced so heavily to be able to interpret real desire from people who live in it anymore. When you are making products to respond to data and no actual physical customer muse, you're pushing on a rope. I'd suggest the company has reached a stage of being post-product, where all that is left are "solutions," to the institutional customers who want some kind of leverage over their userbase, but no true source of human desire.
> LeCun also approvingly links to someone else who writes, in response to AI critic Gary Marcus
The article really fails to explain that LeCun and Marcus have been trading insults for the last few years, it's hardly LeCun snapping at some random person.
I'm not super familiar with the state of the art technology in this space and how these demos were presented, but I think all of these conflicts seem like they should be resolved if companies just put gigantic honking disclaimers on the work these AI tools produce.
If you wrote a flashing big red warning, something like the following, couldn't everybody be satisfied? "CAUTION. This technology is still very early and may produce completely incorrect or even dangerous results. Any output by this tool should be considered false and is only suitable for entertainment purposes until expert human judgement verifies the results."
They did have a disclaimer in every page that AI models hallucinate. But Lecun has been caught in the general 'facebook bad' sentiment and some human broken clocks who are constantly bashing anything DL-related.
this is just a bunch of personal vendettas imho. The model was useful
I know what LeCun originally said on Twitter and what the copy on the Galactica web page said. But to repeat my question, what is the model useful for? How can it be used to achieve something that bears any resemblance to the original claims?
Afaik the claim was not that "it writes correct papers", but that it compiles knowledge. and to that extent it was accurate, along with the disclaimer that it hallucinates things, because it goes on tangents that are often unrelted
The paper was useful for that reason: as a search engine for underresearched and speculative connections within fields, and from what i saw it does well in that. For example, when brainwashing ideas or looking for other research directions to review. In fact, we already have Perspectives papers being published in journals which are speculative in nature.
I used it to help produce LaTeX code representing chemical structures, which normally would've taken me somewhere around 10 minutes but took me 30s instead.
The main criticism I saw was that that seeing disclaimer required clicking a link. I saw screenshots of the homepage implying it didn't hallucinate, and article pages without disclaimers. Were they doctored?
If they manage to create a differentiable information retrieval system (i.e. a database that informs neural nets about facts) then the hallucination part might go away.
I'll put it quite simply. If you promote an ML system that is ostensibly a scientific content generator, the expectation is that it will regurgitate facts, not lies. If you can't get that right, don't release it, except to publish a "negative results" paper.
That is just false. The model had a disclaimer on every page that it is not to be trusted and that it hallucinates. Strawmanning is not a nice academic retort
I'm saying that if your model needs something like that (I certainly didn't see the disclaimer) then it's not appropriate for science. THe expectation of accuracy is much higher.
I disagree. How can you say the expectation of accuracy is much higher when the creators literally tell you to not expect accuracy? That's a problem with your expectations, not the programs problem.
Imagine a program that had some kind of concept of "interesting and novel mathematical proofs", and it could spit them out. But, 99.9% of the proofs were actually logically inconsistent or true but extremely uninteresting. Would that still be an interesting exploratory tool? I think so.
> WARNING: Outputs may be unreliable! Language Models are prone to hallucinate text. Trained on data up to July 2022
The model wasn't supposed to make science for you and any scientist who used it like that should probably not be a scientist. It's a language model, i would think people know what that means by now
Yes, it's clearly part of my point that language models are insufficient to produce high quality professional text.
Facebook's intent with the model was quite clear; the abstract of the paper says this:
"In this paper we introduce Galactica:
a large language model that can store, combine and reason about scientific knowledge... these results demonstrate the potential for language models as a new interface for science. We open source the model for the benefit of the scientific community."
I didn't expect it to make science for me- that would be quite impressive (it's been the guiding principle for my 30+ years in science)- but I do expect the demo to be less bad.
I honestly don't understand what they were thinking. Transformer-style language models are clearly unsuitable for this task. Starting with a foundation that is known to "hallucinate"[1] in this context strikes me as just crazy. There is no way to start with a hallucinating foundation and then try to purge the hallucinations at some higher level; characterizing the "shape" of "the output of this model - the errors this model makes" is a super complicated N-dimensional space no current AI model has any hope of characterizing. I'm pretty sure that space is straight-up super human.
If such an AI is possible, and I won't promise it isn't, it will certainly not be based on a transformer model. I also won't promise it won't "incorporate" such a thing, but it can not be the base.
[1]: Not sure I love this word, I think I'd prefer "confabulate", but hey, I'll run with it.
I though tthat was quite funny and probably better than what most people would write if they wanted to write a parody about space bears
It's only bad if you expected it to provide scientific answers, which was not the claim. IIRC the page said something about it compiling knowledge. which it does in this article
I think you know that the answer is in the links of that link you posted.
EDIT: I mean that it compiles the story of Laika and Karelian bear dogs
In any case , i understand you may have had a falling out with Lecun before, but that is no reason why this research model should not be online for peopel to test it. Let's try to improve things rather than blocking and banning things
Huh? What answer is in the links of the link I posted? If you mean "the russians sent a bear dog into space", that doesn't explain all the detail the model generated.
The statement about bears isn't just factually wrong, it generated specific details that make it appear right! At first, I was going to say that it wasn't half-wrong because tardigrades (known as water bears) have been sent to space.
This is simply a project that wasn't ready for the real world. A wiser R&D leader would have told the team standards were higher, rather than advising they put a disclaimer on it.
EDIT: since you didn't reply, but just edited (you're probabably at your reply limit depth): laika wasn't a bear dog. She was a mongrel found roaming moscow.
Sticking a bunch of disclaimers onto cyanide doesn't mean that it is useful to describe it as almond milk.
If they had done an adequate and accurate job explaining what, if any, plausible value or potential Galactica had, warnings would not have to have to carry so much weight. Or their PR could have focused on their tricks and optimizations, and not characterised Galactica at all.
Instead they'll be known as the people who described a science flavored word salad generator as if it was a tool useful for "summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more"
If they had, for example, positioned this as a tool for creating unscientific, but academic sounding anti-vax propaganda, people could have questioned their morals, but not the "fitness to purpose" of their tool.
They proved that Galactica would be an excellent front-end for a differentiable information retrieval system. This definitely moved the research forward; work on the back-end (i.e. fact retrieval usable for LLMs) is still needed.
To be honest projects that constantly and consistently generate smart falsehoods (the "bears in the sky" that the author of the article mentions) don't have any scientific need, so to speak. Or, if they do, I fail to see it.
They do. Try asking it about a field that is very novel or very very niche, and it will point out various links. Biology connections (which are often very vague and open) are good for that. I found that it generated some links between mechanisms that i found interesting.
Unfortunately the model is down so i cant review it.
But how can you possibly trust anything the model spits out as accurste and true if it's happy to spit out a whole thing on bears in space, something that plainly never happened.
Sure, it be spitting out these links between mechanisms, but it could have pulled them out of the same place it pulled an article about bears in space: the server's ass. And verification is going to take you far longer on something that actually seems realistic, which is a massive problem.
It's not supposed to spit out truth. Meta may have overhyped it but they were careful not to claim that it makes true claims, but that it compiles knowledge, and they had a disclaimer that it hallucinates fact. Anyone who has seen language models knows that if you ask it about space bears, it won't reply that there re no space bears, instead it will try to create a link between space and bears. It seems to me that people were deliberately using impossible inputs so that they can claim the model is dangerous for doign what a language model does. And their ultimate purpose was to defame the model and take it down. (And BTW we 've heard those "dangerousness" BS claims before)
The usefulness of the model was in under-researched fields and questions, where it can provide actually useful directions.
My whole point is said directions are completely useless if you can't trust the system to be based on factual principles.
It's easy to dismiss "bears in space" as not factual. It's harder to dismiss "the links between two underresearched fields in biology" without putting in an exorbitant amount of work. Work which likely is going to be useless if the model is just as happy to spit out "bears in space".
And that was what I had already said. I asked you how you could possibly trust those links it provided. Because you can't. They may very well be novel links that had never been researched. Or they could have been bears in space.
A scientific model which doesn't guarantee some degree of factuality in its responses is completely useless as a scientific model.
They may have had a disclaimer, but they also presented it as a tool to help one summarize research. It was clearly unreliable there. And who cares if it has some good results? How do you know whether it's spitting nonsense* or accurate information?
Yes, I do think there is value in engaging with the community for models that aren't perfect. But it needs more work before it can be framed as useful tool for scientific research.
* I mean subtle nonsense. Space bears are easy to distinguish..but what if the errors aren't as 'crazy'?
Right. my way of evaluting any system is to start with the easiest tasks. If the system doesn't get the easiest task right, I do not proceed to use its output for complex things.
>If you wrote a flashing big red warning, something like the following, couldn't everybody be satisfied? "CAUTION. This technology is still very early and may produce completely incorrect or even dangerous results. Any output by this tool should be considered false and is only suitable for entertainment purposes until expert human judgement verifies the results."
I'll give you the benefit of the doubt since the demo is now offline but there was in fact a giant disclaimer that said more or less: "DO NOT TRUST THE OUTPUT OF A LANGUAGE MODEL WITHOUT VERIFICATION."
It did...but it also said stuff like "ACCESS ALL OF HUMANITY'S KNOWLEDGE!!!"
They were trying to have it both ways, where the headline giveth and the fine print taketh away.
In a better world, this could have been released as "We trained a language model on the scientific literature. We're excited because it is starting to draw conclusions, but there is obviously a lot more to be done. See our evaluation here. Have fun and let us know what you find."
The hype is really annoying and seems bizarre--everyone who even remotely cares knows about Meta already.
That was only present on the bottom half of the "Mission" page - the main page and other pages were full of examples of all the marvelous things it can do (that it can't really do reliably) [0].
I think one of the problems with large parts of academia is that the system thrives by finding flaws within things other people have created.
Criticism is certainly helpful and necessary for advancing the state of the art, but without something to balance it, it turns into a pretty bleak place to work. I guess this is one example that feels more like a personal vendetta than a constructive criticism of the work.
Please correct me if I am misrepresenting a chain of events here.
A.. tool lands that allows one use language model to generate content. People feed it false data and share that, surprise, data it produces from the data the model is based on is false. How is this a surprise? I am still not sure why Meta would pull it? It can still be useful, but it was made not useful. I am not sure what a proper metaphor is for it, but it is almost like I give you a tool ( lets say a knife ) and you complain that the tool produces bad results when drinking soup.
What am I missing here?
<<or maybe it [Galactica] was removed because people like you [Marcus] abused the model and misrepresented it. Thanks for getting a useful and interesting public demo removed, this is why we can’t have nice things.
<<Meta’s misstep—and its hubris—show once again that Big Tech has a blind spot about the severe limitations of large language models. There is a large body of research that highlights the flaws of this technology, including its tendencies to reproduce prejudice and assert falsehoods as facts.
Yann LeCun has lots of faults, certainly with how he treats AI safety in general, but a lot of the criticism he got was saying "He's not qualified to be in the position he is in" which is actually absurd if you know anything about him. Even if you knew nothing other than the fact that he won a Turing Award then he would be qualified for basically any computing/AI/ML job on the planet. \
Also the title of this post is deliberately inflammatory. Should be more like "Head of team that spent months building complex ML system annoyed when people spend undue amounts of time criticizing it."
>"Head of team that spent months building complex ML system annoyed when people spend undue amounts of time criticizing it."
What's undue about it? If a team spent months producing a webapp that failed to live up to its claims and was full of security vulnerabilities and inaccuracies, nobody'd expect anyone to moderate their criticism of it. Having a Turing award doesn't mean somebody can't deliver a shit product, and I don't see why it should make them more above criticism than anyone else who delivered something similar.
I don't think anyone is really "qualified" to have the level of influence, power, and adoration that the very top names in many fields get.
Research "success" certainly takes some talent and LeCun is certainly smart. However, exhibiting that talent also needs luck, timing, and connections, all of which are subject to crazy positive feedback loops: getting an award helps you get subsequent awards, better students, etc. In a world where GPGPUs came a little sooner or later, I think we'd have totally different "superstars".
I think there approach and model are interesting the problem is that they overhyped it to the point that it would be unacceptable academically.
Their abstract says "In this paper we introduce Galactica: a large language model that can store, combine and reason about scientific knowledge... these results demonstrate the potential for language models as a new interface for science. We open source the model for the benefit of the scientific community."
If I was a reviewer of this paper I would ask them to add (if they haven't so) significant section to the body of the paper highlighting the limitation of the model and the ways it can be misused. Including showing examples of wrong output.
I would then ask them to rewrite the abstract to include something along the lines "We also highlight the limitations of the model including inability to distinguish fact from fiction in several instances and the ways it can be misused and outline some ideas on how these limitation could be mitigated or overcome in the future."
A fundamental problem with Galactica is that it is not able to distinguish truth from falsehood, a basic requirement for a language model designed to generate scientific text
Isn't this the same problem that Github Copilot has?
Fundamentally it has no idea whether code works. It doesn't even know what the problem is.
It just spits out things that are similar to things its seen before, including buggy code from Github repositories.
Not sure why it's so popular. I guess it helps you write status quo code faster (the status quo being buggy and slow) -- I would rather it help us write better code.
I work with a system similar to Copilot and find it helpful. It memorizes APIs and code patterns so that I don't have to. I don't expect it to solve algorithmic problems for me. It's a better version of autocomplete, which is a feature that programmers have been using in their IDEs since the 90s.
> and generated wiki articles about the history of bears in space as readily as ones about protein complexes and the speed of light. It’s easy to spot fiction when it involves space bears, but harder with a subject users may not know much about.
Well, that matches our current experience with human-written Wikipedia articles pretty closely then.
The title of this post should tell you everything you need to know about its bias against LeCun, but both sides are in the wrong here. Meta shouldn't be over-hyping tools that have serious problems and the anti AI bunch need to stop beating a dead horse by only judging AI's progress soley by how it can be abused.
This is a good title because it succinctly captures the issue: LeCun hyped this work by making wildly inaccurate claims and cherry picking model outputs. Go read his original tweets about the model's capabilities. Read Facebook's own characterization of what this model could achieve.
Not only did they exaggerate and hype, but they also didn't even try to solve some of the most glaring issues. The efforts on toxicity mentioned in their paper aren't even mid. They barely put effort into measuring the issue, and definitely didn't make any attempt to mitigate or correct.
Toxicity isn't really the point. Here's the point. If you can't prevent a model from being overtly toxic, then why should I believe you can give any guarantee at all about the model's output? I shouldn't, because you can't.
Galactica is just another a language model. It can be a useful tool. Facebook and LeCun oversold its capabilities and downplayed its issues. If they had just been honest and humble, things would've probably gone very differently.
In some sense, this is good news. The deep learning community -- and generative model work in particular -- is getting a much-needed helping of humble pie.
Hopefully we can continue publishing and hosting models without succumbing to moral panic. But the first step toward that goal is for scientists to be honest about the capabilities and limitations of their models.
----
My account is new so I am rate limited and unable to reply to replies. My response to the general vibes of replies is therefore added to the above post as an edit. Sorry.
Response about toxicitiy:
It's a proxy that they say they care about. I can stop there, but I'll also point out: it's not just "being nice", it's also stuff like overt defense of genocide, instructions for making bombs, etc. These are lines that no company wants their model to cross, and reasonably so. If you can't even protect Meta enough to keep the model online for more than a day or two, then why should I believe you can give any guarantee at all about the model's output in my use case? (And, again, they can't. It's a huge problem with LLLMs)
Response about taking the model down:
I'm not at FB/Meta, but I think I know what happened here.
In the best case, Meta was spending a lot of valuable zero-sum resources (top of the line GPUs) hosting the model. In the worst case they were setting a small fortune on fire at a cloud provider. Even at the largest companies with the most compute, there is internal competition and rationing for the types of GPUs you would need to host a Galactica-sized model. Especially in prototype phase.
An executive decided they would rather pull the plug on model hosting than spend zero-sum resources on a public relations snafu with no clear path to revenue. It was a business decision. The criticism of Galactica and especially the messaging around it was totally fair. The business decision was rational. Welcome to private sector R&D; it works a little different from your academic lab for better and for worse.
So, instead of people attacking him for absurdly hyping the model, they attacked the model as being DANGEROUS in order to remove it to .... spite whom exactly? Typical academic catfight
Also, let's not pretend that every academic and institution is not overhyping their work. If you read a bunch of academic press releases you 'd think we are on the verge of curing cancer and fusion any day now.
Blatantly false claims and "hype" need to be addressed immediately and strongly when they're being pushed into places that actually matter, like city streets and medical science.
While I understand you're using "toxicity" here as a proxy, the implication is that if it were nicer you would have more easily believed it's output.
Is that really where we want things to be? Because I strongly suspect it's a lot easier for them to make it nicer than it is for them to make the output good.
Just a reminder, those people on twitter screaming up a storm didn't take the model down, Facebook did. If Facebook didn't want the model to come down, there's nothing the twitter mob can do about that. Twitter isn't a democracy, a million screaming voices have no power.
I came here just to ask: Could someone rewrite the HN title of this post? Currently it feels really clickbait-y.
One of the things I really like about HN is the _lack_ of clickbait titles. Some titles are more informative, some less, but overall I feel like the titles are clear, to the point, and not carefully crafted/engineered to poke the lizard part of my brain in the way that clickbait titles are.
Disclaimer: I haven't read the article so I can't propose a title myself. And with a title like this I'm not going to.
I have to agree. This is not my bailiwick, and, for all I know, the title is accurate, but it is jarring. I feel as if this is a professional venue, so I try to behave professionally (even though I'm not really looking for any work).
Gary Marcus built almost entirely his public reputation (which is positively correlated with his income) by antagonizing whatever Deep Learning scientist he could reach. He speaks badly about people that worked hard with their hands, brains and souls to make incredibly complex things happen.
Yann Lecun, which I personally met a couple of times, is in a way another sort of typical character: the ever-childish researcher that likes money a lot, to the point of accepting a prestigious role in one of the most deplorable companies in the modern world (at least from an ethical perspective). He also like attention and public display of status: he can’t resist to pick a fight with Gary. From a pure research perspective he’s long dead.
The question is: do we have enough of those two? Can we move on? Thanks.
Here's what I (a person with a fairly superficial understanding of how AI works, for context) can't understand - why wouldn't it have been trivial to prevent the kinds of issues that arose by just training the thing to only pull information from scientific papers/literature, and when a question arose on a topic that didn't have information in those places just say "we can't find enough information about that topic"?
It seems like it was good at structuring the writing, both at the article and sentence levels, and for valid prompts it produced accurate responses. But if you entered "write a wiki article about the Alien vs. Predator hypothesis," it would structure it like a wiki article about a scientific topic but just put random AVP stuff it found on the internet into that structure. Why couldn't they just explicitly define which sources are appropriate to pull information from? That seems easier to me than building the actual product they made (but again, I am a layman here).
Because training in this manner requires good data. And by good I mean manually curated.
The ML hype train relies on "garbage in, good out", popularized by an influential paper published at the turn of the century [1]. Anyone with a modicum of experience in experimental science knows that any experiment is only as successful as its ability to collect good data that is representative of the problem, has manageable noise based on known controlled sources and has sufficient coverage of the problem domain to provide meaningful analysis. But of course, if ML admitted that this was a necessary requirement, it would become yet another optimization technique, admittedly new classes of optimization techniques and that promise that the machine learns something new would fall flat on its face.
Sure, I get that part, but in this case it seems like curation (at least to a degree that would avoid the problems that arose) is fairly trivial - just train it on high-quality scientific journals/textbooks/etc., instead of letting it peruse the whole internet (or whatever they did) for scientific data.
LeCun's behavior here cannot be justified, but I think it's partially explained by his background. LeCun's life experience has taught him that when others criticize his work again and again and again, he should ignore them and persist if he thinks he's right:
During much of the 1990's and the 2000's he, along with Geoff Hinton and Joshua Bengio, ignored negative criticism by many naysayers as the three of them persisted on researching deep neural networks, which a majority of AI researchers had dismissed as a dead-end. It wasn't until the late 2000's, when Hinton showed he could train restricted Boltzmann machines efficiently, that other AI researchers started paying closer attention. And of course everyone else piled on after 2012, when a deep neural network (AlexNet) won ImageNet by a wide margin over all other methods.
The difference is that since Lecun made his contribution to MNIST, an entire field of endeavor with many experts as smart as him sprung into reality, and those people are making good criticism. You'd expect him to grow and mature (especially as the leader of an AI org at a major company) and learn to recognize that sometimes, he's really overhyping the technology more than necessary.
I certainly got the same negative response when I worked in ML in the 90s- "computers aren't fast enough, we don't have enough data, and we don't have the algorithms" and to be honest, I didn't really have the capability to disprove the people saying that. So I appreciate that he persisted and was successful.
Expecting Galactica to produce truthful academic papers is just about as sensible as expecting to find a significant other among your kitchen appliances. Language models are a way to emulate the writing style of human produced content, only a fool would expect them to reason about the text to any level resembling a scientific standard.
Written language is a doorway to the full extent of human cognition; unless the problem domain is severely constrained (ie "What is the distance to Mars?"), you are very likely to fall into reflexive traps that rapidly devolve into AGI ("I think, therefore I am?").
The issue isn't that the model isn't truthful, it's that it is effective at writing language that appears factual and looks truthful to the untrained eye. Sure, it is going to give you what you're asking for, but the issue come when you take that and give it without warnings as to its origins to people who can't be expected to fact-check a scientific article.
Flamewar attacks will get you banned, regardless of how wrong someone else is or you feel they are. We've had to ask you this before—e.g. https://news.ycombinator.com/item?id=31086037.
It seems clear that you and the lopkeny12ko simply understand the meaning of the word "editorialize" differently. That's definitely not something to get into a spat about.
It's not ok to post flamewar comments like this, regardless of how wrong someone else is or you feel they are. If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it. Your comment would have been fine without the insults in the first two sentences.
It seems clear that you and Tomte simply understand the meaning of the word "editorialize" differently. That's definitely not something to get into a spat about.
From the title I knew it was about LeCun. Why does it seem like Meta’s culture is riven by sycophants who are unable or unwilling to make work better through criticism?
Look, if you don't like the work because the model is too big and expensive to run or because it is stochastic and can spit out garbage sometimes - that's valid criticism and something that a scientist can respond to. If you're going to level ideological accusations about biases and harms and the gender and race of the authors, then only people who share your ideology are going to be able to respond, and very few do.
It’s a misplacement of priorities and misjudgment of character by ACM that they awarded the Turing prize to LeCun, yet haven’t bestowed it on Spaf.
Deep Learning was promulgated by several computer scientists, and its still early days. Information security in academia? It’s Spaf. Perhaps it could be argued that his contributions don’t have the depth and rigor as LeCun’s research, but they’re broader, more sustained, and with patient good humor.
My impression was that it merely searched through the large collection of articles and then briefly summarized them? Maybe the bears in space output is a result of articles about water bears, i.e. tardigrade articles?
Hope we all recognize that no matter how many parameters the current generation of ML models have, they are fundamentally a bunch of dump memories. Nothing new.
I don't know LeCun personally, but there's a lot of backstory here that this polemical clickbait is leaving out.
- LeCun has a history of getting mobbed by "AI ethics" types on Twitter, and in the past he was very deferential to these folks, and even left Twitter for a while. I wrote about some of that here: https://www.jonstokes.com/p/googles-colosseum
- The MIT Tech Review, which is the author's main source here apart from Twitter, is techlash rag, and they went through a long phase where they only published anti-AI stuff from the "AI ethics" people. Most of those writers I used to follow there on this topic have since moved on to other pubs, and the EIC responsible for this mess has moved on to run WIRED. But it seems they're still publishing the same kind of stuff even with new staff and management. They have exactly one and only one editorial line on AI in general and LeCun in specific, and that is "lol AI so racist and overhyped!" It's boring and predictable.
- LeCun has a longstanding beef with Marcus, and the two treat each other pretty poorly in public. Marcus seems to have a personal axe to grind with LeCun. Given that Marcus has been leading the mob on this, it's not shocking that LeCun got crappy with him.
- Emily Bender, Grady Booch, and the other folks cited in the MIT Tech Review piece all, to a person, have exactly one line on AI, everywhere at all times and in all circumstances, and it's the same one I mentioned above. You could code a bot with a lookup table to write their tweets about literally anything AI-related.
- Yeah, LeCun is a prickly nerd who gets his back up when certain people with a history of attacking him come after him yet again. He should probably should stay chill.
- "AI so overhyped" is a pose, not an argument, an investment thesis, or a career plan. But hey, you do you.
Anyway, I hate to be defending anything Meta-related, but this article is slanted trash, its sources haters who have only one, incredibly repetitive thing to say about AI, and the author is a hater.
Thanks for posting this, it gives good added context that helped me change my original opinion.
I was quite familiar with Lecun's dust up with Timnit Gebru on Twitter, and I had a lot of sympathy for him in that situation.
I think it's quite sad that so much bad-faith argument has infiltrated academia to the extent that it has. Some may say it's always been that way, but it feels worse to me now. One of my "heroes" of unbiased rationality, Zeynep Tufekci, wrote a really good Twitter thread recently about how some of these flat out liars in academia manage to continue their lies unscathed with little pushback: https://twitter.com/zeynep/status/1592210111359250432
Yes, "AI has too much control over our lives, and it's biased" is a repetitive claim made over and over, but it's also, you know, a biased AI ruining lives right now, and we should repeat it until things stop.
Every week there's a random here post about "some AI detection system closed my Gmail account / took down my Android app / froze my Square funds", and Hacker News is seen as the semi-official tech support line for companies who have turned to biased AI to cut costs.
A lot of what AI ethicists are saying is that "if we hook these AI systems up to safety-critical systems, anyone who doesn't fit the model is going to be labeled an outsider", and I don't know why we shouldn't repeat it as many times as it takes to get people to listen... accounts are still being banned, lives are still being ruined.
To counter this with "think about what progress AI has been making!" is missing the point. "Sure, it Markov-chain'd some random facts about space bears and cited random people with papers it made up who are now caught in the cross-fire of machine hallucination, but think about the progress! It could format its fiction to look like a TeX paper and add some random squiggles that look like math expressions!" is not the slam-dunk defense you think it is.
> Every week there's a random here post about "some AI detection system closed my Gmail account / took down my Android app / froze my Square funds", and Hacker News is seen as the semi-official tech support line for companies who have turned to biased AI to cut costs.
I would agree with this if I ever saw these self-appointed AI-ethicists focus on these kinds of harms. But, at least in my experience, is usually focused on the exact same set of concerns that 90% of the time has "intersectionality" somewhere in the criticism.
Yes, I'm being a bit unfair and snarky, but I'd be more willing to pay more attention to some of these criticisms if I felt it included more of the harms you bring up than just what I feel has become a constant bone to pick. I agree with the GP when he wrote "You could code a bot with a lookup table to write their tweets about literally anything AI-related."
>But, at least in my experience, is usually focused on the exact same set of concerns that 90% of the time has "intersectionality" somewhere in the criticism.
You have the choice to avoid google accounts and limit the destruction a google AI system can do to you.
You don't have a choice to not be born black, and not be put in jail for longer just because you are black.
Why don't you care that millions of people will be hurt by these things, and care more that an app developer gets locked out of the app store? Apple hasn't put anyone in jail.
Yes, I used a nerdy example because I figured it would appeal more closely to the computer dork crowd of Hacker News, hoping that, by metaphor and extrapolation, you could imagine all sorts of ways that AI biased against sex or race would be immensely damaging to the fabric of society, and bias gets built into our society. This is already happening, as biased AI is used to estimate how much jail time someone will get [0]. Or pushing rents higher [1]. Or treating people for healthcare [2].
These AI ethicists are complaining about all of this, but of course they yell more loudly about sexism and racism, because, you know, those are fairly serious things that should be addressed first???
I don't think they need to be original, I think they need to bang their drum loudly. "Oh, that women's suffrage movement won't shut up about how they don't have a voice in policy that governs their life, can't they talk about something else for once" isn't an indictment of the people complaining, it's an indictment of the people not listening.
> this article is slanted trash, its sources haters who have only one, incredibly repetitive thing to say about AI, and the author is a hater.
It doesn't "source haters". It quotes MIT Tech Review and Gary Marcus precisely in order to provide context for the subject of the “article” (blog post): LeCun's bizarre "this is why we can't have nice things" statement. This petulant remark seeks to shut down negative feedback as a class, regardless of its merits. That's what the blog post is about. The quotes are there so that the quote from LeCun makes sense, not because they're legitimate criticism.
> Emily Bender, Grady Booch...
These people are not mentioned in the linked post, which tells me you're pattern matching on "techlash" and posting a bunch of only vaguely related context (and a medium self-link).
> He should probably should stay chill.
Exactly?! That's the point of the blog post. It's in the title of the article. It seems obviously true, and I'm not sure how any of what you say adds up to a robust conclusion that "the article is slanted trash" and "the author (Andrew Gelman?!?!) is a hater" other than you don't like some of the quotes.
> and in the past he was very deferential to these folks, and even left Twitter for a while
How many times has he quit Twitter now? Three IIRC. Seems like he needs some coaching on the following through with promises.
Yes I know. They are irrelevant to the point of Gelman’s blog post, which is why he doesn’t mention them. I assume they’re being brought up here out of some kind of guilt by association thing, or to make the self-link seem more on topic.
Yes. To provide the context necessary to understand the "this is why we can't have nice things" quote, you have to say what "this" is. That's how pronouns work.
Thanks for taking the time to your thoughts in such clear and lengthy terms. This article seems more about drama than scientific merit.
> Second, what’s the endgame here? What’s LeCun’s ideal?
This section I think is particularly exemplary - regardless of your take here, I think it’s pretty ridiculous and uncharitable to interpret “criticism of science is immoral” from LeCun’s quoted statements.
> "AI so overhyped" is a pose, not an argument, an investment thesis, or a career plan. But hey, you do you.
> Anyway, I hate to be defending anything Meta-related, but this article is slanted trash, its sources haters who have only one, incredibly repetitive thing to say about AI, and the author is a hater.
I was hoping someone else would also find the actual context and background for what is clearly a poorly written hate piece.
I have very little interest in Meta, but the amount of FAANG hate that the media knowingly perpetuates and the amount of criticism that anything launched receives, would absolutely wear someone down. Lecun is no saint, but the article is very unfairly written.
Wrong. "Here is a totally mind-blowing new thing AI will let you do this week that humanity could not do last week" is an endless source of novelty and eyeballs. Source: Am a publisher in this space and have seen the engagement numbers. You are the guy in the meme standing in the corner of the party, with the words "They don't know AI is just math" printed over his head.
Are these a new type of luddite? Why aren't there "computer ethicists" complaining about real issues with the use of technology in general? I'm being tracked at all times without my consent. These "AI ethicists" are happy to use platforms like Twitter that track you, and even reward you for giving them more info (e.g. phone number).
If I were a lot smarter and had a lot more time, I'd love to paraphrase the article using, instead of AI, a legacy technology that at the time struck many as useless, but after development and acceptance became indispensable. Maybe the internet itself?
The person you’re responding to dodged (with a very fair argument), but I’ll bite the bullet here: not really, and I’m curious to hear what you think the damage could be.
I mean SOME criticism always has a basis, but that seemed to be a large part of the reason they published this technical demo: to get feedback and spark scientific discussion on the state of the art. They did publish it with prominent warnings to not trust the output as necessarily true, after all.
If the worry isn’t with primary users but with people using it to intentionally generate propaganda/falsehoods for others to consume… idk it seems like we’ve long passed that point with GPT-3.
So their goal was to gather feedback (read: criticism) but took it down after 3 days? In lieu of some sort of coercion (which, idk how you’d coerce Meta), it seems like they weren’t all that interested in feedback and discussion.
The fact that people responded negatively to a bad model (where “bad” can vary from unethical to dangerous to useless depending your vantage) has little to do with the anti-AI cottage industry.
Portraying criticisms as necessarily stemming from bad faith actors is exactly the opposite of fostering feedback and improvement.
Not sure how you made the leap from "There is cottage anti-AI industry from people who couldn't do real AI and hoping the next best thing to do is label it as 'racist'" to "You can't see any basis to criticize a "scientific tool" released into the wild that spits out convincing falsehoods?"
Well I'm pretty sure we're on a thread about an AI tool that was released as a scientific tool and it spits out convincing falsehoods. So maybe the cottage industry comment was truly just a non-sequitur, or maybe it was in reference to the topic of this entire HN post?
I'll give you another option: There are valid concerns with the type of output that large model AI generates, and there are experts working in the field who are trying to improve the state of the art by researching and implementing solutions to these valid concerns. There are also a subset of academics whose "one trick pony" is just "veto, veto, veto", without providing valid solutions, or worse, not taking a good faith understanding of "yes, this may not be perfect yet, but that doesn't mean we have to shut the whole thing down."
I'm not as familiar with how this culture works in the AI field, but I absolutely have seen it in the world of open source: people who have little to no programming skill who do nothing but grep repos for instances of "whitelist" and "blacklist" and pretend they are doing God's greatest work by changing these terms, and then cause typical faux-outrage storms on Twitter when their PRs are met with eyerolls.
Like GP I was replying to, it sounds like you’re mostly looking to air grievances here rather than discuss the topic at hand. Thank you for your work on OSS in any case, I imagine that’s a very frustrating experience.
The criticism that Galactica is bad is legit. The criticism that it's dangerous by the Twitter police is not. Some people are professional complainers and somehow got to high ranks in academia. That says a lot about academia in the US.
Why Meta’s latest large language model survived only three days online - https://news.ycombinator.com/item?id=33670124 - Nov 2022 (119 comments)