The trouble is that people have been sold this idea that ML/AI can do amazing things, without properly being told that really the things it can do are quite narrowly-scoped. They've been sold the Star Trek computer idea.
For example, years ago I was working on a prototype/proof-of-concept thing for instrumenting industrial machinery with stick-on small computers. Simple stuff - attach accelerators, temperature, humidity etc sensors to existing machines and collect the data and send it back.
The management thought we'd be able to apply machine learning on the data to get "business insights" from the all-powerful machine. They didn't know what these insights might be, just that ML/AI would generate them and therefore make the business a fuck-ton of money because AI generated novel new "business insights" that no one had thought of before and so transform the business. They thought it was just a magic box that would generate unbounded magic answers for their needs by passing in just some temperature and humidity readings or whatever, and then it would tell them they need to make more brown bread and less bagels in the North East region etc.
In reality, as I understand it, currently ML/AI requires us to know what the possible answers can be before we even begin training the network. So the classic example is it needs to know that the possible MNIST digits are 0-9, or that you are looking for one of 100 image classes etc.
You cant train a network with the MNIST digits, and then have that network tell you what shares to buy or sell.
Sure you can lop off the final layer and repurpose some of the middle layers, but you still need to train it to classify the inputs into categories you define up front. It won't give you a novel answer that you have not trained it for.
... at least that is how I understand it. Things may have changed over the past 5 years or so.
That said, I do agree there have been some cool things lately like machine vision etc. I don't think it will be that huge an industry though - it feels like a lot of it is largely just commoditised now (which is good) and it will be just like any other library you pick - like picking a UI framework for a web app. Just pick up a pre-trained network from modelzoo and get on with your real business requirements for 99% of people using ML, while the other 1% (at FAANGs et al) and academia churn out new models.
That's actually something ML is incredibly useful at, when it comes to machines with sensors - failure prediction / anomaly detection, etc.
In the industry, (preventive) maintenance takes up a pretty huge chunk of resources. It's something techs need to do often, and it's often a laborious task, but it's obviously done to reduce downtime.
So the business insight, as they like to call it, is to reduce costs tied up to repairs and maintenance.
All critical applications have multiple levels of redundancy, so that a complete breakdown is very unlikely, but it's still a very expensive process if you're dealing with contractors. If you can get techs to swap out parts before the whole unit goes to sh!t, then that's often going to be a much cheaper alternative.
But, in the end, it comes down to the quality of data, and the models being built. A lot of industrial businesses hire ML / AI engineers for this task alone, but expect some magic black-box that will warn x days / hours / minutes ahead that a machine/part is about to break down, and it's time to get it fixed. And they unfortunately expect a near-perfect accuracy, because someone in sales assured them that this is the future, and the future is now.
Yep, you and the user you're replying to are both right in different ways. One thing's for sure - machines don't generate "insights" on their own.
Let's define an "insight" as "new meaningful knowledge", just for fun. We could talk about what comprises "new" and "meaningful" but it would be beside the point I'm making.
In a supervised learning problem, the range of possible outputs is already known, meaning the model output will never be categorically different from what was in the training data. The knowledge obtained is meaningful as long as the training labels are meaningful, but it can never be new.
Unsupervised learning doesn't have a notion of "training data" but that means an unsupervised model's output requires additional interpretation in order to be meaningful. It is possible to uncover new structures and identify anomalies in new ways, but this knowledge isn't meaningful until someone comes in and interprets it.
Applied to the specific example where sensor data is used to try to generate insights about machine functionality: Either you can only predict the types of failures you've already seen, or you can identify states you've never seen but you wouldn't know whether they mean the system is likely to fail soon or not.
It's the Roth/401(k) tradeoff. For model output to be useful, someone must pay an interpretation tax. The only choice is whether it is paid upon insight deposit or withdrawal.
> Either you can only predict the types of failures you've already seen, or you can identify states you've never seen but you wouldn't know whether they mean the system is likely to fail soon or not.
Yup, this is something I've seen from both sides. First you mention is basically the standard, while the last is part of the deep learning voodoo black magic that executives and sales love.
I've had people approach me with proposals like "What if we just churn [ALL OF] our data through this or that model, and let's see if it comes up with some patterns we've never seen or thought about"
And that's not just for industrial applications. It's everywhere.
What is concerning to me is that this mentality will surely induce more unrealistic expectations. Before you know it, business execs are starting to ask why we need business analysts at all, because surely those fancy deep neural networks can extract all kinds of features - "only need data scientists to figure out those things".
So yeah, that's my fear. That businesses will blindly start to discard domain knowledge, and just feed black-box models their data, and let the data scientists wrestle with the results.
Yeah this is where it would have helped if I had discussed what I meant by "new".
AlphaGo is a supervised learner that outputs optimal Go moves given opposing play. It yields new discoveries in the same sense that a model designed to predict mechanical failures from labeled sensor data would: I didn't know what the model was going to predict until it predicted it, and now I know.
But what the factory owners want is a machine that can take raw, unlabeled sensor data and predict mechanical failures from that. They want insights. "Why not just feed all our data into the model and just see what comes out?" they ask. "I don't see why we need to hire at all if we have this neural net."
The reason you need a human somewhere in the system if you want insights is because someone needed to program AlphaGo specifically to try to win at Go. At the factory, someone needs to tell the machine what a mechanical failure is, in terms of the data, before it can successfully predict them.
Then, neither "winning at Go" nor "mechanical failure" are states that the system hasn't already been programmed to recognize. That's what I mean when I say a supervised learner cannot generate "new" output.
If the business wanted to track the rate of failures and create predictive models about when things fail, or detect anomalous behaviour, that's what they would have set out with as the goal, and, perhaps, some ML model might have helped, but probably, it would've been too unreliable and any number of standard predictive models with well known characteristics would have been used instead.
That's not what they wanted.
What people are being sold is AI/ML as a magic bullet that will do something useful regardless of the situation,
and it lets business people avoid making decisions about what they actually want, because AI/ML can be anything,
so they just signup for it and expect to get 20 things they didn't know they wanted handed to them on a plate.
Turn out, it's not enough to just collect a bunch of data and wave your magic wand at it. It wasn't with web analytics 10 years ago, it's still not.
What you actually need is someone who has a bunch of tricks up their sleeve, and has done this before, and can suggest a bunch of Business Insights the business might need before they start building anything, people that actually decide what to do, and actions taken to investigate, and solve those problems.
I mean, to some degree you're right; perhaps ML models could be useful for tracking hardware failures, but that's not what the parent post is talking about. The previous post was talking about just collecting the data and expecting the predictive failure models to just jump out magically.
That doesn't happen; it needs a person to have the insight that the data could be used for such a thing, and that needs to happen before you go and randomly collect all the wrong frigging metrics.
...but hiring experts is expensive, and making decisions is hard. So ML/AI is sold like snake-oil to managers who want to avoid both of those things. :)
Projects rarely end up doing what was planned when they started. As long as ML is solving real problems in practice, upper management will keep treating it as magic fairy dust to sprinkle around aimlessly.
It's all about how you package things. ML connected to an audio sensor could predict failure modes that are dificult to detect otherwise. Now that might not be was was asked for, but a win is a win.
I think both your post and the one you are responding to are correct.
I’ve experienced what the OP was alluding to...namely, it helps tremendously to start out with an understanding of the problem you’re trying to solve, more so in supervised learning. It’s incredibly frustrating to ask managers what business problem they are trying to solve only to be met with “We don’t know, that’s what we want the software to tell us.”
On the other hand, if they say “we want to predict machine failures” or “reduce maintenance downtime” now we have a lens in which to view the data.
If AI could do the magic as those managers hoped, they would be out of a job
I work in this space, and you've called out the thing that drives me absolutely bonkers. No matter how accurately I can diagnose the state of the equipment, what everybody wants is to know how long until it explodes, so that they can take it down for maintenance ten minutes before that.
This is a good explanation of how things work and things haven't changed much in the past 5 years. What changes the most is the amount of data and computing power ("compute") that are used in training neural nets.
Architectures are also tweaked slightly although they continue to be largely based on the two architectures that started the current deep neural net boom, Long-Short Term Memory Networks (LSTMs) and Convolutional Neural Networks (CNNs), which are largely used for natural language processing and machine vision respectively.
Every once in a while there is a bit of excitement about results that come out of adding a new parameter which is then given a fancy name like "attention" and generates much breathless copy about how neural nets will soon become as intelligent as a dog/ your three year old/ your personal assitant etc. Usually though it's not clear whether these new parameters or architectural tweaks are really responsible for the success of the proposed techniques and when one looks at the field as a whole it becomes more and more apparent that the most successful work is backed by the most data and the most computing power.
To slightly er tweak your prediction, I think what'll happen is that, indeed, like you say, there will be libraries and APIs and so on (well, there already are) but all these will be controlled by large companies that can afford the ground work. So, I don't expect small outfits, research teams at universities or tiny startups, to make a big difference. Academia is fast losing the ability to produce work that beats the state-of-the-art anyway (that's from personal comms with other researches). So in terms of academia perhaps we should expect a shift away from neural nets, to something that's easier to do research on.
> but all these will be controlled by large companies that can afford the ground work. So, I don't expect small outfits, research teams at universities or tiny startups, to make a big difference. Academia is fast losing the ability to produce work that beats the state-of-the-art anyway (that's from personal comms with other researches).
That makes me particularly sad. In a utilitarian sense, the ideal situation would involve new approaches being democratised. Instead, we're ending up with walled gardens.
The current situation isn't dissimilar to what has happened to the Web Platform during past 5-10 years. And, similar predictions are made wrt VR and AR (both seem to attract different types of investors).
The best way to think of today's AI capability is: automation.
You train it with a lots of examples of "given this input, this is the output i want", and hopefully it learns to get the "correct" (similar input => similar output) output for new inputs that you feed it. i.e. you've now automated the process of figuring the correct output for a given input.
There is also the "reinforcement learning" AI paradigm where the trained AI is choosing actions (from a given repertoire) in order to maximize an action outcome score based on some scoring function you provided. This is appropriate in a situation where you want the "AI" to do something more than just select the correct output, but again no magic - you're having to anticipate and score the potential outcomes.
In my opinion this type of tasks is the typical AI tasks that get most exposed to the general public. The AI/ML model is set to attempt a human task. The aspiration is to get as good as a human (or faster/better more precision etc). Classic statistical inference/ models usually don't perform very well in these scenarios(or do they?).
The typical AI methods train a model that utilises features that don't make much sense to humans (pixels/texture/word tokens etc). In contrast with traditional statistical modelling where each feature were given 'meaning' and their importance explored via investigating the observations (data) using properties of well studied mathematical models, the AI methods that often made the press don't put a lot of emphasis on these properties. The advancement is a push to utilise all available information/data to outstanding 'performance'. The explanation of the inner features within that math space is usually secondary (though I don't mean that authors of novel methods don't care about mathematical modelling).
I might be too naive here but that's how I feel after trying out many methodologies in my field.
The "AI" capabilities that are making headlines nowadays are mostly based on deep (multi-layer) neural networks with millions, or billions, or parameters. These nets do self-organize into a hierarchy of self-defined feature detectors followed by classifiers, but as you suggest for the most part they are best regarded as black boxes. You can do sensitivity analysis to determine what specific interval values are reacting to, and maybe glean some understanding that way, but by-nature the inner workings are not intended/expected to be meaningful - they are just a byproduct of the brute force output-error minimization process by which these nets are trained.
Neural nets are mostly dominant in perceptual domains such as image or speech recognition, where the raw inputs represent a uniform sampling of data values (pixels, audio samples) over space and/or time. For classical business problems where the inputs are much richer and more varied, and already have individual meaning, then decision tree techniques such as random forests may be more appropriate and do provide explainability.
>Neural nets are mostly dominant in perceptual domains such as image or speech recognition, where the raw inputs represent a uniform sampling of data values (pixels, audio samples) over space and/or time.
I mostly agree with that, however there have been advancement in scientific areas such as chemistry and biology (DNA/RNA) where the data are definitely meaningful and a lot times categorical. So the methodologies can be applied in wider areas, they just need a lot of domain knowledge and experience.
Except - and this rather ties into your point - those are not the only possible digits; your network also has to deal with (ie reject) other possible digits such as "P", "E", "3̸̶", or "[Forlorn Sigil of Amon-Gül redacted]"[0], which look like, but are not, decimal digits.
If you only train it on examples of 0-9 then those are the only outputs it's going to give. If you fed a "P" into such a net then the outputs would be the degree of similarity of that "P" to each of the (0-9) digits it was trained on. You could of course threshold the output and ignore any prediction with confidence less than, e.g., 90%.
If you wanted the net to do a better job of rejecting non-digits, or at least some specific ones, then you could include a bunch of non-digit examples in your training data (so now your net has 11 outputs: 0-9 and "non-digit"), then hopefully - but not necessarily - it's highest confidence prediction will be "non-digit" when presented with a non-digit input.
> If you only train it on examples of 0-9 then those are the only outputs it's going to give.
Exactly.
> 11 outputs: 0-9 and "non-digit"
IIRC, this doesn't work (or not well) because the net tries to find similarities between the various members of the set "everything except these ten specific things", but you could just require low confidence for all digits on non-digit inputs as part of the gradient descent function.
The problem is more that - if people with decision-making authority trust the AI to not be insane and evil by default - failure modes like this have to occur to you before the AI starts misbehaving in production.
There's no such thing as truly unsupervised learning - any machine learning algorithm is defined by some guidelines you've provided (or built-in) for what output it has to generate.
For example, you might think of data clustering as an unsupervised problem, but in reality you're still providing supervision in terms of what similarity measure to use and some control over how many clusters it should generate; and, at the end of the day, the output is always going to be a set of clusters, not, say, a stock tip or wry comment on the nature of your data!
Until GPT-3 can write something meaningful, it's really just a showcase of the technology and a gimmick of a product.
Sure it's cool, but what problem is it solving? As far as I can tell the only useful function it has is polluting the internet with pseudo-intellectual comments to promote some agenda (likely political). So now that I think about it, it actually would be incredibly valuable for things like subverting democracy.
I'm still waiting for AI to figure out self-driving cars, natural language processing, and other "holy grail" problems that researchers have been chipping away at for decades but we are still a long way from having "solved" them.
Also in the case of self-driving cars, there is actually a very serious problem of it being difficult to guarantee safety. You can't tie every input to every output to demonstrate how it will behave (it is way, way, wayyyyyyyy too big for that), so you have to show statistically by driving so many miles that it's better. Which is still sound from an engineering perspective! But it leads to things like Teslas driving into trucks parked sideways across the road because the engineers simply never predicted it would ever happen. And now it's happened multiple times!
I don't understand why the goal is to "guarantee safety". It seems to be generally accepted that human error causes approximately 90% of motor vehicle accidents [0]. Surely then the goal of any autonomous or semi-autonomous transport system should be to merely reduce that percentage? If all motor vehicles were "self-driving" and the total number of annual deaths was reduced by one, wouldn't that be a good thing? I would rather reduce my chance of dying early than eliminate the possibility of being killed by a bot.
Because humans are illogical creatures and will ignore objective facts if they believe control (or perceived control) is being taken away from them. The challenges of rolling out true autonomous vehicles are psychological as much as they are technical.
What is "meaningful"? Honest question. Isn't meaning assigned by a reader? If I'm reading poetry generated by GPT-3 and I like it just as much as poetry written by a human poet, does it make it meaningful? What if I finetune GPT-3 (or the bigger next gen version) on every scientific paper ever written, and as a result it generates a novel idea that turns out to be valid and useful, should I care if it happened by accident, or without "understanding"? Can the knowledge encoded in the model parameters be interpreted as some form of understanding? If no, why not? What's missing exactly? What is different from how human scientists operate?
In communication, meaning is a collaboration between writer and reader. The writer does their best to convey something; the reader does their best to understand.
There's also the kind of meaning that scientists and researchers talk about when they extract knowledge from data. That's pretty different from communication; it's more a process of internal generation of notions and explanations that could later be conveyed in communication.
And then there's the meaning that is even more internal. E.g., reading tarot cards or tea leaves, people generate meaning out of nothing. And then there's Pareidolia: https://en.wikipedia.org/wiki/Pareidolia
If you're saying machine-generated text is meaning in the sense of that last category then sure, it's something we loosely call meaning. But it's fundamentally the same as other sorts of cleromancy [1], just more elaborate.
Immanuel Kant would call the former "rendering a synthetic judgement" and David Hume termed the latter revealing the a priori knowledge. Plato would call both simply giving substance to the forms.
It seems none of this is something new. It's just that you no longer need a good education to learn about old ideas; you can come up with them on your own.
Different from what? Regular communication? Because the first involves trying to sync up two minds to have the same idea. The latter involves one mind, trying to generate an idea that ends up being useful. Useful in the George Box sense: "All models are wrong, some models are useful."
A search for "GPT-3 knowledge extraction" doesn't yield much, so I don't know what you're talking about. But as far as I'm concerned, knowledge exists in the heads of people, so I don't think we agree on that.
The thing is, if we could write a specific, closed-end, prescriptive definition of "meaningful" or "understanding" or whatever, then we'd be able to program it. And we can't, so we have to settle for something else, usually how a thing fails to be what we (indeed subjectively) consider meaningful. Still, it's not arbitrary.
The way that something like GPT-3 tends to fail basically is that you 2-3 paragraphs where paragraph 3 will containment statements that subtly contradict the semantics of paragraph 1. Things like seeing things inside closed drawers and other cues that the thing has no fixed world-model. That's what gives an impression of "meaningless" or "no understanding". (admittedly, I've only played with GPT-2 but this is a description of how texts written by these models at first seem plausible and then implausible as you read more).
It's not some philosophical objection akin to "nothing but a human brain can think".
What is "world-model"? What makes you think GPT-3 does not have some kind of a world model? It's clearly not a very good one, but at the same time it does not mean it can't get better. A 3 year old also does not have a very good world model, what's the difference between his world model and one of GPT-3? Again, clearly there's a big difference, I'm just not sure we know enough about what's going on inside 175B model to make any dismissive statements about it. I'm also not sure what will happen if you train a much bigger model on much bigger data. Things might start to emerge at some scale.
Well, GPT-3 isn't any kind of general intelligence - it's explicitly architected as a language model - something that learns to pay attention to prior context to predict what word comes next. The only kind of world model it has is a statistical model of what word is most likely to come next based on the corpus it has been trained on.
You could argue that general intelligence is also based on prediction, and that a human's world model therefore isn't so different in nature, but there are some very significant differences ...
1) GPT-3's model is based only on a corpus of text (facts, lies, errors, etc) it was fed... there is no grounding in reality.
2) GPT-3 is only a passive model - it's not an agent that can act or in any way attempt to validate or augment it's world model.
3) GPT-3 is architecturally a language model .. it can get better with better or more data, but it's never going to be more than a language model.
The difference between a 3-year old's brain and GPT-3 is that the 3-year old's brain is not a one-trick pony ... it's a complex cognitive architecture, honed by millions of years evolution, capable of performing a whole range of tasks, not just language modelling.
The 3-year old's brain also has the massive advantage of being embedded in a 3-year old autonomous agent able to explore and interact with the world it's world model is representing... It you tell GPT-3 pigs can fly then as far as it is concerned pigs can fly, whereas the 3-year old can go seek out pigs and see that, in fact, they can't.
> if you train a much bigger model on much bigger data...
Yep, no one is denying that the potential exists for something to be created that can do something useful.
However, the point being made is that GPT-3 doesn't do anything particularly useful.
The generated text from is not consistent, and apparently fine tuning can result in poor performance (ie. generates random crap). It scored better at a bunch of metrics, which is great, but of questionable practical value.
"world model" or not, it's currently interesting, amazing... probably not useful unless you're writing a spam bot.
What is unclear, is if bigger models will actually solve that or not.
We know much more about the world-model of GPT-3 (which is none, but it has a language model) than we do about the three year old (which we can't build or reproduce in silico) but we know plenty well enough to say that no, things are not going to magically "emerge" at some scale. You need something fundamentally different from endless cloze deletion exercises to learn what a drawer is.
Well you can randomly string words together and occasionally get lucky and make a meaningful argument, but that doesn't mean you've created a good method for constructing new ideas.
It seems to me the simplest way to decide whether or not something is meaningful (in this context) is whether or not the author (which is the algorithm GPT-3) can respond to criticisms against its own argument in a coherent way. In which case it has to get lucky twice, so it's that much less likely to happen randomly. If the author cannot respond to comments in a comprehensible manner, it's hard to defend the author.
can respond to criticisms against its own argument in a coherent way
I'm not sure about GPT-3, but let's imagine GPT-4 next year will be able to do this. It just does not strike me as a particularly high bar to clear. Let's go further, and assume GPT-5 in 2022 will pass the Turing Test (you personally will not be able to tell). What would you say then?
Please don't wildly speculate about technologies you clearly don't understand. "Does not strike me as a particularly high bar" means you're unfamiliar with how these systems work at a deep level (by which I mean, you personally cannot sit down at a terminal and build one). So rather than -ahem- making things up and asking "what then?" -- please ask for textbook recommendations on these topics if you'd like to know more.
I guess we'll just have to wait and see what happens next year :)
p.s. I built my first language model (LSTM based) back in 2014. Then I built a VAE based one. Then a GAN based. None of them were especially good, so I switched to music generation (this actually works pretty well). My most recent project is using sparse transformers for raw audio generation. Building novel DL models is literally in my job description.
Please don't wildly speculate about strangers you meet on HN.
In order to bet, we would have to agree on evaluation criteria. A task like "respond to criticisms against its own argument in a coherent way" is difficult to evaluate. Turing test is also pretty vague, and some people already declared it passed many years ago: https://www.bbc.com/news/technology-27762088
Yes, it would be difficult but not impossible to agree on clear criteria. We would also need an impartial third party to make a determination.
The Turing test hasn't been passed if I'm the judge. Supposing I were unable to tell the difference between any AI system and a human interlocutor defending the same argument, at any time before 2023, I'd admit my prediction was wholly incorrect.
I doubt very much I'll find a taker for this bet, however. The AI field has always been bigger on optimism than results, as we both know.
Yes, I agree TT has not been passed. But probably starting later this year we will be seeing more and more claims it has. At first it will be clear it's not. Then not so clear, and then the goalposts will be moved again, so that when GPT-5 is announced and it is clearly capable of keeping a conversation, the reaction on HN will be the same as the current reaction to GPT-3: "meh".
That seems like a pretty testable prediction. If I can't tell the difference between GPT-5 or whatever it is and an adult human native English speaker by the end of 2022, you'll win the bet.
How many questions will you need to determine if it's a bot? How many sessions, and what percentage of correct guesses will determine the outcome of the experiment?
If it wasn't a bot but a human, we'd typically have an unbounded conversation, except by the bounds of politeness, until I was satisfied one way or the other, so that seems a reasonable protocol here, perhaps with some reasonable upper bound on the time spent (an hour?) to avoid putting a potentially unbounded commitment on human participants. (Actually, "We've been chatting since four, I have somewhere else to be" is a pretty good signal of humanity... and should help you see why you're not going to win this bet.) I wouldn't expect it to take more than a few minutes, and I would expect to be wrong in (much) less than 1% of sessions.
The more time you spent chatting the better your chances are to guess correctly. I admit that in 2 years the models might not be good enough to fool you for hours. If you put enough thought into it, especially knowing what the bot was trained on, you could devise a set of tricky questions which would expose it.
However, I believe the models will be good enough to fool you during, say, a 20 question/response dialog. They will definitely be able to fool vast majority of unsuspecting humans. And they will definitely be able to keep track of conversation (remember what you said previously, and use it to construct responses to follow up questions).
How will the bot answer "when and where were you born, and how do you know?" How will it answer "what color, besides red, best communicates the flavor of a strawberry, and why?". How will it answer "What historical figure does my communication style make you think of most, and why?", or "Which of your family members comes to your mind first?" or "What do you think the context was in which the following poem was written?". I don't need to know what it was trained on to win this bet, and 20 open-ended questions is more than enough.
Between the vast majority of unsuspecting humans and me there is a considerable gap. Mind the gap!
You are kidding, right? All these questions you provided are extremely simple to answer, compared to many other things clever human interrogators might say during TT. I'm starting to doubt your NLP expertise.
The TT ready model I'm envisioning will be trained on many billions of chat sessions. It will contain dozens of preconstructed graphs and will dynamically construct dozens more (personality graph, common sense knowledge graph, domain specific knowledge graphs, causality graph, dialog state graph, emotional state graph, etc), it will have a bunch of emotion detectors, humor detectors, inconsistency detectors, lie detectors, praise detectors, etc. It will have the ability to query external sources (e.g. google search --> web page parsing --> updating relevant graph). All these modules will filter, cooperate, and vote, providing input to higher level decision making blocks. These blocks will use those inputs to condition and constrain response generation process. This is finally where a language model comes in, and this until recently has been the hardest part - generating a coherent, grammatically correct, interesting text, directly addressing a specific prompt. This part has been solved. Until GPT-2 last year we simply could not generate high quality text. Now we can, and GPT-3 is even better at that. Sure, there are plenty of non-trivial problems left to solve, but I don't view them on the same level of difficulty - some of them have already been solved in the process of IBM Watson development, so I'm optimistic. The hardest remaining challenge is probably constructing common sense graphs. [1] looks promising.
p.s. your questions are so naive I'm not sure if you're trolling me. A human might answer them like this (and a bot built 50 years ago could easily imitate that):
"when and where were you born, and how do you know?"
- [personality - redneck] I was born on a farm in Oklahoma. How do I know what?
"what color, besides red, best communicates the flavor of a strawberry, and why?"
- Red is the right color for strawberries.
"What historical figure does my communication style make you think of most, and why?"
- You talk like one of them big city hipsters.
"Which of your family members comes to your mind first?"
- My little bro Jimmy, we just went fishing together on Tuesday.
"What do you think the context was in which the following poem was written?"
- [depends on the poem] I don't get this poem. What is it about?
You're betting that in the next roughly two-and-a-half years, the common sense problem will be solved well enough to fool me, despite not having been solved in the entire history of AI up until now. I'll take that bet. How confident are you?
To fool you for 20 questions, yes. I'm ~70% confident, so I'll bet you $100 :)
To clarify, the common sense problem is a hard one. It is similar to level 5 autonomy driving. That will take a while to solve. But what we are talking about here is kinda like Waymo cars which can drive themselves in ideal weather at slow speeds in Arizona. So in 2.5 years I think the best chatbots will be as far from having common sense as the current Waymo self driving cars are from level 5 autonomy. Which is to say they will be pretty good.
You're on for $100. I'm > 99% confident that I won't be fooled by any AI before 2023, and would have bet any amount that I could afford to set aside.
Ideal weather at slow speeds, with a professional human driver throwing road-condition curveballs at you and challenging your responses? I like my chances.
>But it leads to things like Teslas driving into trucks parked sideways across the road because the engineers simply never predicted it would ever happen. And now it's happened multiple times!
This is a completely wrong reading of the situation. Tesla engineers clearly understood it is impossible to determine whether a stationary object is actually on the street or just an overhead sign with the equipment that is available on current Tesla cars and therefore they simply ignore all stationary objects. Because Autopilot is not a self driving technology this is not considered a problem. If a Tesla with enabled Autopilot ever runs into a stationary object it is clearly the driver's fault. It's extremely predictable and since it's easy to blame the driver there is no need to fix the problem by adding the required sensors to Tesla cars.
There's a ton of problems that a quality language model like GPT-3 can solve, beyond just spamming text generation - stuff like translation quality, automatic post-editing of text, classification, etc.
Isn't it good enough, or very nearly, to generate fake news? And if you can generate something that fools a large percentage of humans, even for a second or two, you can sell ads. It would be solving a problem for anyone who can profit from it. I thought the people who developed it stated that it was too dangerous to release widely? Dangerous = useful to bad people, no?
That was just PR fluff designed to play to the ideological biases of their Valley employee base. They released GPT-2 anyway some months later because other people were going to replicate it anyway, and guess what, the river of fake news we're flooded with daily is still not being generated by AI. It's being generated by journalists with an agenda, same as ever.
Don't get me wrong. You can absolutely generate news articles with these text generation models. But if you look at examples of the "fake news" generated by the monster GPT-3 model, it's not really any different to junk churned out by supposedly respectable news organisations i.e. grammatically correct but filled with logical contradictions, false statements that can be checked in 30 seconds with a search engine and so on. Most people already learned which news sources are trustworthy and which are pushing an agenda. If those outlets replace their journalists with GPT-3 it won't make any difference. The readers who don't trust them will still not trust them, and the readers who just want partisan cheerleading will continue to be satisfied.
The proportion of people who will spend 30 seconds with a search engine to check on something is practically infinitesimal. Even people who do it frequently don't do it most of the time. And it doesn't matter anyway, because it's too late - you only have to fool people for a second, or a fraction of a second.
Great, so focus on solving that problem generally instead of worrying about AI.
What I see is actually quite different. If the people who could detect fake news was practically infinitesimal then you'd see the vast majority of people having super high trust in the media. In fact most people don't trust the media, lots of polls showing that. Sure they may only fact check something occasionally (often by reading about a topic they happen to understand), but people aren't stupid. After they notice or hear about a few mistakes and observe they're always in the same ideological direction, they get the picture.
In ODS.ai Russian slack-chat a Replika developer has showed the increase in metrics that was provided by GPT-3 in comparison to GPT-2. Here you can see it on Telegram (also in Russian, but the pic is in English) without the access to ODS-slack. https://t.me/dlinnlp/931
The cycle keeps repeating. A new advancement in computing power, networking, or algorithms means there's a new batch of low-hanging fruit for AI to pick, so we pick it. Investors say "What about the high-hanging fruit?" and we say "No problem. We just need a slightly longer ladder."
Two years later everybody finally realizes the high-hanging fruit is on the moon.
My first AI teacher (even before '89) compared solving AI with neural nets to teaching pigs to fly by throwing them from a tower. Improvements come from building higher towers.
There's a recent NLP model that was trained on a trillion words. It would take us 10,000 years to read or listen (no breaks, no sleep) to that many words. Problems like attention, and the relation between memory and sequential thinking haven't been cleared up at all. Even semantics, i.e. basic understanding of a utterance or a scene, is in its infancy.
Large neural nets can help with interesting problems, but it's not going to mimic our style of thinking any time soon.
It isn't ever going to model our style of thinking. A "neural network" is just high-dimension linear regression; the idea it has anything to do with the brain is metaphorical nonesense.
No algorithm running on digital hardware can emulate the biological process of animal intelligence.
It seems clear that were limited by access to data and computer power. That has largely changed. However, the hard part has always been around deciding what question to ask and how to score the result. This requires domain knowledge and a scientific process (conjecture, measure, analyze, repeat). There is no magic bullet just better tools. Someone still has to define the problem and solution. Just look at IBM Watson and what a failure that is. They did okay on Jeopardy because they spent years optimizing for just that. Jeopardy could have changed the way they wrote clues by including irony and other language constructs and the humans would have done fine and Watson would have bombed.
"The result is an artificial idiot savant that can excel at well-bounded tasks, but can get things very wrong if faced with unexpected input."
I think this gets to the core of what is still a limitation of current technologies. Venturing into the unknown is still a deeply relevant task that seems unlikely to be replaced by computers anytime soon.
This is changing. In natural language processing just in the past couple weeks OpenAI wrote about their GPT3 model which can learn some tasks remarkably quickly, after only 1 or 2 examples. That model has extreme compute requirements, but it shows strong progress on performing never before seen tasks. There is still some steam left in the current deep learning boom.
https://arxiv.org/abs/2005.14165
Bob's manager: Here's Bob. You paid a gazillion dollars for him, and he can translate whatever you want. Any language, whenever – the true revolution everyone's been talking about. Your life and our economy will never be the same.
Alice: Bob, Can you translate "Eat my shorts" into latin for me?
If models actually were able to tell you what they know and don’t know, then sure. But instead they just give you an output for any input you give them, whether they have a clue or not.
This is such a simplistic view of a vast and evolving field of scientific research, it's too cartoonish of an argument to even warrant a real response. Which is why the conversation around ML gets negatively selected against actual researchers and practitioners, who get headaches from opinions like the one above.
What's wrong with their view. They're right that such models will always give an answer whether it makes sense or not. Has anyone tried asking Bob to solve sudoku? What happens? Does Bob fill in the blank spots according to sudoku rules and is the final solution an actual solution for the given board or does it end up being a random list of numbers?
Honestly last time I tried to get into machine learning the huge limitations made me lose interest. The idea that we can just pretend that these limitations don't exist is absurd. ML lets us do a lot of very interesting things but it's just a stepping stone. It's something we are stuck with rather than something that we want to keep.
This is a good article. I don't understand why people find it questionable and get so defensive about it. It does not deny the fast progress nor the potential of current/future techniques. It is simply an explanation for the layman of things that should be obvious to any practitioner in the field.
> I don't understand why people find it questionable and get so defensive about it
because people in the technology sector have a distorted view of their own importance. Despite the extremely meagre impact of the "computer revolution" on both economic growth and anything outside the world of bits, people in 'tech' fashion themselves to be sort of world-changing figures. AI on top of it is a sort of ersatz religion, the rapture of the nerds as Charlie Stross put it.
In probably 99% of AI/ML use cases the AI/ML is basically just a commodity item and the real “expertise” comes from getting and preparing good datasets for analysis, and having a clear problem to solve. The strategy behind something like AWS SageMaker is based entirely around this idea.
The problem is that too many companies believed it was the opposite so they built and hired all these AI/ML “experts” that just wanted to “built models” but didn’t want to focus in the messy hard stuff like finding and cleaning data. Nearly all of these AI/ML “experts” inside companies were also broadly just applying off the shelf tools and algorithms, perhaps with a bit of ensembling, rather than actually building new AI/ML approaches.
As a result, the big investments inside most companies produced a flash and puff of smoke that got people briefly excited followed by a lot of money spent with little business value returned.
I’m a big believer in ML approaches, but in most cases companies need to be focusing in their data first against clear business problem and just use off the shelf tools for the rest. That’s good enough for nearly all needs.
There’s a big bubble at the moment with all these “AI/ML” teams that’s going to crash hard as businesses realize the above and reset to focus on stuff that works and generates tangible value for the business.
We have also been watching these machine learning models for 6 months:
- increase the volatility in virtually every financial market they touched
- be exploited by adversarial learning networks to amplify funded propaganda as news
- use poorly contrived sentiment analysis to generate incomprehensibly meaningless news headlines
These non-linear "function approximators" have absolutely unpredictable and insane non-linear behavior where learned information was non-existent or sparse.
God help us all if one of these artificial intelligence devices is driving the road and sees a red stop sign that is a square, rather than a hexagon.
"AI" is a very vague term. What you described aren't entirely "machine learning", but a combination of existing linguistic techniques and machine (deep) learning.
People confuse what AI can do, and what is AI all the time. It also doesn't help when there are so many inexperienced data scientist making promises that they can't achieve.
In your example, I'd argue that a human is not necessarily a better driver than a machine. An attentive and careful driver is certainly better than a machine right now, but there are many who drive carelessly. While a person is unlikely to mistake a square stop sign as something else, there are so many drivers that would simply ignore the sign, and traffic lights in general. They'd also drive dangerously because of road rage, and inattentiveness. And the majority of traffic accidents are caused by these drivers. A machine is unlikely to do these.
That said, until we figure out how to run all these deep learning models without a crazily expensive and power-consuming GPU, it is unlikely AI would be used as general purpose programs.
Whether humans or AI are "better" drivers is completely beside the point. The point is that we can characterize human drivers. We know where they succeed and where they fail, both in a statistical sense and in an individual sense based on their age, attention, vision, chemical impairment, etc. But we cannot characterize ML networks. We take it on faith that they work and then we find (because somebody dies) that they run right into an overturned truck or a pedestrian or under a truck crossing the road.
Until we can characterize the behavior of these systems, they must not be put in control of life-critical processes like driving.
Just playing the devil's advocate but : when you take a taxi, what do you know about the driver? You can vaguely see if he's sober and that's all. You EXPECT him to have a driver's license, to have a good eyesight, etc but you KNOW nothing about it.
If he has an heart attack while he's driving on the highway, could it have been predicted (by you or the company)? No.
So i don't see why this distinction between AI and humans is made : both are black boxes. Perhaps humans have less "edge cases" but as long as the error level of AI is the same or lower than the one of humans, I don't care if the car crashed because the human driver looked at a sexy woman on an ad on a billboard or because a variable was poorly set in the car's code.
I also agree on this. I think in terms of liability humans who one can sue when they make a mistake is more valuable than a machine.
That's why in life critical applications companies who are capable of taking the risk are scarce, because when accidents happen, the company has to take responsibility. It cannot be resolved by just firing employees.
You can fix a software, but you can only punish a human driver, hoping it will fix itself. Also, both can be forced to train, but you can reproducibly test only the software, no guarantee that your retrained human driver will not succomb to the same road rage in the near future.
Even in something not very well defined as a neural network, you can try to retrain it, or also to modify its architecture, or its postprocessing, and verify reproductibly on test cases that it behaves better.
Also, to address your critics, you can add test cases (just like in any sotware. But actually they also do exactly that for hardware too).
Am I being a pedantic numpty or am I illustrating a point about the many ways errors creep in, regardless of the natural- or artificial-ness of the intelligence?
You're being a pedantic. Human beings are tremendously better at driving than machines despite sometimes saying hexagonal rather than octagonal. Humans and current AIs both make mistakes but humans manage a kind of robustness, ability to deal gracefully with unexpected situations, that current AIs don't seem to be progressing towards.
What happened to sensor fusion? There's no reason self-driving AI has to be as unreliable as toy or research AI. People made these same FUD arguments about computer in cars decades ago. Home computers were unreliable so cars will surely crash if their brakes or throttles are controlled by computers too.
Yeah. Totally makes sense to compare a human driver driving an average 7 year old 30k$ car with not so good safety ratings driven by average person in snowy, rainy pothole ridden roads to a newish 70k$ luxury car with good airbags/crumple zones driven in mostly sunny California roads by mostly young drivers with a driver assistance system.
And then for you to argue that the driver assistance system is actually better than a human driver if given the car alone!
Serious (and likely ignorant) question - what does linearity have to do with anything here? linear over what and why does non-linearity make something 'unpredictable'?
Linear models have more bias, so they represent current data less well and are more predictive of future, unseen data (think of a straight line through a point cloud).
Non-linear models have more variance so they represent current data better and are less predictive of future, unseen data (think of a line snaking around a point cloud).
An added complication is that deep neural net models are, in practice, vectors (or, well, tensors) of numbers so they are difficult to interpret. This and their extreme variance makes it hard to know how they will behave in the future.
The bias/variance trade-off is not really related to extrapolation. Think of a point cloud following a quadratic shape. A linear model will extrapolate terribly.
Well, "more predictive" doesn't mean it's a perfect fit. Every model has error. A line through a point cloud curving upwards will still represent some of the points in the cloud. So it will have high error, but it's still a representation of the data.
And yes, the bias-variance tradeoff is about generalisation (i.e. the ability to extrapolate to unseen data). But this is more related to the fact that in the real world, problem spaces don't have nice, friendly, regular shapes nor do their shapes stay put after we've trained a model.
My understanding is that generally, the error when extrapolating to areas not covered by the training data distribution would be considered to be part of the "bias" part of the bias-variance tradeoff.
The way I see it, the variance is the part of the error that you can reduce by collecting more data from your distribution and increasing model complexity if needed.
The bias part is what will not get better no matter how much you sample your distribution, and extrapolation problems fall into that category.
>> The way I see it, the variance is the part of the error that you can reduce by collecting more data from your distribution and increasing model complexity if needed.
Ah, apologies, I see what you mean. That is true, but this "error" is in-sample error, so increasing your model's variance will increase its ability to interpolate but not extrapolate to out-of-sample data, as I explain in my longer comment.
"In-sample" means all the data you've collected to train and test with. It includes training/validation/test splits. At the end of k-fold cross-validation, your model has "seen" all the data in your sample and the model that performs best is the model that best represents that data.
But, because the data was sampled from a distribution that is most likely not the true distribution of the data (since that distribution is unknown), the sampling error (i.e. the differences between the true and sample distributions) will be reflected in the model. A high-variance model will suffer more from this than a high-bias one.
Sorry I didn't understand immediately what you meant. The longer comment above is correct but probably doesn't help answer your question directly.
Thanks for taking the time to write the detailed reponses. Definitely led me to think more closely about these vaguely held intuitions about bias and variance! I think you are exactly right that the crucial aspect is the variance when looking at out-of-sample predictions, not just across several samplings from the original training distribution (a la k-fold crossvalidation).
Bias and variance are characteristics of the model, not components of its
error as I think you're saying. In the most simple sense, bias and variance
refer to the shape of the function represented by the model (let's say "the
shape of the model" for simplicity). A model with a more "rigid" shape
(approaching a straight line) has more bias and one with a more "relaxed"
shape (further from a straight line) has more variance.
The extent to which a model can extrapolate to out-of-sample data depends on
how well the shape of the model follows the true distribution of the data.
This is true regardless of the bias and variance of the model. It just happens
that most of the time, in interesting, real-world problems, the true
distribution of the data is more or less different than the sampling
distribution of the training data- i.e. there's always some amount of
"sampling error".
Sampling error can't be reduced by collecting more training data- you just
have more data with the same sampling error. Increasing model complexity
increases variance, so if you start with high sampling error, you wil get a
high error on out-of-sample data because your model matches the "off"
distribution of the training data too closely. What training with more data
and with a more complex model can do is increase the ability of the trained
model to interpolate, i.e. to accurately represent (new) data points that
are in the same region of "instance space" as the training data points.
A high-bias model can extrapolate well if the sampling error is not too high
and the shape of the true distribution is not too irregular. However, a
high-bias model will also not interpolate as well as a high-variance model. Its
rigid structure will "miss" many data points. Like you say, this will not
change if you train with more data. Anyway, that's the tradeoff.
Now, the reason why deep neural nets, which are extremely high-variance
models, are trained with large amounts of data, is that they can interpolate
very well but can't extrapolate very well. If a model doesn't extrapolate very
well but its training sample is a large enough chunk of instance space, it can
still be very useful, because it's still representing a large number of
instances.
How to put it? Mabye your high-variance model has seen examples of white dogs
and black dogs in training, but no green dogs. Your model will not be able to
generalise to green dogs, but if green dogs are rare, it will still be able to
represent most dogs, so it's still useful.
Of course, looking at the output of a trained model (its behaviour) doesn't
tell you anything about what it was trained on. So a model that has very high
accuracy on a large number of tasks will look impressive, even if it can't
generalise at all.
I'm not good at math, but I'm confused by the association of AI with non-linear stuff, setting aside the association of non-linear with "bad". I thought ML involved linear algebra or something (says xkcd!) which would presumably be...linear?
The inner activation function (AF) of neurons is inherently nonlinear; it has to be in order to solve any problem that is not linearly decomposable (which is basically all of the interesting problems). Often the AF nonlinearity shows up as a thresholding operation following a linear weighted sum, but that's not the only mechanism.
And yet neurons are not "pure" binary thresholders the way logic gates are because you can't take the derivative of a binary function, and you can only do backpropagation on differentiable functions. The compromise neurons make is a "smoothed threshold" or sigmoidal curve which is differentiable but still very nonlinear.
I'm not sure where the "linear" in "linear algebra" comes from. You hear about
linear algebra in relation with machine learning a lot because training a
neural net (with the backpropagation algorithm and friends) requires some
matrix arithmetic. Inputs to neural nets are vectors or matrices, their
weights are (arrayed in) vectors or matrices, their outputs are - well,
usually scalars but can also be vectors or matrices.
Also, the use of linear/ nonlinear in machine learning is a bit misleading. A
"line" is not necessarily a "straight line", but usually when we say "linear"
we mean "straight" and so when we want to say "not straight" we use
"nonlinear".
In any case, when we say "line" in machine learning we mean a function, the
function of a line. So a "nonlinear" function is a function that curves and
turns, e.g. a sigmoid, whereas a "linear" function is straight as a rod.
Why a line? Classifiers er classify by drawing a line through space. "Space"
means a Cartesian space where our training examples are represented as points
(hence, "data points"). Data points are located in Cartesian space according
to coordinates that represent their attributes, or features (these coordinates
are the "feature vectors" that are input to neural nets). We classify data
points by drawing a line between those that belong to one class and those that
belong to other classes. More to the point, when we train a classifier, we
find the parameters of a function of a line that separates the points of
separate classes and when we want to classify a new point, we look at where it
falls with relation to that line.
So that's where all that stuff about lines and "linear" and "nonlinear" models
comes from. A "linear model" or "linear classifier" can only draw straight
lines. A "nonlinear model" can go twirling around madly.
Finally, "non-linear" doesn't mean "bad". There are tradeoffs- in particular,
the "bias variance tradeoff" that I hint at in my earlier comment. A linear model
is more limited in what it can represent, but a nonlinear model is less likely
to represent data that it hasn't seen in training.
- "linear" in "linear algebra" comes from "system of linear equations"
- NN can absolutely represent non-linear functions, and they are based on solving system of linear equations.
- The non-linear function here has nothing to do with the linearity of the system of linear equations used to construct it.
- The two main sources of non-linearity are, (a) the inputs (e.g., an image, or a series of images varying a non-linear fashion), and (b) the activation functions.
The underlying derivatives are linear (like all derivatives) but neural networks' ability to approximate arbitrary non linear functions is one of their biggest strengths.
Yes, so I'm left wondering, when making the association of the math to the badness, how do you decide if the linearity or the non-linearity is the salient part?
Mathematically, you can think of "linear" AI problems as "easy to solve", and non-linear as "difficult". That's part of what the parent means.
Some function being linear means it's easier to guess. If a real world phenomenon is tied to a linear function, then it's easy for AI to guess/approximate.
If you have ever opened up Excel or a similar program. One of the more useful options is to generate a regression line-fit on your data points.
One option is to specify a polynomial function, you can specify how many coefficients you want. One of the measurements is the mean-squared-error between the line-fit and the points.
You can add as many polynomial coefficients as you want, and you will be able to decrease the mean squared error. But the more polynomial's you choose, two things will be true:
1. The line-fit will be far more likely to go through the points.
2. At points in the line where there was no data, the line will less approximate the underlying physical reality.
That same mathematical property is what is relevant here. There is nothing inherently evil about non-linearity, when the non-linear math model properly maps to the physical reality. But when you over fit a line, many of the functional solutions may be completely wrong.
I'm confused. I agree that overfitting can lead to very bad models.
But, what I don't understand is that I thought that "linear" in ML contexts was normally used in the sense of 'linear transformations', which is a sense of linear that 'line-fit' from excel isn't -- it's affine.
Is a linear model with thousands/millions of weights/parameters (like deep learning models) really substantially simpler to understand? Can it do anything useful?
I suppose from the perspective of someone implementing these models, yeah - it is linear, but it is not bijective. In a system with only one layer, that manifests as an alias (assuming the output dimensions are smaller). In a system with multiple layers of either `N->M` or `M->N`, those aliases tend to manifest as apparent "non-linearities".
So, I guess looking from the bottom up the system may look non-continuous and linear. But if you look from the top down, it would look continuous and non-linear.
I assume they are using non-linear to mean non-continuous, which implies that there can be large, hard-to-understand changes in behavior when the input is changed only a small amount.
Polynomials with large degrees are continuous. It's just that they can still change by a large amount (i.e. having a large derivative) when the input is changed by a small amount.
I invite you to construct the Lagrange polynomial (i.e. interpolating polynomial) for points on a nice, simple curve with some noise. They will, by definition, pass through every point given, and yet it will likely behave very badly outside the range of the given points.
There is nothing wrong with using a non-linear model, though; x^2 or x^3 regressions make sense on many datasets.
Non-continuous is also not the perfect terminology, but I argue that it is more precise than non-linear: the chief idea being that the model "changes unpredictably."
How have machine learning algorithms negatively affected financial markets in the last 6 months? Markets have been volatile because information about the real world has been volatile. I don't think markets in an earlier era would have handled a global pandemic any more robustly than they did in 2020.
Deepfakes (faceswap but for porn). Decensored hentai. And of course the question came up again: how ethical are generated pictures depicting illegal content?
Intelligence is the amalgamation of many smaller problems working together and building on top of each other.
* Facial recognition/detection
* Facial synthesis (deepfakes)
* Speech synthesis, including mimickry
* Speech recognition
* Natural language processing
* Gait/walking algorithms
* Motion planning
* etc.
Complexity arises from simple units working together in parallel. We're working on the smaller, specialized problems that will, in the next generation, be put together to build more complex and complete systems.
I'm no fan of the 'black box' nature of neural networks but it's clear they're getting results. As they become more accessible to the lay person, we'll see a profusion of use cases that are both anticipated and surprising.
I'm always flabbergasted by the doom prediction. The path we're on seems apparent.
I agree with the notion that artificial intelligence is a graph of smaller problems, as is human perception.
The problem is a question of informational density. Biological systems are computationally very dense. Far more dense than the 4nm transistor fabrication available today, and with a far larger volume of size.
Consequentially, the computational capability of most AI systems is far lower than its biological equivalent. And as you find in most information finite discretization problems - the lower density information system will alias against the higher information system.
So, that means you will have a hierarchy/pipeline of computational stages - each aliasing reality. Eventually, you will find that your parameterization of each perceptual stage has a strange property. The size of each subsequent layer is important... but the relative computational space of each subsequent stage is even more important. Because mismatched stages results in nothing but numerical interference and noise.
And I think that is where we are today. The IQ of a krill shrimp.
> Isn't "the amalgamation of many smaller problems working together and building on top of each other" a fair description of the theorical Unix system?
Yes. Note the success of Unix and the ability to scale, do work and provide an environment to be productive in.
> Aren't your criteria for "intelligence" human-centric, implying that there is no other form of "intelligence"?
Your use of 'human-centric' is odd. I would have thought the traditional 'human-centric' theory of the mind is something monolithic and indivisible. Suggesting that it's many small processes communicating with each other is basically taken straight out of nature, from ants, schools of fish, birds flocking, etc.
Whether there are other forms of intelligence or no, it's clear that incremental progress in individual processes that can then be composed together is a productive way to traverse the energy landscape. This is why (imo) we see so many symbiotic relationships from cells on up to higher level animals.
> Aren't your criteria of the "black box" type, given that AFAIK no human can really completely explain how he recognizes faces/does NLP/walks/...?
I'm not quite sure what your point is here. If you're critiquing me about neural networks being black boxes and not giving us real insight into the underlying system, that's fair and the reason why I said I didn't like the black box aspect of neural networks. I will say that if there is a black box model that can be easily manipulated, this will probably lead to deeper models much quicker.
If you're saying that human cognition is not describable by any human and, I guess, implying that it's indescribable, I would point out that one doesn't follow from the other. Not having a good model right now doesn't imply we won't understand it at some future date and, in my opinion, this is precisely what's happening. Having no human be able to describe the underlying computation (of face recognition, nlp, walking etc.) doesn't mean it's indescribable, it means it's not describable by anyone right now.
At one point we didn't know how birds flew. We still might not know, to your satisfaction, but we have a basic understanding of how to make things fly, both in practical and theoretical terms. Planes fly and we understand how even though they don't flap their wings. I have no doubt we'll figure out how to do complex human-level computation even if we don't have a deep model of the specifics of human thought.
>> Aren't your criteria for "intelligence" human-centric, implying that there is no other form of "intelligence"?
> Your use of 'human-centric' is odd. I would have thought the traditional 'human-centric' theory of the mind is something monolithic and indivisible.
Sorry, my answer wasn't clear. It was not tied to the "small processes communicating..." approach but to your very list of "problems" (facial recognition/detection, facial synthesis (deepfakes), speech synthesis, speech recognition..) which seems to me expressed in a way tied to human activities, while at least part if the underlying "intelligence" underlying many of them may also exists in other forms of life (other mammals, birds, fishes...).
> Suggesting that it's many small processes communicating with each other is basically taken straight out of nature, from ants, schools of fish, birds flocking, etc.
Exactly. My point is that analyzing the ways "the smaller, specialized problems" are tackled by non-human living beings seems pertinent as self-analysis (as humans analyzing human intelligence) is difficult, and as various species may apply various solutions, some more easy to grok. Focusing on "problems" too specific to the human being may be a sort of "framing" detrimental to the quest. Moreover the famous Dijkstra quote ("The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.") may be pertinent.
> incremental progress in individual processes that can then be composed together is a productive way to traverse the energy landscape. This is why (imo) we see so many symbiotic relationships from cells on up to higher level animals.
I agree. My point is about _how_ we consider the system(s) (our "point of view"): framing it to human characteristics, globally or locally (dualism)... It seems to me that the very organization of a system may be neglected when we consider it a stack of "small processes communicating...". Pirsig's "Metaphysics of Quality" may be pertinent.
>> Aren't your criteria of the "black box" type, given that AFAIK no human can really completely explain how he recognizes faces/does NLP/walks/...?
> I'm not quite sure what your point is here. If you're critiquing me about neural networks being black boxes and not giving us real insight into the underlying system, that's fair and the reason why I said I didn't like the black box aspect of neural networks.
This was my point and I agree with you.
> if there is a black box model that can be easily manipulated, this will probably lead to deeper models much quicker.
I'm less optimistic, as it is only 'probable', and AFAIK won't give us more real insight into the underlying system.
> Having no human be able to describe the underlying computation (of face recognition, nlp, walking etc.) doesn't mean it's indescribable, it means it's not describable by anyone right now.
> I have no doubt we'll figure out how to do complex human-level computation even if we don't have a deep model of the specifics of human thought.
I agree, we will enhance ways to "approximate" (tricks leading us to a solution to each "local" problem) up to the point of being able to solve real-world problems. However it may reach some hard limit (as far as I understand this is the point of the article), and using a powerful tool/method insufficiently understood may be dangerous.
Watching this presentation did not make me any more confident about going in a self-driving car.
Based on the way it was presented, I got the feeling that they are just essentially manually identifying cases and addressing them as they see them. Is that solution really helping to make the system more robust when encountering an unexpected situation?
This is so strange. If you use facebook, google, netflix, apple, microsoft, amazon or a whole host of other services you are interfacing with AI all the time. To think there’s no value there is asinine. Comes up a lot on HN. Seems like people set in their ways who don’t want to progress forward.
oh yeah, magnificient AI at Google search. Picked up my ebook-reader again, wanted to know about the state of linux there. so do a search: "<model of ebookreader> linux ssh" (since a good shell is the point, where you can start developing). Turns out, the first 3 pages want to sell me the same thing I already own, with one outlier selling nutritional supplements. Oh well done AI!
Unfortunately, Google Search (and Amazon and MS search and all three companies' assistants) doesn't work that well when it comes to selling you what you want to buy either:
idk, the thing Google has been doing lately where they suddenly render a block of ads under your mouse as you're clicking on a result seems like the kind of thing an AI would do to increase ad clicks
There's a nearly unimaginable amount of money and computing power going to minimize the net value of the service + ads. If you get significant value, it's going to be optimized out in a few minutes. The ads on Youtube recently ramped up to where even something short is unwatchable for me, and it doesn't work at all with an adblocker.
Economist editor to writer: This year, X is on the down slope of hype cycle and no one is talking about it, gimme something ASAP.
Economist editor: opens their "pessimist template"
"X has over-promised and under-delivered, A, and B have not commercialized yet, and may never be. X cannot do C yet. The challenges of X are D, E and F"
Replaces A...F with the most prominent examples they can find, Boom, we have an article.
I read through the article, hoping for a bit of important info. I wish we had a vote in HN: "Isn't worth your time, don't read it"
My general guidelines about AI (Machine learning, programming):
1: Computers can't read minds! Your algorithm might know that I like the Beatles because I listen to them a lot, but it can't predict that I woke up today craving to listen to some music from my childhood.
2: You don't know what you don't know! Your algorithm might make 24 frames per second film look smoother at 60 fps, but if something like wheel spokes move backwards at 24fps, it'll have a tough time getting the wheel to move the right way at 60fps.
3: Just because you have the information, doesn't mean you know how to write a program that can extract the knowledge you're looking for.
Which really means we need a super advanced AI with a worldview and context in order to automate certain kinds of information processing. I don't expect this anytime soon.
Like others are saying the progress towards AGI isn’t great but each individual subdomain is seeing great advances all the time. Object detection, facial recognition and NLP with GPT are much better than they were a few years ago. Each of these can provide business value to a certain degree, but I would agree that something resembling AGI holds the most business value. For this to happen all of the pieces have to be put together somehow - right now research focuses on specific subdomains and improves the SOTA on them. Once someone figures out how to make everything work together, it could mean a second, much larger wave of AI. So the question is, when will that happen?
Apart from a true GAI, the research has progressed really fast. We’ve seen a huge leaps in e.g. image generation and speech synthesis during the last few years that I’m really interested on what the future will bring.
For a rather complete view of what deep learning still does not do, I recommend the work of Gary Marcus and Ernest Davis. While seeming solely critical, I think they make very good points about the limits inherent in deep learning as we know it now, and how it needs to grow to overcome those deficits.
Maybe it's because everyone talks/chats every day with "virtual assistants" at banks and every other organization, and never ever finds them useful. Their main purpose is to frustrate you enough so you give up trying to connect to a real person.
Judging by the number of times and different places I've heard the sentence "Please listen closely, as all our options have changed" even when they haven't changed in years suggests that this is new whine in old bottles.
I think there's an opportunity right now for new, human curated indexes in the style of the old Yahoo index. AI content generation is getting too good at fooling AI curation and I'm getting less and less value in broad searches on Google. I would pay a monthly fee for hand-vetted lists of the best content on topics I'm interested in.
It's limited but so effective! The other day a friend asked for photos of her sister (also a friend) that I had because it's her birthday and she wanted to make a collage. I just searched on Google Photos by her name and it found a bunch because of the face classification. That's some good shit.
The hard part of that would probably be getting unauthorized access to the photos in order to modify them in place, rather than the adversarial image perturbation per-se.
These days, you can translate text by pointing your phone at it and taking a picture. Thirty years ago, this would have been unambiguously AI, because it would have been not only impossible, but stupid impossible like something out of a soft SF novel where little self-flying robots deliver stuff to your house, or you can ask a computer a question in a natural voice and reasonably expect a civil, natural-language answer sourced from global databases.
Ah, but all that works. If AI is perpetually defined to be "that which does not work" then it's perpetually potential, perpetually postulated, perpetually possible perfection. It never has to be compared to a clunky translation, or a drone that gets shot down. Unwritten novels never have story problems in the third act, unwritten programs never give ludicrous output.
Yes, but you're missing the point. In 1950's sci-fi, those marvels were possible because there was imagined to be something like a general artificial intelligence behind the technology. We've achieved narrow AI, but the perception is that in order to get it, we would already have general AI, which is why people are disappointed.
Whether 50s sci-fi imagined or implied other things is irrelevant to the question of whether or not the current capability described (point camera at sign, get translation) qualifies as AI.
The point is we have current things that are quite amazing, and would at one time have been considered to be the sort of thing that only an AI would be able to do, and yet we keep moving the goalposts. As if AI is defined as "that which humans can do but machines can't, done by machines".
I don't get your Jesus analogy, I mean Jesus is a proper noun for an individual.
A technology is different. Its capabilities pretty much define it. Unless you are going to try to get all philosophical about it and say it doesn't count if it doesn't experience qualia or something (which is nonsense) Or, unless you are defining it in ways that specifically call out the implementation details. A helium balloon isn't a hot air balloon, not because it doesn't have capabilities, but because you've specifically said in the name that it must use hot air.
If a client/muggle asks me to build an “AI” I’ll be wary that their “spec” is sometimes just examples of what the “AI” should be able to do: play chess, write a poem.
In their mental model, the AI is far from defined by these capabilities. They won’t be happy unless there’s an actual AGI whose capabilities happen to overlap with the spec.
So my point is historically we have cheated our way out of defining “intelligence” and instead given necessary but insufficient examples. I think this is the mechanism behind the goalpost-shifting in “AI”.
That’s the metaphor: it would clearly be absurd for engineers to define Jesus by some examples of capabilities. People who are waiting for the second coming won’t be satisfied.
In the lab we maybe should define AI by some set of capabilities. But clients, journalists etc. picture the Hollywood version, and narrowly fulfilling their spec won’t actually satisfy them.
this is also one-sided though - the translation is cool but only kind of works and the quality varies a lot based on language, becoming unusably bad for many languages. so there's some amount of dismissal by AI boosters of its failures that people have grown accustomed to.
I presume that is not due to a failure of the ML, but instead is often due to the lack of a good corpus of text, or the lack of appropriate manual training/nudging/correcting of the model for unprofitable languages.
I’m a longtime AI skeptic who has been arguing passionately against the doom-sayers, many in my own family and in casual conversations with laymen, for several years. I stand by this article I wrote which summarizes my views https://medium.com/@seibelj/the-artificial-intelligence-scam...
The hype on AI was absolutely astonishing. I’m glad it’s finally coming back to reality.
The more I understand AI technology the less magic and powerful it seems.
More traditional AI is often logic or algorithm based. This is powerful and expressive, but the "intelligence" part is as flexible as the specific implementation. This is intelligent in the sense of a design being intelligent, which can be attributed to the author of said design.
Modern AI is often statistical. We have more data, so we can use that to generate a de-facto decision making database with statistically sound heuristics.
These techniques are all useful and can be very impactful. But we have to understand them as tools.
Your article agrees with this sentiment in a very entertaining way.
That article is well argued and I am inclined to agree. But then I noticed that we seem to strongly disagree on other topics.
I find it fascinating that you can both see AI as what it truly is (a rebranding to solicit investment) yet still cheer for cryptocurrencies, which I personally believe to be not much more than a pyramid scheme to dupe those investors that buy in late. In my opinion, both AI and crypto are quite similar in the ways that they mislead investors by promising a golden future.
I am a longtime crypto guy, and totally 100% understand how you see the links. I have a very nuanced opinion on this, but it's difficult to explain briefly here. There are a lot of scams in crypto, but if you are a believer in Austrian economics and are not a fan of Keynesian theories, fiat currency, and the Federal Reserve, then Bitcoin / crypto is very attractive.
But that's a separate discussion, unrelated to my opinion of AI and is primarily a philosophical issue. Cryptocurrency / blockchain is fascinating but we are not promising robots cleaning your house.
I agree that AI has been way worse with the overpromising.
But when I hear IPFS = interplanetary file system and then wee how poorly it performs in practice and that it's mostly used for illegal content, I cannot help but think that the crypto side also likes to oversell their practical utility.
I believe I have yet to see an application where the Blockchain is truly a critical component. In most cases, it seems that people end up caching its data in sql to speed things up, meaning that they're working on their own private copy now.
With the current transfers fees, it is difficult to do much apart from rarely moving large payments. For old-school currencies, that would be called speculation.
If you now consider that both the mining difficulty grows over time and the mining reward drops, then you clearly have a system with a built-in advantage for early adopters.
So people buy in, wait a bit, then exit at a higher price. But that only works as long as you have a large enough stream of newcomers.
I'd be willing to consider it an investment if there was some sort of inherent value on the other side, so if buying cryptocurrency gave you a claim of ownership on something real. But the critical fundament of cryptocurrency is that it's unrelated to the real world and only controlled by its members. In other words, the value of a cryptocurrency is determined exclusively by what people believe it should be.
I agree with your skepticism, while also acknowledging that A.I. in all its incarnations is a powerful tool. But the argument against AI, apart from "this is obviously limited", is much more subtle than what you are claiming in your article. (IMO, it comes down to quantum physics... Hopefully one day this will become obvious to everyone when we all have quantum computers.)
People are going to complain that AI is lame right up until the point that it gets general enough to make them all irrelevant in terms of work productivity. Then rather than modifying society to distribute the gains, they will leave the outdated structures in place and try (too late) to suppress it.
At no point (until it's too late) will there be be effective legislation discouraging the creation of fully general and autonomous digital persons that compete with humans.
Amazon retail is currently in the process of baking advanced deep learning models into every area of their retail process. Its all becoming automated. Just because its not talked about openly doesnt mean its not happening.
These people at the economist are completely out of the loop.
I'm still not sure what successful AI implementations there have been. Stuff like Amazon/Spotify recommendations seem sensible. Is there anything else out there that is impressive?
Autocomplete on emails which allows me to tab over for common statements. But I agree - the places where AI has improved business can be counted on with one hand, and each finger is extremely thin.
Everyone always looks to consumer tech but at least where I am at the far more successful and interesting domains are outside of that (and frankly most serious jobs too).
I used to work for a lab doing coral reef sea floor mapping to track reef progression year over year, this was only possible because of recent advances in ml. This is just an example of course.
There is tons of work being done (and successfully used today) in healthcare domains, weather modeling, wireless rf tech, geospatial remote sensing etc etc etc.
My point is AI is being leveraged a lot and is moving forward quickly but the applications for average consumers that are both cost effective and highly useful are growing thinner.
If you want to see consumer tech examples that have been at least moderately successful I think some highlights are Google photos image recognition, current speech to text processing in most places (Apple google amazon), image processing in most smartphone cameras (wrt ml especially night photos) and unfortunately (IMO) a lot of content aggregation algorithms (YouTube, Facebook etc) optimizing for engagement.
Nvidia's DLSS[0] seems to be pretty successful in achieving its goal, which is to allow games to render on lower resolutions without sacrificing too many details.
Google search has gotten worse over the years though. Perhaps that's because the amount of information is growing exponentially. So much of the search results seems spammy these days.
It's still quite smart though. For example, asking a question about a programming problem often brings up the correct stackoverflow page, even if the wording is completely different.
maybe the best way to conceptualize this, inspired by @random_walker, is to compare AI in the 21st century to machines in the 20th century.
in the industrial age, machines automated rote physical tasks.
in the information age, AI could automate rote mental tasks.
the more objective and templatized the task, the more vulnerable it is to AI displacement. conversely, the more subjective and creative the task, the safer it is from AI displacement.
The problem with articles like this is that they don’t properly represent the spectrum of what an ML project can be for a business.
If all you’re talking about is self-driving cars or voice-operated assistants, then sure, the article’s mostly right. Modern techniques that have revolutionized ML in the past ~15 years have not translated to massive new economic gains in many areas they were anticipated to affect.
But this is the vast vast minority of all ML projects.
Many of the most economically successful ML projects I’ve run in my career are very simple, and ruthlessly focus on business value from the outset. A lot of them involve automating inefficient manual processes, things like spam filtering, phishing detection, fraud detection, automatic keyword tagging, automatic metadata classification in images or text, simple time series forecasting for logistics or consumer demand, simple models for customer churn, and a wide variety of different customized search engines for big & small content collections.
Just for one example, I worked on a project to automatically validate metadata about human models appearing in images, to flag discrepancies between documented ages / ethnicities within legally required model release documentation and the real appearance in images, to find fraud (especially when minors were used in stock photography).
This saved _millions_ of dollars annually in human review & legal costs for when that platform incorrectly approved photography with invalid or fraudulent accompanying release documents for the human models.
In just one project, a team of six engineers paid for itself about 5 times over and the delivered software requires minimal maintenance and only became more valuable as the platform grew larger. In fact that was one of the only times in my career when a non-finance company chose, discretionally, to pay larger bonuses than in employee job agreements as a reward.
That project did happen to use deep neural network for image metadata prediction, but it was fairly mundane and easily trained on 2 average GPU machines from a dataset of only a few hundred thousand images.
Edit: added below
I’ve also observed across several companies that there’s a big variation in outcomes and success of ML based on the level of investment in infrastructure.
It’s not about pumping money in for some crazy GPU cluster or huge framework for massively parallel training, but you do need to separate ML operations away from the ML engineers who research solutions for products and internal stakeholders.
It’s a situation where domain specialty has to be used efficiently or you’ll waste a ton of money and time. If you hire an expensive senior engineer for ML (salary easily north of $200K in large cities), but you task them with managing a database or operating kubernetes or debugging partitions in HDFS, you probably won’t get a good return on your investment.
There should be something like an X prize for a recycling device. Take a trash pile, sort through it and get things ready for recycling. This would solve a big environmental and would deliver progress in robotics, vision and AI. I'd be more excited about this than self driving cars.
Perhaps continuous learning. A machine learning system that can adjust to changes over time by retraining itself.
However, training is computationally expensive and generating labels for training data is still done mostly manually by humans. Just like people, a continuous learning algorithm will have trouble distinguishing true from fake and could be corrupted by feeding it fake data. There is machine learning that doesn't require labels, but much of the recent success has been built using human labeled data sets. Human biases get into the labeling, and appear even in the methods of selecting and collecting what data to train on. One way to design continuous learning systems involves wrapping simpler machine learning algorithms with data collection and feedback loops to handle the retraining. Ensuring accuracy is a difficult problem unless your problem domain comes with some built-in measures. Some measures of "correctness" are arbitrary such as those defined by culture and can change over time or geographically.
Robot manipulation in unstructured situations is still not very good.
(See the videos of the DARPA humanoid challenge) "Common sense", defined as the ability to predict the consequences of actions and to use that to plan, hasn't progressed much in years. Machine learning doesn't seem to have helped much with either, so far. Those are key areas for doing physical things in the real world. Humans are good at those.
So that's where to look for hard problems.
The payoff is low, though. Those skills are common to all healthy humans, so there's a huge pool of unskilled labor available with them. If you did a startup, and you solved, say, robotic shelf stocking, it would not be a huge win over low-wage people.
A "programming guru", to me at least, doesn't cover his accomplishments in algorithms, among others. It conveys someone who produces a lot of good, high quality code, but not novel research.
So now that it's not the latest hype train, we're back to calling it "AI"? For the love of Christ, dear Economist, ML and AI are not the same thing and we have words for both of them. Use your words, people!
An understanding of AI's limitations is as far from sinking in as the average MBA is from comprehending Finnegan's Wake. An understanding that machine learning is not the entire 60-plus-year-old field of artificial intelligence would be nice to see for once from an institution that supposedly prides itself on precision of language and accuracy of reporting.
An understanding of Crichton's Gell-Mann amnesia effect is slowly starting to sink in, though, to at least one former subscriber.
Do you have NoScript or some other blocker on? As far as I can tell, Economist.com is using smallcaps for all acronyms but for some reason, the smallcaps are implemented solely through JavaScript, so if you read without that, you just see lower-case. The lowercase is because if you have uppercase, smallcaps does nothing - it can't become 'small capitals' because it's already large capitals, as it were. Letters need to be lowercase to be smallcaps. So, that's why you see 'ai' instead of 'AI'. So the 'ai' can properly transform into 'ᴀɪ'.
This is bad because JS is not necessary, and it is also not necessary to write in lower-case in the first place! I know this because I do a similar thing on gwern.net: in addition to manually-specified smallcaps formatting, a rewrite pass while compiling automatically annotates any >=3-letter acronym. However, I do it purely via CSS, and I also don't need to lowercase anything. How? Because a good font like Source Serif Pro will include a special feature which will smallcaps just capital letters: 'c2sc' in `font-feature-settings` CSS. So to do smallcaps correctly, of regular text & uppercase, you have 2 CSS classes: "smallcaps" and "smallcaps-auto". "smallcaps" gets the normal 'smcp' font feature and operates the usual way, lowercase gets smallcaps, uppercase is left alone. "smallcaps-auto" is used by the acronym rewrite pass, and it does ''smcp', 'c2sc'' instead, so "AI" does indeed get smallcaps.
This way, I need 0 JS, I don't need to write everything in lowercase, copy-paste works perfectly, I don't need to manually annotate acronyms unless I want to, and everything Just Works for every reader.
(I don't, however, smallcaps two-letter things like "AI". That's just silly looking.)
I just disabled JavaScript by setting Firefox's "javascript.enabled" flag to false, and it still renders fine for me. Here's what the HTML source looks like when I grab the page using curl:
predicts that artificial intelligence (<small>AI)</small> will add $16trn to the global economy by 2030
So it looks like it's coming through in caps from the server. I don't think a lack of JavaScript by itself is causing the problem...
> I don't think a lack of JavaScript by itself is causing the problem...
<small>, however, does not do lowercasing. All it does is make the font smaller. 'AI' should be caps regardless of whether it is wrapped in <small> or not. That implies that something in the CSS or HTML is overloading <small> to make it lowercase, so it can be correctly transformed to uppercase smallcaps for the reasons I explain above, which it assumes will be done by the later JS (which however doesn't run under NoScript). Using <small> just makes it an even uglier unnecessary hack...
Crucially, it's not "beginning to sink in" at the Economist, since they don't have a faintest clue what they're talking about. If there's anything the past 3-4 months have taught us (or at least the smarter subset of "us"), one should be careful about predicting future state of exponential processes _even if_ one is expert in the field.
I'm not sure how anyone who's watched the exponential growth of a brand new domain can pick a point today to and say that things aren't as good as we expected. What may have happened was that some eager CEOs have overpromised on timelines and resources. But the revolution is coming, ML is already starting to change society.
We're building the tech. Right now. The author does not even realise the immeasurable potential that general purpose function approximators are having right now across all domains. The applications are not only limitless but they're being realized right now and it's insulting to have my work dismissed so shallowly by someone who speaks with such authority.
There's far more to do with ML and AI than self driving cars and shitty ad recommendations. It's just that the people who live at the intersection of mathematics, programming, and third domains are rare; give us some time. Even $16T in value by 2030 is 2-4 US GDPs from something which basically did not exist 5 years ago.
> There's far more to do with ML and AI than self driving cars and shitty ad recommendations.
Yeah, there's also shitty sentencing recommendations[1], new-age phrenology[2], and high-tech redlining[3].
I think your entire field needs to take a year off and take some ethics and philosophy courses before going any further. Otherwise we're all going to end up much worse off.
I agree that plenty of people have been too cavalier about slapping together some models, predicting something, and calling it a day. On the other hand, it's not like fair sentencing or fair loan recommendation is a solved problem for humans either. There is evidence that, when carefully designed, algorithms can produce more equitable outcomes than humans, for example when deciding who and how to release on bail [1].
So I think we don't want to hand over sentencing decisions to a neural network that nobody understands, but I also think careful use of (simpler) machine learning can still improve a lot of our decision-making. The question is how much these decisions improve over what we have now. There's not exactly a surfeit of wise, highly trained ethicists who are happy to make consequential decisions all day. Many human decision-makers are quite flawed too, they just get to hide behind the opacity of being a human rather than an algorithm.
To that end, a whole "fair machine learning" field has sprung up over the past few years to study this. There are like a dozen papers in the area at NeurIPS and ICML every year. There's some progress.
Besides, the options are not restricted to "a machine decides everything" and "a human decides everything".
We can use machines to guide the humans, finding flaws or biases, recommending further analysis, and in a lot of other ways while still keeping the decisions made by humans.
This is a bit like telling the whole CS community in the 90s that we need a year off the internet to study ethics because people can use it for child porn, money laundering, and cybercrime.
Parent's condescension drips a little heavy, but anyone building power tools for a species like ours ought to reflect a while on the potential consequences, imho
Yes. The lack of ethics requirements in CS curriculums is a terrible oversight.
Would an ethics course and professional society eliminate child porn online? Of course not. Would it have shaped and possibly tempered social media giants into something less toxic? Maybe.
Do you literally think these people are creating harmful technology because they have had a deficit of ethics courses?
I'm not convinced ethics course requirements do much for a moral living. (And a peculiarly large fraction of those curricula often seems to be concerned with agonizing over whether to push various small or large groups of people on to railway tracks to be run over, rather than trying to stop the frigging train)
The examples you link don't seem very difficult to assess if they're approached with high priority placed on not causing trouble for the innocent.
Also, presumably the people purchasing and deploying systems like that have usually taken plenty of formal education about ethics (and possibly philosophy), and yet there seems to be plenty of demand. Why is that you reckon?
You cannot imagine how many times I've heard this line over my career. What it actually means is "we do not yet understand the limitations of this technology".
If I had a dollar for every time I heard somebody talking about unlimited growth in the dot-com era, I'd have a lot more money than my options ended up being worth.
I completely agree with this. Let me add a non-US perspective that may surprise some people.
I think a big part of what is holding back many companies from making effective, genuine, real-world use of AI is that a significant majority of the individuals involved are bad at their jobs.
On the business side, there is an widespread unwillingness to acknowledge that technical people may be better placed to make decisions than businesspeople. In my view, this includes things that would traditionally be considered business decisions. The business is unwilling to transition from "working on the solution to the problem" to "working on the AI system that solves the problem" (assuming that they even understand what "the problem" is).
On the technology side, I see a massive oversupply of badly underqualified and generally ignorant "data scientists" and "ML engineers" who have just enough bootstrapped understanding and familiarity with the plethora of (absolutely fantastic) open-source tools to fake an entry into the area, but do not have the genuine depth of background that is needed to actually plan, design, and deliver a high quality solution.
Yes, I have met people without "credentialed" backgrounds who are very good indeed, and yes I have met forward-thinking business people, but the number of people who are in these categories is a tiny fraction of the number of people who think they are in these categories.
And that's why businesses are getting less keen on AI.
>On the business side, there is an widespread unwillingness to acknowledge that technical people may be better placed to make decisions than businesspeople.
Maybe acknwledging this would mean losing their jobs. Buisness people have mortgages and children and they probably don't like the idea of having their job replaced by a robot. It is important to remember that the admin/money-side people are not playing the game that they tell the operations people they are playing. They want to keep their jobs, accrue power and money, they don't really care about AI or sales, those are just tools for their own game.
They said technical people, not robots. Perhaps a subtle distinction ;)
Agree totally with the latter part though. In vast swathes of the business world, technology is not perceived as a way to gain a competitive advantage but rather something between an interesting novelty and an annoyance. The problem is there's a cultural expectation that you be "innovative" which tends to be treated as a synonym for deployment of computer-related technology upgrades (although of course it doesn't have to mean this).
But to people who have no intrinsic interest in technology, there's no real way for them to actually innovate. They just don't know where to start. So they latch on to trends they read about in the Economist or NYT on a truly massive scale. They spend days or weeks making PowerPoints about the transformative potential of blockchain, IoT or AI. They take a few programmers who are kicking around but seem bored and allocate them to an "innovation lab" where they putter around making prototypes and having fun but never impacting the business in any way.
This ticks the box labelled "we are innovative" without requiring anyone to think too hard, learn anything new or take any risks, all things that the generic unskilled graduate managerial class in our society hate doing. And of course technology upgrades are risky. When run by non-technical people they tend to go wrong in spectacularly expensive and mysterious ways, so a lot of business executives would rather pull their own teeth out than plan a major IT upgrade. This is partly why tech firms exist as a concept.
As for AI, you can't replace most of these jobs with AI because their outputs are undefined to begin with. And to be frank, my observations over the years has been that a lot of these jobs appear to be undefined in order to enable a form of cultural stuffing. If you're a non-technical executive you wouldn't want your nice business to end up filled with geeks playing board games and aggressively proving other people wrong now, would you? Better keep them diluted by hiring lots of PMs, "operations specialists", "customer analysts", "business insight teams" and so on. That way you can meet your diversity numbers and be surrounded by like-minded people.
Your perspective is not just 'non-us'. The thing is since the early 80s computers can generate huge swaths of data. ML gives you a way to filter that data in a particular way. The same was true of data warehouses, smart systems, etc. The issue is not the business people vs technical people. It is an understanding of what do you want it to do. A few years ago I had a system that could generate 2k in data samples every few seconds (switches, voltages, temps, etc). What are you even looking for in that pile of stuff? You can not just feed that into a ML network and hope for the best. You have to describe what you are looking for. I had this same conversation over and over when working with data warehouse projects. A good BA on a project like that is amazing. Someone who is just kinda meh on it will kill the project dead. I do not understand your business, you do. I can apply what I have learned from other companies but only to a point. After that point I basically have to become a BA in that company just to understand what to write.
I think you've identified the basic, classic difference between artists/thinkers/scientists and everyone else: just as a musician can be given a piano and in time they create complex compositions, those that can see and operate in abstract conceptual spaces will see and use AI to create wonderfully complex compositions everyone else will be slackjawed in awe. This is human nature. AI is not an intelligence player, it's an instrument to find and tune like steroids one's own intelligence. Just like a piano, AI is useless to most people, because they do not have the artists/thinkers/scientists drive and comfort with abstract problems that are easier with even the most basic AI.
Your argument boils down to “AI won’t be useful unless humans are twice as smart as they are” (your examples of the businesspeople and researchers), and thus doesn’t really say anything.
I am an underqualified and generally ignorant data scientist.
I'm really good at understanding technology and talking to people though, so all the true experts who I know enough to talk shop with love me for keeping management off their backs.
Well of course you say that, you're an ML researcher who likely went into the field expecting further steady progress comparable to what we saw between 2012 and 2016. If actual progress in the field would be slower than what is currently still expected by the majority of people this would have dramatic consequences for future research investment, which is why you see most ML researchers reinforcing the hype or at least not talking the hype down.
While the accomplishments in the last 8 years have been impressive and applications of those techniques have and will continue to have impact in the real world I think that there are extremely big obstacles in the way towards something that would be truly transformative and which would put a lot of people out of work, which the $20 trillion industry estimates assume. Unfortunately I have seen no evidence that there are sufficiently good ideas in the field to bypass the upcoming roadblocks.
I think a major problem in judging the advancement of AI is that its successes are not general purpose -- it works great for narrow tasks and not well for broad tasks that we humans do every day, especially those that require what we call "understanding" (as in modeling and employing semantics or relationships among objects like causal inference).
Deep learning's big success is its ability to attach a label to a complex signal -- an image or a sound. That's pattern recognition: speech in, speech out, and image recognition. These tasks were largely beyond computers until 2012 but something that a child or a squirrel does very well, so it's been long known that skills like these are not useful signs of intelligence or rising levels of cognition.
Other tasks DL does well are those that benefit from memorization of results form death-by-search and from vast amounts of RL simulation, like playing of board games and the transforming of patterns (images and sounds) in fun ways using GANs. But ever since Deep Blue's win over Kasparov using clever pattern matching of past chess games (by memorization), we've known that narrow skills like game play also are not useful signs of intelligence.
Yet pattern matching skills are 99% of what deep learning hath wrought. Yes, that's useful, but it's not really intelligent. It shows no signs of thinking / cognition aside from probabilistic association / clustering. So there's no reason to imagine that techniques like deep nets will take us all the way to thinking like a human. Today, because of DL, we're much better at pattern matching. But in terms of what's essential to cognition, pattern matching achieves surprisingly little if your real goal is to THINK.
As far as "thinking" tasks go, cognitive tasks like machine interpretation of intent in written text and machine translation between languages still suck, despite the impressive advances in semantic-surface associators like BERT and newer transformer-based NLP engines. To do more than answer basic questions about nouns and verbs, you need a model for deeper semantics and an understanding of logic and relations between actors. Until deep nets can model semantics that are not present in the test data, and employ logical inference, it can't be said to think, much less intelligently.
> it's insulting to have my work dismissed so shallowly
You haven't had to deal with end-users much have you? I've seen years of really excellent work (not my own) blithely dismissed with expressions such as "but what does it now?" or "why can't it do X [impossible thing]?"
People have been told AI is potentially going to take over from the human race. Unless that's in your pipeline, they're going to basically treat your work as eaten bread.
Which is precisely how it's supposed to be. Nobody is required to respect your work in progress. The world will either care or...care not.
Your comment is a little hand-wavy and strongly worded ("revolution", "immeasurable", "limitless", "all domains").
Many things have exponential growth - bacterial reproduction, compound interest, certain chemical reactions. It's important to understand that this does not automatically result in miraculous universal transformation, but must be considered in the context of the world we live in. A little humility is always in order.
Your analytical thinking misses the human element. To give an example: Steve Jobs didn't have humility, nor did he shy away from hyperbole. You would have had the same response to his words, but look at what he - literally, in the physical world - achieved. Human motivation - no matter how deluded - is not actually something not to dismiss. It has effects on others, which translates to action. The inputs are not all machines. At least not yet.
While not wanting to downplay the marketing and design genius of Steve Jobs who created a brand identity so strong that for a while, "smartphone" became synonymous with "Apple", Steve Jobs really stood on the shoulders of giants in terms of innovation.
Without the Alan Turings, John von Neumanns, Edgar Dijkstras, Donald Knuths, the researchers at XPARC and Bell Labs, etc., there would never have been a Steve Jobs. He does play a role, but not nearly one as big as all those people that came before him.
Some people like to push for progress for progress's sake; others mostly to make money out of it (not saying this is bad!). The latter get more rich but it doesn't mean they're more important.
> I'm not sure how anyone who's watched the exponential growth of a brand new domain can pick a point today to and say that things aren't as good as we expected.
They do worse than this; one of the articles in the section uses a bad Google translation in Yoruba, from 2018, and uses that to claim it “is therefore still baffled by the sorts of questions a toddler would find trivial”. I'm not sure Yoruba even used neural nets at that time!—‘massively multilingual’ training only really came about in 2019, and was productiviced June this year.
A disappointing show from The Economist, given how little effort it would have taken to check with the newest model.
It already has—look at how much companies try to shunt you through a robot before you speak to a human despite their being mostly useless compared to humans. It’s clear this revolution we both see happening will come with horribly kafkaesque failures that will take 30 years to “discover” so we in the tech industry can “learn our lesson”.
Hell you can already see this with the use of fraud detection rather than implementing basic secure transactions on credit cards. The direction we’re headed in is clearer than ever: technology does not lead to better products but rather larger scale consumption of lower quality.
There's value, the tech works, but applying it is surprisingly hard.
I met someone who dedicates their life to using machine learning to replace/aid/automate pathologists 6+ hour days searching for cancer tumors in lungs. They have been at it for 5 years. There is an insane amount of approvals, red tape, knowing the right people, convincing the hospital to use it - all tasks not related to the tech actually working. Building it is the easy part. They were on the way there, working in the research section of a hospital. But they are still more than 5 years away from me being able to walk into a hospital and get my lungs scanned for tumours.
I personally can't wait for these general purpose function approximators to make all our lives better.
It's not that the technology isn't there. It's not that the technology can't be technically applied successfully. It's that convincing systems of people to change is way harder than we might think.
I am a practicing pathologist and I have seen many attempts and publications to use ML in pathology, which all lack in these aspects: 1. ML is trained on simplified sets (preselected ROIs, limited choice of diagnoses), 2. ML is biased by the experts who labeled learning sets, 3. there is no obvious process of learning from failures after initial training, 4. who is responsible in case of ML error with substantial consequences for patient?
The first point is especially for the lack of better word.. wishful. In the daily practice we are used to account for "things unexpected" - non-representative biopsies, parasites in tissue where tumor was suspected, foreign body reaction from previous operations, laboratory accidents (such as swapped paraffin blocks of two patients), and so on (the list is much longer). We deal with it. That ML can discern between 5 most common diagnoses is fine, but it is rather narrow problem to solve.
Thanks for the very insightful response, you're right. All those problems are barriers to using the ML practically. So it seems like the technology actually is the problem.
There is a strong chance that the technology to create digital beings will be available before human society is able to integrate and adjust to the current generation of AI.
So what may happen is that the way that AI really gets integrated is by actually replacing human beings who largely die off.
That's a really interesting perspective and a good point. Reminds of "Age of Em". Interested to see what the effects would be of digit beings before integrating current generation of AI would be.
Ok, consider me intrigued. What is that we are going to see/experience once you folks had some time? Genuine question! Can you give us a basic idea of the things that you are already sure by now will see the light of day?
"Hi AlphaGo, let's play tic tac toe". Consider me bored of it all. We're not close to anything. The emphasis is on "artificial", not "intelligence". It should be renamed Glorified Calculation. There are some neat tricks, writing your first neural net is great fun, but we're using the word intelligence with spectacular liberty. I'll keep an eye open for something inspiring, but I've seen nothing of the sort yet.
Proof of AGI progress would be a significant enough achievement (given there hasn't been any for decades) that it wouldn't require looking hard - but please share any links to it you have.
Nobody else seems able to, especially the AI ("more power and money needed! we're just around the corner!") blowhards.
Mathematical modeling that is 3-6 orders of magnitude faster, we are already talking deployment.
Same for ML powered solutions to data management - I don't want to say enough to identify anything.
My team has been working on a rudimentary humanlike reasoning engine based loosely on what AlphaGo proved: that machines can learn heuristics identical, equal to, or better than those of humans.
And for perspective, AlphaGo was what...4 years ago? Just look at the growth of compute available cheaply in the cloud since then! These next few decades are going going to see a phenomenal acceleration that's already starting...assuming we don't hit some unforseen limit or block. But even with what we've discovered now, we have more than enough in the way of new tools to refine across industry and society for decades.
> And for perspective, AlphaGo was what...4 years ago? Just look at the growth of compute available cheaply in the cloud since then
How would we know the difference between a problem that was interesting to solve but was hard versus a problem that just needed Google's money to solve?
Besides, since when was cloud compute was getting cheaper? I'm pretty sure, considering how increasingly profitable it is despite the "discounts," cloud compute has gotten more expensive, in the sense of increasing margins for the seller.
I guess we'll all find out what OpenAI does with its Azure credits.
mtgp1000 says>"mathematical modeling that is 3-6 orders of magnitude faster, we are already talking deployment."<
How seriously can we take that phrase? Please mathematically model the Covid-19 pandemic's next 6 months. The epidemiology community dropped the ball and you can hardly do worse. Take the "Covid Challenge!"
mtgp1000 says>"These next few decades are going going to see a phenomenal acceleration that's already starting ...assuming we don't hit some unforseen[sic] limit or block."< -
Nobody can predict next year's economy and certainly nothing beyond 5 years! You're predicting for the "next few decades"?
mtgp1000 says*>"But even with what we've discovered now, we have more than enough in the way of new tools to refine across industry and society for decades."
since you’re working on the field : i’m still under the impression that modern ML (neural network in particular) are not producing any science, nor are they producing any level of understanding of the phenomenon it’s trying to model. And that as such, we can’t provide a reliable estimate of the limitations of the produced model, other than by feeding it tons of inputs and measure the result.
It would be like trying to understand gravity by throwing thousands of rocks, measuring the motions, and build an approximation on a huge matrix, without ever being able to get the « real » netwonian formula.
Some of this would be considered a feature by companies. It's more defensible to have unknowable AI deciding to do illegal things than programmers hard coding illegal things. Which really boggles my mind. When my kid does something illegal I'm held liable. When an ML algorithm programmed by a team of people does, nothing we can do about that!
I'm not sure if the courts will see it this way. Anyway, in some fields some stuff (e.g. credit scoring in banka in my country) is regulated and it's the humans who have to make the final decisions - algoriths can only provide an input. In practice, the input is "approve/deny", but it's still a human who is making a "decision" based on this "input".
I don't see this as really a unique situation or one where people are exempt from liability. If someone starts a fire, they are held liable; that's analogous.
On the other hand, the difference between a team of people and a particular person with mens rea, that's not unique to AI either.
What makes you think Newtonian physics is in any form more real than computation? If Wolfram is correct, physics is inherently a computation in the form of cellular automata.
Intelligence might be so messy that the only way to get to it is to grow it organically, which to me seems to be happening in research. People are trying a bunch of stuff based on intuitions for what may work, and post hoc adding some theoretical justification for why it's working. Human intelligence didn't emerge from some grand theory of intelligence either.
Your training set must be something unique to make those claims - very intrigued to know if you chose another closed system environment (I.e. any structured game with predefined scoring), or an open system with carefully selected labeling with multiple “scores.”
Also complete novice so be merciful if I’m off base.
This has been on my watchlist for a while. The problem is the inablity of these techniques to scale to larger problems. That being said, once that happens, it could be revolutionary.
DARPA threw some money into the problem, and stuff advanced from toy problems to things that are still small but often useful.
See e.g. pyro.ai is already working on non-trivial datasets.
Ultimately, I'm convinced it's a hard compilers / static analysis / abstract interpretation problem. That requires lots of resources and a decade or two of work.
One big one is farming, think harvesting machinery. There's a number of startups trying to get that going. I think most are not at the scale to be successful the way the market themselves, but their collective learning will eventually lead to some consolidation.
Why AI for that, it's vision AI to know when fruit is ripe or vegetables are ready for harvest. Then hand eye coordination to not bruise the fruit/vegetables and adjust force/sensitivity as the equipment wears down.
Scaling that will put likely many workers out and transform the business of harvesting. What I think is most plausible is one large manufacturer will consolidate designs and rent the equipment to do the harvesting much like the contract labor used now, but at much lower prices than humans could do.
Depending on the crop you’re talking about, we are still very very far away from this.
People have been trying to design machines to pick fruit, even in toy scenarios, for decades, and we have little to show for it. I don’t think that image processing is the bottleneck here, it’s literally the mechanism as far as I understand.
I’m skeptical that any mechanical equipment will ever be able to do these tasks as cheaply, quickly, and efficiently as a human in our lifetimes.
> What I think is most plausible is one large manufacturer will consolidate designs and rent the equipment to do the harvesting
No way. Everybody wants the machines at the same time. You're going to incur the capital cost of buying/manufacturing the machines, the cost to store and maintain them for 50 weeks out of the year, and then rent them out at a profit for 2 weeks? No, the farmers will just keep hiring human labor.
If you can solve this problem, you can sell a lot of machines for a lot of money for a short few years. After that, they'll be a commodity and be dominated by all the other capital-intensive manufacturers that can run a factory better than you can.
>> The applications are not only limitless but they're being realized right now and it's insulting to have my work dismissed so shallowly by someone who speaks with such authority.
What exactly is your work? In research, including AI research, it is ordinary and expected for one's work to be subject to the most ruthless criticism. This is as it should be: only work that survives all criticism can really be expected to last.
If you are an AI researcher, you should be used to that by now and not take it personally.
Do you see any irony in your comment complaining about CEOs over promising and then you go on to describe “immeasurable” gains and “limitless” technology?
Are there examples in history where things requiring great time and effort were built then abandoned?
In recent times, due to a relative dearth of high yield investment opportunities, there is a lot of money with nowhere to go. Some believe much of it is "dumb" money. Overfunded startups are one possible symptom. Heavily-funded "AI" may be another.
This is before we even consider the moral and ethical issues of using "AI" as a substitute for human judgment. It may "work" but that in itself may not be an adequate justification for its use.
> Are there examples in history where things requiring great time and effort were built then abandoned?
The communist countries seen a huge misallocation of funds into giant projects that didn't make sense (e.g. huge steel mills in Poland that were always losing money).
Absolutely. To be perfectly honest, it surprises me the extent to which ML naysaying seems to be popular on HN. The evidence of enormous progress seems pretty obvious to me.
Siri still isn’t able to understand « do NOT set the alarm to 3pm », and many image classifier produces aberrations that no human would ever commit.
Many people feel that ML has so far only produced « tricks », but still doesn’t show any sign of « understanding » anything. As in, provide meaning. It may be unfair, but i think that at this point people would be more impressed by a program « smiling » at a good joke than by something able to process millions of a positions per second, or « learn » good moves by playing billions of games against itself.
Sure, ML is not at human-level intelligence - it is very much a tool-AI where we give a task to the machine and let it get very good at that.
Nonetheless, it seems like progress on that front has been made incredibly quickly. Sure, Siri might not always "understand" what you're asking but the ML is able to very accurately transcribe your voice to text, something not really possible a few decades ago. Image classifiers produce misclassifications, but so do humans - it is a very alien technology to us, so we view some of the machine's misclassifications as absurd, but at the same time the machine might view some of our misclassifications as obviously off-base as well.
None of what you said seems to explain the consistent negativity on HN to what has really been a transformative technology in a lot of ways.
Progress in speech recognition has been slow and incremental, not fast and impressive. Hidden Markov Models did a decent job in the 90s. Now we have more data and more computing power.
Both voice recognition had first working research machines in the 1950s. Of course present models are way better than these, but fundamentally these are the same as those in the '50s, "just" with tremendously better hardware and algorithms. But there is zero intelligence in these, the models have no internal concept of language or the world around them.
This will certainly produce many great specific solutions for specific problems, but generic problems like driving a car are most probably not solvable without an internal world model.
I'm one of the naysayers and have a background in educational science, I'm constantly baffled why AI research seemingly never looks at theories of human learning (though it may very well be that I haven't looked closely enough!). What essentially all AI approaches do is modelling associative learning (https://en.wikipedia.org/wiki/Learning#Associative_learning), but employing more and more processing power for training. This is akin to having a fruit fly brain and copy-pasting that over and over again, in the hope somehow a higher order will organize from that by itself. But most things humans reason about are not learned this way, but rather by infering meaning from things and situations, i.e. one learns how a steering wheel works not through hundreds and hundreds of trial and error cases, but through single Aha! moments in which ad-hoc generated mental models (concepts) are validated in the environment. And GPT-3 has no knowledge organized in a hierarchical system of concepts, just as ELIZA didn't have one.
There are tons of research projects that do exactly what you're talking about, initializing an agent with zero knowledge in an environment which just provide rules of discovery and reward. The agent then takes actions in that space and learns from its own experiences. It's just that it is hard to compare with humans, who have had the benefit of evolution over millions of years. A basic example is that of AlphaGo Zero[1], which learns to play a top level game by being given enough time and just the rules of the game. This is similar to how a child learns to walk, it is just that we only have the capability to model toy situations (a board game in this instance) right now, and access to harder instances (movement in the real world) will slowly come about. There are cases of robots being programmed to poke / move / pick objects to try and learn about their shapes[2], in case you are interested in another such example.
AlphaGo Zero definitely falls in the same category as GPT-3 - yes, this is unsupervised learning, but it still is fundamentally the same approach, exactly because of the way discovery and reward work on the model.
This isn't how a child learns to walk at all: A child a priori has the concept of walking, the concept of self, the concept of movement in space, the concept of willing to walk etc. - it just doesn't have the motoric control. The small part of training motoric control through repeated trial and error is indeed similar to what unsupervised learning models, but the important part is missing.
While I can't speak for Siri, Google's voice recognition continues to surprise and even frighten me. And if you go online and look up deepfakes or AI generated voice of famous people, it also is freakishly accurate.
Sure, there are some issues and most of them still fall somewhere in the uncanny valley.. but we are just starting to fully exploit this technology. AI that people don't tangibly see but make a giant impact in their lives is for example the Facebook algorithms that decide what shows up on your feed.
When a large subset of the population gets majority of their news from Facbeook, Facebook has effectively created a mechanical system that decides what a large chunk of the population is aware of. I don't think people fully realize what effect that has had on our entire society, including elections. The future will only see more of this type of thing.
I'm always disappointed with Google Asistant. When I say "navigate back home" it tries to route me to a company called "Home". Great job. I've included a hint "back". I've never been to a company called "Home". I'm after work. This is a fun example, but I never get good results. I say "weather" and it shows me the weather, just in a different city. FFS. Maybe in the next 5 years I'll be finally able to do something using voice commands. Right now I just want to smash my phone.
a) Good general-purpose automatic speech recognition, which was an unattainable holy grail for DECADES. Previously, you would have to record a lot of your own voice to train the system for you in particular before it was at all reliable. Now it basically just works.
b) Making good use of the results of (a). Your examples are clearly in this category. This is ALSO a very hard problem; voice interfaces are basically brand new creatures, and I expect we'll be seeing 'best practices' form up for a while yet.
Fair enough. Usually when I use "Navigate to the nearest gas station" or "navigate home" or "set an alarm for 8am on Tuesday" it all works perfectly for me, but I concede I don't use the feature that often so perhaps I've just been using it for well-adjusted commands.
I recently watched a podcast with Stephen Wolfram (few months old) where he runs a photo of himself through Wolfram Alpha "ImageIdentify" and it classifies him as a plunger with 50% probability and 8% probability it's a human [1]. He then goes on saying how he's working on systems that would allow a self driving system decide what to do based on if the collision detected is a human or inanimate object, etc.
This was just one particularly ironic example of what I'm seeing with ML - we saw this level of accuracy in classification years ago - where is the enormous progress ? There have been pockets where ML shows it's applicable with encouraging initial results but after that it seems to me that we suck at perfecting those system and the progress has been incremental at best.
The only exception I can see is stuff like self-play where the system can generate insane amount of training data.
I work in “making the sausage” - deep learning techniques have already completely transformed my field and are transforming many others. Most of the people I see naysaying don’t appear to causally work in the field, because their critiques are not the same as the ones people in the field are using.
Can you give a concrete example of deep-learning techniques which have transformed a field? Something not theoretical or confined in a lab? Something which may have changed my life?
Not OP but I can speak with a small example from my previous field - coral reef ecology. It used to be standard to collect video footage and hand annotate small portions of the footage to draw up some statistics about the reef progression. Today we can collect the footage and let ml annotate the entirety of it to get much more accurate and precise statistics about the progression/recession. Not only that but we can do that on an organism basis and also see do highly local analyses. This wasn’t possible 10 years ago. I think the applications for ML outside of consumer tech are far larger than within but everyone here seems focused on that for some reason.
I think ML naysaying stems mostly from extravagant claims of pop-sci articles around papers or of the authors themselves. It is always good to have some amount of people that point out that the models aren't some cyborg superheroes or has to be a step towards AGI but be more practical.
> It seems a lot of applications boil down to some type of discriminator or pattern matcher
This doesn't seem to imply that no learning is going on or that there is some sort of simple process. If I ask you to say whether something is a cat or a dog, you are essentially functioning as a discriminator there - but there is still a lot of process that goes on behind that.
I work on a production ML platform, so I spend way too much time rabbit-holing on interesting looking projects.
If you're looking for interesting startups/projects-not-from-big-tech:
- Glisten.ai (https://www.glisten.ai/). Recent YC startup, uses a combination of different models to parse product information (actually a huge manual problem in retail/ecommerce) and expose it as an api.
- Wildlife Protection Solutions - Recently deployed a model that can automatically detect poachers in nature preserves. Detects twice as many poachers as previous monitoring solutions.
- Ezra.ai - Uses models to search MRIs for cancers, operational in a few different US cities.
- AI Dungeon - A text adventure game built on GPT-2 (now GPT-3). Super fun, if a little silly.
Now, those are just a handful of smaller companies whose core products are ML. There are a ton of financial institutions using ML for fraud detection, real estate platforms like Reonomy that use a variety of models for evaluating investments, and security companies using ML.
But of course, the obvious answer to this question is "Every popular app you use incorporates ML."
Facebook/Instagram/Snap/image apps: A variety of models for recognizing faces, object tracking, etc.
People have this weird "Skynet or it's snake oil" paradigm they use to evaluate ML, ignoring the fact that production machine learning is more or less ubiquitous at this point.
So THAT's why the ETA given is always too short! If it's based on how often a typical driver makes it, and typical driver is a speeding asshole, then no wonder that the estimates are unrealistic for someone who actually drives under the speed limit. It's a shame that Google is actually normalizing assholiness.
What's wrong with speeding? People generally drive at a speed such that they're taking an appropriate level of risk. The speed limits usually set these risk limits too low. Moreover, the job of google's ETA is to be accurate for the most number of people. If you're in the minority, then too bad.
Moreover, I'm not sure how speeding makes you an asshole.
By speeding, you increase the risk of death/injury of other people, you increase their stress level and you also increase road noise. Moreover, you unilaterally (and self-servingly) decide that the norms established by authorities are stupid and that you know better. This all sounds like a description of an asshole to me.
I'm not much into machine learning myself, but I must admit that ML has been invaluable for good voice recognition and good voice synthesis. Writing programs to interpret speech accurately and synthesizing a "natural" sounding voice by hand is practically impossible, but the perfect application for ML.
Got bad news for you man. 16T in value doesn't come from ML. It comes from inflation and printing money out of thin air. The world has limited resources at the end of the day. When one industry does well, it's at the cost of a different industry.
Perhaps basing our decisions on general purpose function approximators (whether forced or voluntarily) has not served us so well, ultimately. I think they play an important role in causing, or at least enabling, much of the trouble that we live in nowadays.
Is it not better to engage with the critics and correct their perception in that case? I'm sure the author would not mind a direct response that outlines where they went wrong and why.
It sounds like some buzzword speak. Most things that are heralded as ML are nothing but data science idiots from python schools applying some basic math transformations and overselling them.
Nothing wrong with python or JS. But there are many things wrong with bootcamp / workshop people who come out just learning Python with some ML library or JS with some web framework. They don't know even basics of CS and arguably produce bad, inefficient code.
There is a famous word for latter - 'webshit'. The former doesn't have such a title. But there is an obscure one originating from a Slashdot comment IIRC something like:
'you are one of those data science idiots from python schools eh?'
Edit: I didn't mean disrespect to those doing meaningful research. But that's not what most of things sold as 'ML' are.
Can you be a little bit more concrete about the tech you are building right now that is not possible to build using classic ML approaches? You see, I am actually going to work thinking about how should we reduce the model size and still keep it within reasonable accuracy bounds so it would not be worse than a rule-based mixed with classic ML system that we use. And here I learn about immesuarable potential, insults and billions of dollars.
I think the problem is that in a long list of edge cases, it's neither fake clever nor real clever but just "dumb". This is, of course, always an issue with computers, but what makes ML so vexing is that it's inscrutable.
ML works by taking a huge amount of data and finding patterns and is pretty much incapable of judging individual cases. We have ugly words for humans who judge people based on generalisations instead of judging the individual...
Computers and algorithms are devoid of empathy and humanity, I am extremely apprehensive in using them to make any sort of judgement call, especially if the calculation is inscrutable. Humans may not be perfect, but at least they're ... human.
ML is an interesting tool which can be used for many things, but right now it's also being misapplied for many things. I think a lot of opposition comes from that, rather than your fridge or whatnot.
A scarily large amount of human speech is essentially word prediction, especially in cases where someone wants to seem impressive without having actually done the work. We're all familiar with the problem of people "bullshitting" about technical topics, especially in the business realm where there are lots of people with no technical background who want to learn and have authoritative opinions on technology, like in this article. The GPT models are quite capable of producing this kind of "sounds legit" speech. And if that isn't a form of intelligence, then what is?
We've been able to produce that kind of speech before though. I wrote programs using Markov Chains 10 years ago which could produce almost real sounding speech but I didn't think it was intelligent.
The GPT models are essentially massive databases along with the use of probability models for how words are connected. Do you think that means intelligence? Can it generate new knowledge of things outside its database? Can it generate novel theories for things outside its understanding?
The GPT models often generate streams of text which can found word for word in its database, i.e. it's not really generating anything new as just mimicking. When a parrot repeats a human, it doesn't understand what the words mean, it's just making sounds.
But at some level our own brains are just very large databases and very good prediction engines, at least when running on auto-pilot and lacking any deep understanding. Hence the phenomenon of BS artists. Sad to say but I've heard speeches that were less coherent and interesting than what GPT-3 produces.
Where in the GPT models is the conscious (or even subconscious) thinking, comprehension, understanding, relating to experience, imagination etc, etc, you know: actual intelligence as a human would feel and use it, general or otherwise?
Tic Tac Toe, Chess, Image Classifying, Translation, Go, Sentence Construction, etc.; We are creating phenomenally impressive calculating worm-equivalents, but nothing human-level, or even remotely so.
Where in the human mind are these things? We judge that they exist based primarily on what people say and do. GPT-style models can't do anything, but they can say. And for better or worse a whole lot of ordinary, everyday speech doesn't have a whole lot of thinking, comprehension or imagination to it. It's just people saying what other people seem to be saying in order to try and stay with the herd. GPT-3 is scarily close to matching that.
It's pretty fascinating that nobody (AFAIK) really predicted that, except perhaps Isaac Asimov. Sci-fi normally presents AI as either superintelligent or non-existent (just basic voice commands). I don't think I've seen one which presents it as an AI ecology of idiocracy.
It's even hard to say how "genuine" cleverness in human beings works - and we've had a lot more time to study ourselves - granted that most of that time we've not had the tools to understand how our brains work at the algorithmic level. I am not claiming that achieving some version of "genuine" AGI necessarily involves understanding how human intelligence works, but it is reasonable to expect that knowing more about it will definitely reduce the search space of possible systems.
Is it reasonable? What would you learn about wheels by studying cheetahs? Or about steam engines by studying strong elephants? Or about cranes by studying giraffe? Or about jet engines by studying birds? What if AI is like that to a human brain? Completely different and way better in some dimensions?
>There has been next to ZERO progress towards genuine AGI
I mean, really? In the most pessimistic evaluation of ML research, we still know which techniques and paradigms won't work for AGI. That's not zero progress. Nobody is expecting this to happen overnight.
My comment was in the context of yet another completely unsourced claim of progress.
No evidence - as always. No progress.
Of course, if you - or anybody else - can show evidence of a genuine advance towards AGI, rather than breathless hype and vague assertions based on the impressive but irrelevant narrow-domain "AI" expertise of Alpha-x and the like, I (and many others) would love to read it.
I would say the only tangible evidence of a "genuine advance" towards AGI would be the growing amount of computational power available to us every year.
Perhaps we do, but if we don't, then the continued lack of progress is in fact evidence of our ineptitude and inefficiency in the face of an embarrassment of riches in terms of tools and power.
It's kind of like a having a searchlight that is more and more powerful and still not being able to find the object in the darkness we are looking for because we have given it to a blind man.
Plenty of practitioners ready to impress us with buzzwords, empty assurances and paper after paper on the brilliance of obscure statistical formulae and their money pit implementation in server farms.
But all this power and cliquey self congratulation is just noise drowning out a very embarrassing truth:
This is so strange. If you use facebook, google, netflix, apple, microsoft, amazon, tesla or a whole host of other products and services you are interfacing with AI all the time, sometimes as the core product of the service. To think there’s no value there is asinine. Comes up a lot on HN. Seems like people who get excited for these types of articles are set in their ways and don’t want to progress forward.
Yet, every "suggestion" is infinitely worse than "My List", that I curated for myself. Interface hiding the fact of dwindling library and quality of content. Chronology gets perverted with random posts that later can't be easily navigated or referenced. Menu-items unexpectedly fade in and out for unclear reasons and lack of anticipation. In many cases these improvements are to user's detriment and confusion, worsening interface, UX and value for money proposition. In too many areas, end user is themselves the target, for ads, manipulation, propaganda, manufactured outrage and lies.
Amazon retail is currently in the process of baking advanced deep learning models into every area of their retail process. Its all becoming automated. Just because its not talked about openly doesnt mean its not happening
Many AI systems are used in the backend to increase revenue. Netflix has a very complex recommendation algorithm based on deep learning/statistics. Amazon uses a lot of machine learning to optimize transportation (NP-hard problem!) , sales, etc..
A while ago I watched a 4-part documentary about the every-day lives of ancient Egyptians. Pretty interesting.[1]
For weeks after that my recommendations were filled with bullshit such as "PROOF ALIENS BUILT THE PYRAMIDS!" and such.
So yeah, maybe those "very complex recommendation algorithm based on deep learning/statistics" is perhaps not always such a great idea. In this particular case, it's just a mere annoyance for me, but imagine a 13-year old watching a few genuine documentary videos on Egypt and then seeing this bullshit; they don't have the capacity I have to see it's bullshit.
And imagine if it was on a more serious topic than who built the pyramids...
If I were to ask a YouTube engineer "why did I get this recommendation specifically?" then the answer would probably be "dunno".
An additional issue is that the YouTube of yesteryear was much better in browsing random videos. Now everything is based on what I've watched before, instead of just "give me a list of science videos" or whatnot. This is also an issue I have with Netflix (or rather, had, since I no longer have an account).
It seems to me that inscrutable mindless AI learning has a part in the spread of misinformation and bullshit. I'm not sure how large that part is, but I suspect it's significant. I'm hesitant of the total value in these cases, regardless of what it may do for the bottom line in terms of revenue.
My biggest issue with YouTube is that it shows me adverts for the same product for a about 2 months at a time and I therefore end up hating those products and would never buy them.
My tastes may be a bit hard to model but I don't get any value from Netflix's recommendation algorithm at all. I find new things on Netflix by reading human curated reviews online and then searching for titles by name.
Nlp is getting better at a rapid rate. Articles like this are worthless click bait. No one knows what ml will look like in ten years. And for the record, ten years is a short period of time for a technology that could have such a massive impact.
I think it's also a matter of expectation. People expecting conscious thinking machines with reasoning abilities are going to be disappointed but that doesn't mean AI as a field is useless and a big hoax. There are plenty of narrow problems where we can benefit from the use of AI. I am an MD and I see a lot of use cases of AI based technology in the future within the medical field though it needs more co-work between AI researchers and medical researchers. Good quality scientific validation of these systems needs to take place and that will take some time.
People don't understand what exponential improvement means. GPT-3 is a 175B with B parameter model. Another few rounds of doubling and we could be seeing models spit out short stories and novellas.
They "can't reason" but they can solve symbolic integration and differential equations (https://arxiv.org/pdf/2006.06462.pdf), beat us at all board games, and more recently get to be almost impossible to tell apart from human writing (GPT3).
That's still true. It can write a paragraph that's usually grammatical, and can stay on topic, but it's missing things like facts, or even the ability to remember which side of an argument it's taken previously.
In the near future, gradient descent optimization of simple targets like PROFIT and REVENUE - with even less accountability to negative externalities than human CEOs have today - is going to FUBAR literally everything
For example, years ago I was working on a prototype/proof-of-concept thing for instrumenting industrial machinery with stick-on small computers. Simple stuff - attach accelerators, temperature, humidity etc sensors to existing machines and collect the data and send it back.
The management thought we'd be able to apply machine learning on the data to get "business insights" from the all-powerful machine. They didn't know what these insights might be, just that ML/AI would generate them and therefore make the business a fuck-ton of money because AI generated novel new "business insights" that no one had thought of before and so transform the business. They thought it was just a magic box that would generate unbounded magic answers for their needs by passing in just some temperature and humidity readings or whatever, and then it would tell them they need to make more brown bread and less bagels in the North East region etc.
In reality, as I understand it, currently ML/AI requires us to know what the possible answers can be before we even begin training the network. So the classic example is it needs to know that the possible MNIST digits are 0-9, or that you are looking for one of 100 image classes etc.
You cant train a network with the MNIST digits, and then have that network tell you what shares to buy or sell.
Sure you can lop off the final layer and repurpose some of the middle layers, but you still need to train it to classify the inputs into categories you define up front. It won't give you a novel answer that you have not trained it for.
... at least that is how I understand it. Things may have changed over the past 5 years or so.
That said, I do agree there have been some cool things lately like machine vision etc. I don't think it will be that huge an industry though - it feels like a lot of it is largely just commoditised now (which is good) and it will be just like any other library you pick - like picking a UI framework for a web app. Just pick up a pre-trained network from modelzoo and get on with your real business requirements for 99% of people using ML, while the other 1% (at FAANGs et al) and academia churn out new models.