Some years ago, I worked on a team "Ads Human Eval" - we had raters hired to do A/B testing for ads. These evaluated questionaires carefuly crafted by our linguists, and then analyzed by the statisticians providing feedback to the (internal) group that wanted to know more about.
So the best experience was this internal event that we had, where the raters would say that certain Ad would not fare well (long term), while the initial metrics (automated) were showing the opposite (short temr). So then we'll gather into this event, and people would "debug" these and try to find where the differences are coming through.
Then we had to help another group, where ML failed miserably detecting ads that should've not been shown on specific media, and raters came to help giving the correct answers.
The one thing that I've learned is that humans are not going to be replaced any time soon by AI, and I've been telling my folks, friends or anyone (new-born luddities) - that automation is not going to fully replace us. We'll still be needed as teachers, evaluators, fixers, tweakers/hackers - e.g. someone saying - this is right, and this is not, this needs adjustment, etc. (to the machine, ai, etc.).
Maybe machines are going to take over us one day, but until then, I'm not worried...
(I've also understood I knew nothing about staticics, and how valuable linguists are when comes to forming clear, concise and non-confusing (no double meaning) questions)
I dont think most people are arguing that machines will replace everyone anytime soon - it is that they will replace a huge portion of people. If one person can do the job of 10,000 by being the tweaker / approver of an advanced AI that is still 9,999 jobs eliminated. That might be hyperbole (you still probably will need people to support that system)
I agree, but it's true that some jobs should not simply exists.
To this day, if I go to our airport in Sofia (Bulgaria), and my baggage is over the limit of 20 or was it 25kg I have to go to another place, pay for it and come back (why? bureaucracy - not only I have to do it, but I'm slowing anyone waiting for me to this - it's like 25-50 meters one place to the other)
Unlike Frakfurt, Munich or Heathrow airport where I can pay that fine right there.
The baggage weight limit has nothing to do with the weight of the aircraft. It is about keeping the bags light enough for handling, to protect the backs of baggage handlers (and automated equipment/belts). Unlike thier heavy bag, a heavy passenger is generally capable of walking themselves onto the plane.
If that's so, why is the luggage weight allowance applied cumulatively across multiple bags for one passenger? And why is it different between airlines and between ticket classes?
Because first class baggage is labelled and treated differently so it never gets bumped. The lower/higher numbers are also uses to force the penalty payments, something first class travellers don't want to deal with at check-in.
This seems unrelated to your original point about safety of baggage handling workers. The purpose of a max single bag weight is to facilitate handling. The purpose of a per passenger baggage allowance surely is to manage total plane load, hence high excess charges.
Even first class passengers can't turn up with unlimited luggage but, in my experience relating to families flying between London and the middle East, have household staff who make arrangements and coordinate with carriers.
Might be talking at cross purposes here: per bag 20kg is so the bag can be safely handled by staff.
When I fly between London and East Asia I shop around between different carriers and, in economy class, I'm normally offered up to two checked bags with a total weight of between 30 and 35kg depending on carrier and promotional offers.
Obviously you've never lived through having the weight balance of the baggage on your aircraft being deemed improper. 30 minutes to move to the other side of the airport, whereupon the weight balance is deemed OK, 30 minutes to return. Yes, Chicago.
There are reasons for it. Probably less so today now that most people just credit cards, but having one place that handles money solves some issues as opposed to having a bunch of places handling money.
While there are some jobs that can go the way of the buggy whip maker, this was probably not a good example.
Jobs don’t just exist to provide the best possible consumer experience. They are fundamental to individual feelings of self worth and societal stability. Unless you have a replacement for those things, endless automation may ultimately do much more harm than good - your inconvenience in Bulgaria not withstanding
You don't even that to go as high as 10,000. Imagine someone suddenly doing the job of 10 persons, that's entire teams being decimated. Go to a job board and imagine 9 of out 10 job postings not existing. How much harder it'll be to seek another job.
Unfortunately, the previously incorrect usage won and it doesn't mean that anymore, according to major dictionaries: "kill, destroy, or remove _a large proportion of_" is given as the first definition in the Oxford dictionary used by Google. ("Unfortunately" is, of course, for people who have the original meaning activated because of how the word looks and/or know the historical context and feel the tension with the contextual meaning.)
To add completion to this, decimation was a Roman Legion punishment that was really high up in terms of severity.
The Centuria was a roman military unit of 100 men (The centuria size/meaning actually varied over time) but when the decimation punishment was applied that would mean that (by draw) every 10th men in the centuria would be killed, here is the catch, the people of their own centuria had to kill their own mates in decimation.
People today have jobs, but what we don't hear about are the people who were made redundant and the suffering they went through. They're just forgotten about, or even ridiculed, like the Luddites are today. For the Luddites who were not shot by the state or factory owners, or were tried and literally executed by the state for machine breaking, the rest of them and their families lived and died in utter destitution having formerly lived comfortable lives as skilled laborers and small business owners themselves.
Post-2008 did that to a lot of workers, as well, causing some of them to leave the workforce entirely. I know more people than I'd like to who are over 60 now, but were made redundant in the wake of the financial crisis and economic restructuring. They never worked in their fields again, some of them were forced to retire and others are working dead end jobs for $10 an hour despite having been middle to upper middle class a decade or so prior.
Sure, maybe their kids or their grandkids might be able to dig themselves out of the hole their parents were forced to drag them down into, but that doesn't do much for people who lost their livelihoods and have died/will die in destitution.
>but were made redundant in the wake of the financial crisis and economic restructuring.
The recession didn't make anyone redundant. The people that were laid off were laid off because the people hiring them ran out of money to pay all of their employees. Yes, it is true "many people are laid off because they are redundant", but in the wake of the recession, companies ran out of money and they had to stop doing things that cost money, and then lay off people that did those things.
Also, to be clear, automating work didn't cause those people to lose their jobs - because their jobs were stopped.
Also, I'm very sorry for all of the people that were impacted by that recession. It was big amd a lot of people hurt because of it.
> it is that they will replace a huge portion of people
And not just any people. Lower to middle class, blue collar people.
The type who are the least able to travel for work, most likely to have families, least able to transition between careers etc. And crucially in many countries the people who often decide elections.
Yes, in the future, while this job goes away now. The problem is likely the interval between "now" and "then", when suddenly this mine closed and you have thousands of miners out of work and unprepared to do anything else. The kids will probably learn to do a new job in a new industry, while their pops will stay at home drinking or go out to kick some butt (or both) - neither outcome very pleasant for the society.
If you wonder why we still need humans, check this out - https://www.google.com/search?q=beverly+hills+properties - spotted the issue? maybe an AI could've caught it... or maybe lots of amazon-turk (or like) employed folks that give feedback on things...
While not exactly aligned to the research, I've been surprised how poor Nest Thermostat's learning feature is.
The main selling point for Nest is having a "learning thermostat". Perhaps my schedule is just not predictable enough, but the auto-generated temperature schedules it generates after its "learning" period is not even close to what I would manually set up on a normal thermostat.
Maybe I'm just an "edge case" or part of the "long tail"
"Why am I sweating right now? Oh, the Nest set the temperature too high again!"
And then after a few instances, I just turn off all the automation and set up a schedule like normal.
Same with the "away from home" which seems to randomly think I'm away and I have no idea why.
Oh, and the app doesn't show me filter reminders, only the actual device, which I never touch all the way downstairs. There's not even any status to let me know if it's accepted a new dialed-in temperature, as I've had it fail to capture a request, and then I go back, and see it never updated/saved the new temp. Just zero feedback to confirm that the thermostat has responded to any input, and zero notification from the app if this happens.
Just thoroughly unimpressed.
Thankfully I didn't buy this junk, as it was pre-installed by the owner of my rental. Can't imagine actually paying for something that's only real feature is being able to remotely control my temperature once in a while.
I've always heard this, and so when I went for my first smart thermometer I went straight to Ecobee (which I'm very happy with btw).
So I gotta ask HN...what the heck was so popular about Nests?! It's one thing to be go after shiny lures like new iPhone apps or luxury items...but a Thermostat?!
It looks good on the wall, has a bright large display that lights up when you approach and intuitive enough for non-techies to operate. Also it can be installed without a common wire.
The industrial design of the thing is really nice. I wish someone could/would copy them. I don't use any of the smart features nor do I connect it to the app, but it's still the nicest looking thermostat out there.
The problem I think with all of these ML functions is that there is never enough of an in between full manual and full auto magic. The Nest could simply ask you a few times a day how you feel about how its doing (too hot ? too cold? too expensive?) or you could opt in somewhere and to give feedback. And then it could keep doing its behind the scenes mumbo jumbo.
How I feel is at least psychosomatically related to how the system is operating, but also how things are outside of the controlled environment.
Generally this is an algorithm that might work for one person:
* Is it equal or higher than the upper temperature outside? Run COOLing function (if it exists) until the LOWER inside temperature bound is reached.
* ELSE Is it equal or lower than the lower temperature outside? Run HEATing function (if it exists) until the UPPER inside temperature bound is reached.
In both cases DO NOT continue to run the circulation fans, at least above a whisper quite slow circulation, past the thermal shift operation. Luke-warm airflow makes the system perceptibly anemic and the results dis-satisfactory.
This is more of a product problem than a technological one. I don't think the issue is lack of imagination, but hostility from consumers.
There are a million and one cases where a pdocut could be 10x better with 10% more effort from me,and where that tradeoff is more than worth it (hell, it's why I use Linux). But the average consumer is aggressively turned off by having to do any "work", and the type of person hanging out on Hacker News is very high-percentile for wanting to put a nonzero amount of work into optimizing this type of thing.
I have already a solution where I can put zero amount of work for optimizing it, it's called a button. Why would I bother with something as needy as this "smart" thing?
I don't understand your comment or how it's remotely relevant to the thread. Is this just a non sequitur Luddite rant about the very existence of new tech?
Same story. We moved into a house that had a Nest preinstalled. Got everything set up, and noticed after a couple of days we would always wake up freezing in the early morning. Nest was all over the place and I just turned off the automation.
>And then after a few instances, I just turn off all the automation and set up a schedule like normal.
If you have a fairly regular life I would think a schedule would outdo ML pretty much all the time, because you know exactly what that schedule should be. ML might be useful for a secret agent whose life is so erratic that a schedule would be useless.
That is to say ML is maybe better than falling back to nothing.
ML can learn patterns that humans might not be aware of, so you there might be certain things that happen that show you will be on a mission to East Asia for a couple days.
Only when data is supplied to it to match the trained pattern,
ML is pattern recognition. Anything outside of that is still AI, but it isn't ML. I can think of very few feature sets we could supply to help predict someone will be deployed to East Asia for a few days other than scraping calendars and mail for religious and military organizations.
From a design perspective, Nest and others are either additively learning in situ to enhance a base model or they are working from a base model that doesn't directly learn, just classifies workflow to categorize observations on a base model. I doubt heavy training is occurring where the Nest and similar is treated as the central compute node.
One niche that ML seems to be growing into is /assisting/ humans, but not doing the whole task. ML might give you an image that is 90 percent what you want but needs a few tweaks.
If the task is clear enough, ML can take it on by itself, but this requires clear rules and an absolutely unambiguous definition of what winning means. For example, the best chess players in the world are machines, and are FAR better than the best human players. Same for Go (the game, not the programming language).
I bought one back before Google bought the company, because it seemed like a well-designed product with a good UI. In addition to the problems you mentioned, it was constantly updating its firmware. That sometimes bricked the device temporarily and sometimes it changed the UI so I had to relearn how to use the device. One update removed the ability to manually set "away" mode. I finally wised up and reset the thing so it couldn't attach to my wifi any more. Which made the app useless of course.
Then it became clear the thermostat wasn't getting enough power from my 2-wire thermostat transformer and that made it even flakier. I finally threw it away and replaced it with a $20 dumb thermostat, which will still be working fine after the zombie apocalypse. No more Nest products for me.
> Same with the "away from home" which seems to randomly think I'm away and I have no idea why.
Away from home should be an easy problem to solve assuming Nest can talk to your phone(which is almost 100% true in real life). IN my experience there are several easy heuristics that can achieve ~ 90% precision and recall in home detection, like are you connected to a wifi or even combining it with some IMU data to be more confident.
Everything should be an easy problem to solve yet it somehow stays a problem. For me it's quite binary: the problem is solved yes/no. And if the easy problem is still not solved, it's still not solved, period.
Which is not respecting your users. In fact, in my previous house the Nest was provided by the utility company and they used it /exactly/ for this purpose (although were legally mandated to notify us and allow us to opt out on a daily basis) where they'd intentionally raise your temperature during the hottest part of the day to reduce energy usage. But the thing is, I work from home, and if I'm sweating out a liter of fluids while I'm trying to work, I am getting nothing done and look unpresentable on meetings to boot.
In the end because most of the house was empty, I let the Nest do its thing and installed a separate mini-split AC in my office I kept set at 72 year-round because that's a sane and reasonable temperature for an office. Don't try to "nudge me into tolerating higher temps", respect my agency and choice about what is a comfortable environment for me to work in.
As a side note, I will never again buy a Nest product.
Things like this are exactly why I went with less "intelligent" smart thermostat. (Honeywell T9)
The only learning feature it has is figuring out how long it takes to heat or cool the house given the current weather. Before a schedule change, can heat or cool the house so it hits next target temperature on time. This seems to work extremely well.
Everything else like schedule and away settings are configured by the user.
Once nice feature is it is fully programmable from the thermostat, without internet. You only need the app for setting a geofence for automatic home/away.
Building my own thermostat so I have total control was a fun project, I learned a lot about electrical engineering and built a circuit with some TRIACs to control the HVAC lines. Though I still need to give it an interface so I can program it some way other than uploading the program as a JSON blob to my raspberry pi!
This has been on my project list for a long time haha, would you happen to have any blog posts or references for those of us who aspire to do the same?
Sorry for the late reply! Hmm, I don't have any particular posts I followed on hand, but I learned a lot by watching ElectroBOOM and Ben Eater's videos on YouTube. A general background in C++ development made getting started with Arduino code (you can use arduino frameworks for chips like the ESP8266) a snap as well
When people hear that FAANG is involved in something an "Emperor's Clothes" effect kicks in and people stop making the usual assumption that "if it doesn't work for me it probably doesn't work for other people"
Not only does the Nest ignore my preferences, I think it actually lies about the current temperature.
Example:
Setting is 72, reading is 73. AC is not on, I guess the thermostat is trying to save energy. I lower setting to 71, reading instantly drops to 72! I don’t think it’s a coincidence, this has happened several times.
> Setting is 72, reading is 73. AC is not on, I guess the thermostat is trying to save energy.
This can be explained by hysteresis [0], which all thermostats use to avoid cycling the A/C too fast.
But the second part where the reading drops instantly is strange. Sounds like some kind of software heuristic where they're trying to make the user feel more comfortable about the hysteresis interval. Or something.
I also hate how Nest only let me download at most 7 days of "historical" data. They have the rest of my historical data, but I can't get a copy of my own data.
The Ecobee is not really any better. It has various "features" which all end up in setting the temperature at a very uncomfortable setting even when you're home.
If I wanted those kind of savings, I could have just turned down my thermostat myself. Jeesh.
Matt Risinger (youtube expert builder guy) mentioned these are not anywhere near as valuable as they seem, and I'm inclined to agree. It's nice to be able to flick it on vacation mode when you're away I suppose.
I'd still buy it again, nice geeky metrics, and it's a quality company, but it doesn't save me anywhere close to 30% (or whatever the claim was).
Nah, you're not. I just gave up on mine and have a schedule. I also turned off "pre-cooling" because it would just kick on at like 6pm to "cool" the house for bedtime. I also bought several temperature sensors to use, which are fun. At night I have the thermostat use the sensor in my bedroom, then goes back to the main thermostat during the day.
- Use a python script to switch heating from my heat pump to my gas furnace when energy prices invert.
- Have live monitoring of my HVAC pressure health so that my servo controlled actuators don't bork my system.
This has been my experience as well. I was very excited to turn on the learning mode when I got it, only to turn it off a few weeks later in favor of scheduling. I've tried turning it on a couple times since then, and I always revert to a simple, manually-programmed schedule.
Nest is a terrible product. The learning aspect did not work at all and their app always takes forever to refresh. I switched to ecobee and couldn’t be happier. HomeKit and home assistant integration without hassle. Google products are generally quite terrible.
Well, the main selling point when it came out was that it was the iPhone of thermostats. It was the only thermostat at the time that did not have a terrible UI cobbled together by communist residential block designers or people who think that setting your own IRQ pins with jumpers is fun. But yeah I never understood the point of the learning feature; maybe a checkbox that needed to be ticked or a founder's pet feature.
While the chief absurdity is very clear (also mocked by Spitting Image - J.B. on a date: "You loved that steak? Good, I'll order another one!"), I am afraid that the intended idea may be that your memory about the ads of what you just bought will last as much as said goods.
Utter nightmare (unnatural obsolescence, systemic perversity, pollution...) but. I have met R'n'D who admitted the goal was just to have something new to have people want to replace the old, on unsubstantial grounds.
Sometimes you want a company like Facebook or google deliver ads to your potential costumer, but you don’t want, that the ad company knows how much product you sold.
You know your ROI, but your ad company doesn‘t.
I heard multiple stories from amazon, that hey still let the ad campaign or targeting running even if you bought the product for random times so external companies could not get insight in your businesses.
If you got the washing machine you will not click to buy another. If you pay per click, it makes no difference how long you let the campaign running if it is targeted.
College Humor--may it rest in peace :( :( :(--did a great bit about this a few years ago. https://www.youtube.com/watch?v=KbKdKcGJ4tM It is absolutely ridiculous how useless targeted ads actually are... and seeing a real person try to sell you this stuff really underscores the insanity.
Which is an extremely trivial check to add - if you got assigned that ticket, you'd probably point it at like 2 or so hours.
However, they've been like this for over a decade so it's likely there intentionally. here's one way that could be possible:
There could be some popular third party service that's integrated on many e-commerce sites that sells this information and doesn't actually give a damn if you bought the refrigerator or not. They're selling you, not the refrigerator.
That's the problem with data brokers, it's mostly low quality data.
> Which is an extremely trivial check to add - if you got assigned that ticket, you'd probably point it at like 2 or so hours.
Yep, totally a 2hr task for an engineer who works on homedepot.com to “check” that you bought a fridge from lowes.com after you first price shopped the other site. Also a two hour task for a Google engineer to know you bought one in person at Best Buy after researching online first.
Yes, there are basic cases (buying from same merchant as who’s suggesting) that should be handled, but let’s not foolishly pretend that’s the average case, let alone majority / all.
I haven't worked at Amazon in this vertical so if people know definitively feel free to correct.
The Amazon case could be the same problem I discussed before. Third party sellers can pay fees to promote/boost their listings on Amazon so ultimately the same incentive structure holds if there's fees for impressions and not just sales.
I cannot remember the reference now, but the reasoning I read was a person who just bought an item x might:
1. return the item if they are not satisfied with it and get a replacement Or
2. buy another one as a gift if they really like it.
Both of these result in a higher fraction of conversions in this kind of targeting vs other targeting criteria.
I think the reason this happens is that when you start looking for washing machines, you start getting ads for them. Then when you buy nobody tells the ad companies that you just bought a washing machine so they still send you ads because they think you’re still looking. Even if you just went straight to the model site and clicked “buy”.
Amazon can’t suggest a “product you might like” if their life depended on it. They have 19 years of purchase history for me spanning thousands of product categories. Want to know a category I’ve never looked at or bought from Amazon, women's handbags. What’s near the top of the suggestions right now on the homepage for me? You guessed it!
The weirdest thing is that at one point, I think it's now almost 10 years, they had somewhat OK recommendations. It's not like "you bought programming books and Battlestar Galactica DVDs, so you might be interested in Neal Stephenson's novels" is really advanced but it's still miles ahead of the "you just bought a washing machine, so you might be interested in washing machines" level.
I think the issue is that it’s no longer “recommendations”, instead it’s now “here’s the things which we can make the most money off you buying based on your recent spending”.
A slightly more ridiculous one I experienced recently was when I searched where to buy tool X. Started getting ads for why I need tool X and why it's the best tool ever. I already want one, I'm looking where to buy, not trying to learn what it's for!
> GPT can solve this! I prompted it with "Sarah bought a washing machine and a ". It completed "dryer.".
The most natural interpretation there is that Sarah bought a washing machine and a dryer simultaneously, not that, after buying a washing machine the month prior, she was finally ready to buy a dryer.
Fair enough regarding the washer/dryer, but I'm unimpressed by the hammer/nail example. If anything, I might want to buy "some nails" rather than "a nail", but I would argue that's not a brilliant guess either—if I'm buying a hammer I most likely already own the item that I want to hit with it. Some more interesting suggestions could have been for example "some pliers", "a set of screwdrivers", "a pair of work gloves", "safety goggles", etc.
I can personally vouch that Amazon, Twitter, and YouTube all do horrible horrible jobs predicting my taste. And they have got worse over the years, not better
I had just preordered novel 9 of The Expanse, and I got an email recommending something else from the same authors: novel 8 of the Expanse. A more sensible recommendation engine might have assumed that someone who preorders part n+1 of a series may already have part n. Not to mention that Amazon should have known that I already had novel 8 on my Kindle.
I guess generating personalized recommendations at scale is still too expensive. We just get recommendations based on what other customers with vaguely similar tastes were interested in.
> Not to mention that Amazon should have known that I already had novel 8 on my Kindle.
Amazon doesn't seem to understand many things surrounding the Kindle. For example, it calculates the progress reading through a book by the last page I looked at. That means if I finished a book and jumped to the introduction it'll now be convinced I only read 1% of the book. This is so dumb, and I don't know why they even do it that way - the Kindle hardware should easily be capable of precisely keeping track of what pages I looked at.
Funny thing is, your very comment is an indirect praise of the very thing they were advertising to you, and here it is being read by thousands of people. Are we so sure absurdly terrible ads don't actually beat out actually good well tailored ones? Looking through the history of radio ads, television ads, it seems like the best ads are always the stupidest. "Head on, apply directly to the forehead!" isn't so far off from "You bought a washing machine? Buy another!". The reality is, advertising optimizes to target stupid people because stupid people spend money. It is easier to trick a moron then sell a smart man something they actually want.
Yep, Spotify, Amazon, Youtube, Google etc. they all use the same three algorithms:
- the more of the same thing algorithm. You clicked this thing, would you like to click it again. And again. And some more.
- the ever popular "we've shown you this thing a hundred times now and you never clicked it; we'll just assume we are right and you are wrong" algorithm.
- the ooooh we've detected that your ip address is in Germany and predict that you are now fluent in German. Would you like some Schnitzel with you Schlager music? This one in particular drives me nuts. I have user profiles with these companies for many years, browser settings that specify a preferred language, etc. I consistently never do anything in German with them. And they'll go ... here's some German content for you. Completely useless and obviously the only criteria they use for these recommendation is location. Worse, if I travel they'll unhelpfully suggest things for those locations as well. Basically, most of their top recommendations are the same generic stuff that they serve to everybody else in the same location.
Recommendation engines are hard and these companies gave up years ago and instead routinely by pass their own AI with some simple if .. else logic. I know how this stuff works. There's a little corner in the UI for the cute AI kids to do their thing but essentially all the prime real estate in their UIs is reserved for good old if else logic, basic profiling, and whatever their marketing department wants to promote to everybody.
If user in Germany recommend generic German stuff. There's no logical explanation other than that for the absolute garbage recommended by default. If you clicked a thing, here's some more things from the same source in random order. Amazon has a notion of books being in a certain order ... so why recommend I start with part 21 of a 50 book series by an author I've never bothered to read? Maybe book 1 would be a better start ... Book series are great value for them because if I get hooked, I consume the whole thing.
Most recommendations are just variations of simple profiling (age, sex, location) that consistently trump actual recommendations combined with very rudimentary similarity algorithms. You don't need AI for any of that. I work with search engine technology, it's not that hard.
The one thing I've been consistently impressed with is TikTok. If I compare recommendations on YouTube to what I get on my TikTok FYP, it's like comparing a 5-year-old to a college graduate on a math test.
Literally to the point where YouTube never pulls me down into the rabbit hole anymore, I watch one video because it was linked from somewhere else, then I bounce.
Part of the reason they're horrible is because people don't have consistent interests. I might be interested in raunchy content right now, but I won't be a few hours later. What determines whether I'm interested in the former is outside of the control of these algorithms - they don't know all of the external events that can change my current mood and preferences. As a result of this it makes sense for people to have many profiles that they switch between, but AI seems incapable of replicating this manual control (so far).
Sometimes I want to watch videos about people doing programming, but usually I don't. When I do though, I would like to easily get into a mode to do just that. Right now that essentially involves switching accounts or hoping random search recommendations are good enough.
> Part of the reason they're horrible is because people don't have consistent interests. I might be interested in raunchy content right now, but I won't be a few hours later. What determines whether I'm interested in the former is outside of the control of these algorithms
I don't think that matters at all. People don't complain that they're getting recommendations that would have been great if they had come in an hour/day earlier or later. When you get a recommendation like that, you consider it a good recommendation.
Instead, they complain that they're getting recommendations for awful content that they wouldn't choose to watch under any circumstances.
I think it matters a lot, because that "awful content" is popular enough that others watch it. People can watch and read things they swear they would never be interested in, but this only happens sometimes. The algorithm taking cues from that is itself a discouraging factor on clicking on them.
Youtube's is actually pretty surprisingly good, in my experience. After years of use, it consistently filters out all the absolute trash I don't want to see, and recommends me things I do actually want. It's not perfect, but it often directs me to channels I wouldn't have heard of otherwise that have solid content.
It often finds a video I would like and throws it on my front page. I avoid it for awhile thinking it wouldn't be a good fit (I don't recognize the creator, bad thumbnail/title, unclear why the content would be of interest to me, etc) but find it was great and I should have watched it days ago.
If I log out of my account, the front page of the site is just awful, makes me want to throw up.
Same. For a long time they wanted me to watch angry white guys complaining about pop culture. But all I want to watch are PBS science shows and stuff about ancient history! Eventually, they gave up, and now half the videos they recommend are ones I already watched years ago.
If you go deep into all the settings you can turn all of amazons predictive jibber jabber off and turn off a ton of tracking. It has been awhile but I swear there were settings hidden everywhere. I just went back to amazon and it still seems to suggest some products based on what I just viewed so now I am wondering what the hell I even turned off.....it is actually fairly accurate now though (I looked at a timex watch and it is suggesting a very similar timex watch)
edit: My bad - it was just suggesting products I had recently previously viewed
That may make sense of you are not the average consumer. Optimizing for the most common case makes sense. I see that with Google search prediction, it's good but many times it predicts very sensible words for general use but not in the topic that I'm interested.
ML is there to maximize business income--nothing else.
If ML was benefiting me, it would know that 90% of the time I fire up Hulu I plan to watch the next episode of what I was watching last time. And it would make that a one click action. Instead I have to scroll past promotional garbage...every single time. Assholes.
I don't know why you assume the goal is "help aaronax watch what he wants quickly" vs "make sure when aaronax switches to his next series/movie it's on Hulu"
Yeah, in these cases people try to balance for both "engagement" and "retention" (probably called something like "discovery" - the more they can show you about what other options they have, and the more shows you start watching, the less you quit after the first show you watch).
The problem is that they don't have a way of collecting "annoying" metrics as easily as they do those two. So it's a big blind spot in terms of "should we tweak more in favor of one or the other."
Customer satisfaction often translates into more dollars, though, because it means they won't cancel their service. I've had the same thought: if only this multi-billion dollar company could figure out that I want to continue watching the show I watched yesterday.
I would think it would be long-term satisfaction optimization. I'm not trying to optimize your binging of a single show (which you might watch then cancel after), I'm trying to get you to love enough of my product line to stick around.
Honestly a lot of this ML to me seems eerily similar to how in older times people would use sheep entrails or crow droppings to try and predict the future. I mean basically that is what ML is, trying to predict the future, the difference is they called it magic, we call it math, but both seem to have about the same outcome, or understandability.
> I mean basically that is what ML is, trying to predict the future
If being so reductive, that's also the scientific method. Form a model on some existing data, with the goal of it being predictive on new unseen data. Key is in favoring the more predictive models.
> they called it magic, we call it math, but both seem to have about the same outcome
One of the oft overlooked, yet critically important aspects of the scientific method is the hypothesis. You don’t design an experiment having absolutely no idea what to expect. You have an educated guess in mind (the hypothesis), and you design the experiment in such a way that says “this result will rule out my hypothesis, while this other result might confirm it.”
Just trying two things at random and picking the one that makes some arbitrary metric go up, is not the scientific method. It’s gradient descent.
Note that I'm being intentionally reductive to argue that the net buscoquadnary threw around ML models and future-telling with sheep entrails also includes the scientific method.
I do also think that ML as a field progresses through the scientific method ("I theorise that this network with residual connections will converge faster, lets see if there's a significant difference") - but maybe not to the full extent it could.
> Just trying two things at random and picking the one that makes some arbitrary metric go up, is not the scientific method. It’s gradient descent.
I'd say that's closer to evolutionary algorithms. GD finds (locally) the direction to tweak the weights to improve predictions on a given batch.
> for most of the more interesting consumer decisions, those that are “new” and non-habitual, prediction remains hard
Translation: Computers can't read minds.
A bigger generalization is that, whenever a software feature becomes essentially mind reading; someone's either feeding a hype engine or letting their imagination run away.
The best things to do in that case is to pop the bubble if you can, or walk away. I will often clearly state, "Computers can't read minds. You're making a lot of assumptions that will most likely prove false."
Always interesting to see outsiders writing papers about this, using anecdote and unrelated data (mostly political and real world purchase data in this case) to argue that ML doesn't make useful predictions. Meanwhile I look at randomized controlled trial data showing millions of dollars in revenue uplift directly attributable to ML vs non-ML backed conversion pipelines, offsetting the cost of doing the ML by >10x.
It reminds me a lot of other populist folk-science belief, like vaccine hesitancy. Despite overwhelming data to the contrary, a huge portion of the US population believes that they are somehow better off contracting COVID-19 naturally versus getting the vaccine. I think when effect sizes per individual are small and only build up across large populations, people tend to believe whatever aligns best with their identity.
> Always interesting to see outsiders writing papers about this, using anecdote and unrelated data (mostly political and real world purchase data in this case) to argue that ML doesn't make useful predictions. Meanwhile I look at randomized controlled trial data showing millions of dollars in revenue uplift directly attributable to ML vs non-ML backed conversion pipelines, offsetting the cost of doing the ML by >10x.
I regularly buy the same brand of toilet paper, socks, and sneakers. Machine learning can predict that.
But, machine learning can't predict that I spent the night at my parents house, really liked the fancy pillow they put on the guest bed, and then had to buy one for myself. (This is essentially the conclusion in the abstract.)
Such a prediction requires mind reading, which is impossible.
The key insight missed by this paper (and people from the marketing field in general) is that cases like that are extremely rare compared to easy to predict cases. They don't matter right now at all for most products, from the perspective of marketing ROI.
Also ML can predict that, BTW. Facebook knows you are connected to your parents. If the pillow seller tells Facebook that your parents bought the pillow, then Facebook knows and may choose to show you an ad for that pillow.
> Also ML can predict that, BTW. Facebook knows you are connected to your parents. If the pillow seller tells Facebook that your parents bought the pillow, then Facebook knows and may choose to show you an ad for that pillow.
I think you're letting your imagination run away, and I think you're trying to exceed the limits of the kind of information that you can collect and act upon.
What you're trying to do is mind reading, and computers physically cannot do that. (Nor can people)
You are friends with your parents on Facebook.
Your parents buy the pillow.
The pillow seller tells facebook that your parents bought the pillow.
Now Facebook knows that somebody who is your friend recently bought the pillow.
Facebook may decide to show you an ad for that pillow because somebody who is your friend recently bought the pillow.
The result may look like "mind reading", but it's actually very simple in terms of actual prediction.
I think you may be conflating the topics and goals of adjacent exercises; predicting consumer behavior is not the same thing as optimizing a conversion pipeline.
The examples they give in section two are directly relevant to optimizing conversion pipelines. They pretty clearly intend to be describing something relevant the e-commerce user experience.
Are you really sure you're not just fooling yourselves with your randomized controlled trials? As Feynman famously said, the easiest person to fool is yourself. And in business even more than science, you might even like the results.
Have you ever put this data up against something similar to the peer review system in academia, where several experts from a competing deparment (or ideally competing company) try to pick your results apart, disprove your hypothesis?
well, certainly it's possible to fool yourselves with A/B testing, it doesn't mean you must be fooling yourselves. I've also seen similar results in recommendation settings in mobile gaming, not once but over and over again across portfolio of dozens of games/hundreds millions of players. You don't need to predict 20% better on whatever you are predicting to get a 20% increase in LTV and it's even better if you are doing RL since you are optimizing directly for your KPIs
The actual conclusion of the study is so absurd that it's not worth engaging with seriously.
That is, to maximally understand, and therefore predict, consumer preferences is likely to require information outside of data on choices and behavior, but also on what it is like to be human.
I was responding to the interpretation from the blog post, which is more reasonable.
Yes, the review paper appears to be roughly conditioned on "using data that academics can readily access or generate".
Clearly, this doesn't generalise to cases where you have highly specific data (e.g. if you're Google).
However, cases with large societal impact are more likely to be the latter? They may perhaps better be viewed as "conditioned on data that is so valuable that nobody is going to publish or explain it", which kind of is in the complement of the review?
If your ML model is able to predict what consumers are going to buy, the revenue lift would be zero.
Let's say I go to the store to buy milk. The store has a perfect ML model, so they're able to predict that I'm about to do that. I walk into the store and buy the milk as planned. So how does the ML help drive revenue? The store could make my life easier by having it ready for me at the door, but I was going to buy it anyway, so the extra work just makes the store less profitable.
Maybe they know I'm driving to a different store, so they could send me an ad telling me to come to their store instead. But I'm already on my way, so I'll probably just keep going.
Revenue comes from changing consumer behavior, not predicting it. The ideal ML model would identify people who need milk, and predict that they won't buy it.
This is incorrect. You can predict many things that drive incremental revenue lift.
The simplest: Predict what features a user is most interested in, drive them to that page (increasing their predicted conversion rate) -> purchases that occur now that would not have occurred before.
Similarly: Predict products a user is likely to purchase given they made a different purchase. The user may not have seen these incremental products. For example, users buys orange couch, show them brown pillows.
Like above, the same actually works for entirely unrelated product views. If users views x,y,z products we can predict they will be interested in product w and we can advertise it.
Or we predict a user was very likely to have made a purchase, but hasn’t yet. Then we can take action to advertise to them (or not advertise to them).
ML is useful for many things. I'm asking the question of whether prediction is useful, and whether it is accurate to describe ML as making predictions.
The reason to raise those questions is that for many people, the word prediction has connotations of surveillance and control, so it is best not to use it loosely.
The meaning of the word "predict" is to indicate a future event, so it doesn't make grammatical sense to put a present tense verb after it, as you have done in "Predict what features a user is most interested in." Aside from the verb being in the present tense, being interested in something is not an event.
You can't predict a present state of affairs. If I look out the window and see that it is raining, no one would say that I've predicted the weather. If I come to that conclusion indirectly (e.g. a wet umbrella by the door), that would not be considered a prediction either because it's in the present. The accurate term for this is "inference", not "prediction".
The usage of the word predict is also incorrect from the point of view of an A/B test. If your ML model has truly predicted that your users will purchase a particular product, they will purchase it regardless of which condition they are in. But this is the null hypothesis, and the ML model is being introduced in the treatment group to disprove this.
You can predict a present state of affairs if they are unknown to you.
I predict the weather in NYC is 100F. I don’t know whether or not that is true.
Really a pedantic argument, but to appease your phrasing you can reword my comment with “We predict an increase in conversion rate if we assume the user is interested in feature x more than feature y”
That is a normal usage in the tech industry, but that's not how ordinary people use that word. More importantly, it's not how journalists use that word.
In ordinary language, you are making inferences about what users are interested in, then making inferences about what products are relevant to that interest. The prediction is that putting relevant products in front of users will make them buy more - but that is a trivial prediction.
Exactly. I know someone who does this for a certain class of loans, based on data sold by universities (and more).
Philosophically -- personally -- I think this is just another way big data erodes our autonomy and humanity while _also_ providing new forms of convenience. We have no way of knowing where suggestions come from, or which options are concealed. Evolution provides no defense against this form of manipulation. It's a double edged sword, an invisible one.
If the store knows you will want to buy milk, it will have milk in stock according to demand. If it doesn't have a perfect understanding of whether or not people want to buy milk, the store will over/under stock and lose money.
No, I'm the person who doesn't know the great things to buy with my Raspberry Pi. Thanks to great predictions from Amazon's part, they get me to buy more. Similar to how Netflix does a pretty good job of recommending movies.
I know this is slightly off what the article is concerned with, but the important question in a business context is whether this prediction is worth anything, i.e. whether it can be turned into revenue that wouldn't be generated in the absence of the prediction.
> Sophisticated methods and “big data” can in certain contexts improve predictions, but usually only slightly, and prediction remains very imprecise
The worst part of big data is the data itself. Used to be common will be shared on Facebook webs about "what is your political compass". There results were used to create political profiles of users and targeted propaganda.
You don't need ML to predict the data that there user already has given.
> Currently, we are still far from a point where machines are able to abstract high-level concepts from data or engage in reasoning and reflection
Of course when an AI does that, we then say its just doing statistics, not reasoning.
Until you have built a recommendation engine from scratch, it is hard to appreciate the complexity. I don't mean the complexity of the code or algorithm (ALS and Spark are straightforward enough) but the contextual problem. Models end up being large collections of models in a complex hierarchy, with hyperparams to tune higher level concepts such as "surprise" or business targets such as "revenue", "engagement" etc. TikTok have nailed this, as has Spotify.
>Of course when an AI does that, we then say its just doing statistics, not reasoning.
no, AI simply doesn't do that. Even Demis Hassabis of Deepmind fame in a recent interview pointed this out. Machine learning is great on averaging out a large amount of data, which is often useful, but it doesn't generate true novelty in any human sense. AI can play Go, it can't invent Go.
In the same way today's recommender systems are great at averaging out my last 50 shopping items or spotify playlist but they can't take a real guess at what truly new thing I'd like based on a genuine understanding of say, my personality. Which is reflected in the quality of recommendations which is mostly "the thing you just bought/watched", which is ironically often incredibly uninteresting.
> but it doesn't generate true novelty in any human sense.
Few humans do either, even great artists create from, or react to, the art they experienced in their life. At the end of the day this is a rather metaphysical question. A Go model is trained to play Go, not create new games.
> on a genuine understanding of say, my personality
That is exactly what they are trying to do. The purpose of many recommendation systems is to uncover the latent variables and categories that you might like. They are not just averaging what you listened to. It is feasible (but very unwise) to predict certain users like to listen to happy upbeat tracks on Fridays, and sad songs sung in Romance languages on Mondays.
Each time you skip a track or relisten to a track (implicit feedback) or like a track (explicit feedback) you are giving it information, which might be more than just the interaction (the timestamp, the IP address location, how quick you were to skip the track etc.).
On the other hand, the purpose of those systems is simply to optimize what generates the most short term income, in which case, you get 'boring'/simple shopping recommendations that on average make more profit.
Systems have tried to predict model the human experience, but in the past there have been epic fails so nobody is trying to explicitly do that anymore. The classic example would be stores deciding teens were pregnant and inadvertently alerting their parents. Nobody wants to see ads for Amazon's in-house toilet paper brand just as they are about to get up to go to the toilet.
>to predict certain users like to listen to happy upbeat tracks on Fridays, and sad songs sung in Romance languages on Mondays.
This is still entirely superficial. There's no recommender system that could even tell what a 'sad track' is other than by human labelling, or distinguish say, a satirically sad track from the genuine thing, or accurately tell what any individual even perceives as happy or sad which is highly subjective.
It's not about metaphysics but about a very practical fact that these statistical systems fail at. Making genuinely novel human recommendations requires and understanding of human nature in general, individual nature of the person in question, and understanding of context and meaning. In short a system like this needs to be able to reason about what it is that it's recommending and who it's talking to.
There is no system that could invent a game like Go because it has nothing to do with data in any direct sense. It'd require being able to reason about what makes games compelling and aesthetically pleasing to human beings and that's not a question of mapping one button click to another.
Somehow I have the opposite experience of everyone else on this site?
I frequently get targeted ads for exactly the item I'm interested in, e.g. a pair a shoes that actually look very similar to a pair I was considering but from a different brand.
My YouTube recommendations bring me a ton of new videos on topics I'm interested in. I do a tiny amount of curation, but overall rarely get objectively bad videos recommended.
Amazon recommends consumable products for me at approximately the rate I consume them. When my pet died and I stopped purchasing treats, they seem to have figured that out and stopped recommending them.
The "you just bought a car, would you like to buy another car?" does happen from time to time, but not all that often to be honest.
My guess is targeted advertising works great up until the point you share your devices with other people. parents and their kids sharing accounts, laptops, tablets leads to a bunch of junk input and this makes up the majority of population
Personal anecdote but my personal experience is that targeted advertising seems to be predicting my past me. I can't describe it exactly, but it seems to never realise that I'm done with (something), especially with once-in-a-moon purchases (like laptops).
Google seems like they target by age, gender and income rather than by interests. Sometimes it's convinced I'm a yuppie and keeps showing me luxury cars, personal care/beauty products and high end electronics (when I have zero interest in any of those products).
Ironically I find the "dumb" ads on cable tv news to be a lot more effective since they have to target by interests.
I'm not surprised at this result, mostly because of the inaccurate noise that the business of "marketing," (i.e. specifically marketing people selling their not-very-effective services) generates.
It is the same on Netflix. I have phases where I watch a certain genre for a few weeks and then move on. For example after a few Scandi crime series it is time for something else. However, at the same time my daughter loves Animé and pretty only watch that. It is really hard for an ML algorithm to grab these nuances.
Netflix makes a far more obvious sin: not having “who is watching” as boolean choices. If I am watching with my partner, I want both of our accounts to mark that series as viewed. And I really want Netflix to tell me what I’m watching with her so that I don’t continue watching it without her because I will be single if that happens (again).
Google has fallen off the radar for me. I'm not one to waste time on someone who, if you ask me, is just another advertising company. It's the same old wine in new bottle. They pretend to predict correctly. Their false positives are true negatives.
I’ve only ever seen good ads on Facebook pre-iOS privacy update.
Every other online ad has been a dumpster fire, from gross images, things aimed at people decades older or younger, or large items I just bought and aren’t buying again in years.
As far as I'm concerned, the question is how ML/AI stacks up against the competition -- humans. I don't know, but I'd bet the answer is that ML is much better. Let's say at least 20 percent better, but I imagine it's much higher than that.
Second, this is only saying that right now, ML's performance is "not that good." It says nothing about future technical advances. If you look at the track record of ML in the past three decades, it's amazing, and if that performance is repeated in the next three decades, who even knows what things might look like. (Machine sentience? Maybe.)
They'd also do well to add some common sense filters. Amazon must have 5+ years of regular purchase history on my. You'd think thye could deduce by now that I am not in the market for pet and baby stuff.
Sometimes I'd settle for some vanilla common sense. Watch Episode 1 of something on Prime. You'd think next time it would suggest Episode 2, right? It does...its thumbnail #25 in the grid & below the fold.
It seems to me like the "generation" use case of ML is much more promising than the "prediction" or "classification" use case. It's tough to predict things in general because our universe is fundamentally uncertain. How is some computer going to predict that some mugger sees a target at some random spot and decides to mug them? But the progress in text to image and text generation really blows my mind.
That's probably because people themselves are bad at predicting their own choices. Depending on mood, the day, even your hydration level, you can feel vastly different about something.
I believe this was a conclusion of the Netflix Prize contest, getting much more accurate at predictions was hard because people would not reliably rate a movie the same.
I've shared this before on HN, but it never fails to make me laugh when I think about it:
>Several years ago a conversation about a similar topic prompted me to look at the ad targeting data Facebook had on me. At the time I'd had a Facebook account for 12 years with lots of posts, group memberships and ~500 friends. Their cutting edge data collection and complex ad targeting algorithms had identified my "Hobbies and activities" as: "Mosquito", "Hobby", "Leaf" and "Species": https://imgur.com/nWCWn63. Whatever that means.
> We suggest marketers focus less on trying to predict consumer choices with great accuracy and more on how the information environment affects the choice of their products.
Similar to the thesis of How Brands Grow by Byron Sharp
As far as I can tell every single penny of the hundreds of millions (billions?) of dollars that have been spent on ML recommendation engines has been wasted and the team of highly-compensated developers who spent years burning through cash building Hadoop clusters, buying GPUs, building dashboards and churning out thousands of tables and charts for execs to ponder over just to build something that recommends I rebuy something I already bought could have been replaced with a 24-year old communications major who also understands the products being sold and the people buying them.
I think that the main issue is less the technique (although… yes, please use RL if you can) and more the lack of data. Browsing gives very little insight: dwell-time is a poor proxy for interest, and mixes horrid ideas that are so bad they are worth sharing with friends and confusing photos where you need to squint to figure out if it’s what you are looking for.
Both e-commerce and social media are really not good at gathering express feedback for what people want and valuing that expressly. Please, let me tell you that I did spend time looking at this thread about the latest reality TV scandal but I don‘t want to hear about it ever again! Please, let me tag options as “maybe” or let me tell you what you’d need to change for me to buy that shirt. Public, performative Likes and Favourite lists that are instantly reactivation spam-fodder… Come on, you know better.
I used to work for a big e-commerce site (the leading site for 18-25 y.o. females). We had millions of references (really) and it was a problem. The search team had layers upon layers of ranking algos, incredible papers at conference… but still, low impact on conversion. It was more than anything else that we could do, but nowhere as transformative as it could be. Instead, I suggested copying the Tinder interaction in a companion app:
* left, never see that item again;
* right, add it to a long list of stuff you might want to revisit. We probably would have to separate that from the Favourite list to avoid clutter, but maybe not, to make that selection worthwhile.
The learning you could get from that dataset, even with a basic RL algo to queue suggestions… People thought it was “too much” which I’m still bitter about.
How are bandits used in consumer choice problems? Bandits solve almost the inverse problem: which choice to offer/take when it's uncertain which is best, but the problem under consideration in the blog post is about predicting which choice a consumer will pick, a standard marketing problem.
The title is oversimplified clickbaity, and their actual finding is [from abstract], and my reaction below:
Our main conclusion is that for most of the more interesting consumer decisions, those that are “new” and non-habitual, prediction remains hard. In fact, in many cases, prediction has become harder due to the increasing influence of just-in-time information (user reviews, online recommendations, new options, etc.) at the point of decision that can neither be measured nor anticipated ex ante. Sophisticated methods and “big data” can in certain contexts improve predictions, but usually only slightly, and prediction remains very imprecise—so much so that it is often a waste of effort.
My initial reaction on skimming it is "savvy consumers are becoming increasingly desensitized to a sea of Facebook ads, Amazon fake reviews and rigged star ratings, undisclosed compensated influencers", aka the "unprecedented information environment" as the author describes things. I wouldn't call an Amazon review or FB ad/influencer post/affiliate link "information" as distinct to "this laptop has a rated battery life of 18h" or "this toaster comes in the following 5 color choices"; it's simply an influencing attempt; whether those contain any information (/misinformation), and whether users trust that they contain accurate information, seems to be something the study doesn't want to look into. Really the authors seem to be giving a very-judgment-free pass to anything calling itself "information".
Consider "unprecedented information environment" could also characterize the 2016 and 2020 US elections, 2016 UK Brexit referendum and 2022 Philippines election: "it is important to first understand how consumers make choices, particularly in the current information environment in which they have access to an unprecedented amount of information at the time they are making decisions." [obviously third-party political advertising is far less trustworthy than consumer advertising, but still].
What if the authors merely succeeded in proving that the rise in targeted influencing attempts has rendered the public more wary of targeted influencing attempts?
So this machine learning and deep learning hype has shown that it is a gimmick isn't it? After years of surveilling, collecting and training on user data it still doesn't work or gets attacked very easily over spoilt pixels and many other attacks?
What a complete waste of time, money and CO2 being burned up in the data centers.
I don't know.... I think back on google search back in the ~2014 era. It was good. Like scary good. Like I'd type "B" and it would suggest "Btu to Joules conversion" and things like that. Actually it was better than that... it would anticipate things I hadn't even searched for before with very very little prompting. It seemed to adapt to context whether I was at work, on my phone, at home, etc. The results were exactly what I was looking for.
Then it got taken over by ads and SEO and corrupting influences and it's just not that good anymore. IMO, the problem with DL isn't the tech. It's the way its being used. The reality is: For 99% of things advertised to me, I don't want to buy the goddamn product, and no amount of advertising will make me want to buy it. It's gotten to the point where if I see an ad for a product I think I'm more likely to buy a competitor whose ad I haven't seen because I assume the competitor is investing more in the product than the marketing.
And everyone seems to have forgotten about hybrid approaches of ML and human beings that, IMO, are really good. But alas, "they don't scale".
But at the same time, it's really interesting. For as much data as facebook should have about me, their ad rec's really suck and always have. (Perhaps it's because my only ad clicks ever are accidental ones?) I'm kind of astounded at how poor that result is. That said, I'm always very impressed by spotify's recommender system. I think it's one of the best on the net.
Another thing I find interesting is that non-vote-based social media feed systems all really suck. Once they ditched chronological ordering it stopped appealing to me, and I don't know exactly why that is. Evidently I'm on some tail of the curve they don't care about.
No, it just isn't a silver bullet for every problem under the sun. But quite a few record holders on various problems are ML solutions and that is unlikely to change for the foreseeable future.
It's just that as soon as you start out on every problem with 'ML will solve this!' that you're going to end up with a bunch of crap. The right tool for the problem wins every time.
So the best experience was this internal event that we had, where the raters would say that certain Ad would not fare well (long term), while the initial metrics (automated) were showing the opposite (short temr). So then we'll gather into this event, and people would "debug" these and try to find where the differences are coming through.
Then we had to help another group, where ML failed miserably detecting ads that should've not been shown on specific media, and raters came to help giving the correct answers.
The one thing that I've learned is that humans are not going to be replaced any time soon by AI, and I've been telling my folks, friends or anyone (new-born luddities) - that automation is not going to fully replace us. We'll still be needed as teachers, evaluators, fixers, tweakers/hackers - e.g. someone saying - this is right, and this is not, this needs adjustment, etc. (to the machine, ai, etc.).
Maybe machines are going to take over us one day, but until then, I'm not worried...
(I've also understood I knew nothing about staticics, and how valuable linguists are when comes to forming clear, concise and non-confusing (no double meaning) questions)