It's worth pointing out that this recommendation behavior is only unintelligent if it has a downside, i.e. reduces Amazon's profits. Companies (and their AIs) only care about your business transactions, not whether you're annoyed.
I'm sure Amazon engineering is well aware of the jokes about their recommendation engines. But their recommendation engines are not calibrated to reduce jokes, they are calibrated to increase sales.
If you only know ONE thing about a person, pinging them about that one thing might still be better than ignoring them, or offering completely random recommendations. A memorably annoying ad can work better than a boring and inoffensive one, sometimes.
That's not true, putting a product I've already ordered on my recommended list instead of something i actually want to buy is costing them money in lost sales.
In that case, it may be better to just put the hot new item in a category they know you purchase in. For instance, if I have a subscription to paper towels, maybe they should send me a subscription recommendation for toilet paper.
I'm sure someone thought to A/B test the recommendation engine against the whatever-happens-to-be-selling-right-now engine and the recommendation engine won.
That very much depends on the domain. In this case, I suspect it would be considered phenomenally good since a human would achieve a much lower number. Conversion rates for web ads created by humans are typically not more than a few percent.
Amazon also doesn't know if this is a single use or consumable item. Some things have significant marginal utility beyond 1, and others can be consumed and need to be reordered. (Charging cables, Amazon echo for multiple rooms, paper goods, etc.)
How often do I buy a HP Color LaserJet printer, versus toner cartridges for it? Or ink for another printer of which it has no evidence that I own)?
Yet I bought a color laser printer and my first two pages of recommendations are for other variations on the same ("Hey, you bought the wired MFP! Perhaps you want the wireless too! And I know it's a HP printer, but maybe I can suggest some Brother toner for you!").
You're correct that Amazon should be smarter about some recommendations. I'd say give them the benefit of the doubt in that they may have found an algorithm that works most of the time (or at least when it comes to their bottom line), but edge-cases stand out to us.
Still though, there could be a method to the madness. You bought a printer - are you setting up a home office? Perhaps you want a second printer for upstairs?
Also, you just bought an HP brand printer. Were you replacing a printer that just ran out of ink? Maybe THAT printer was a Brother printer. Perhaps this Brother ink cartridge can extend the life of the hypothetical printer you may be replacing...
My point is that I can contrive cases where the recommendations might make sense - not to you, specifically, but to a large enough cross-section of consumers.
I don’t doubt there are entire divisions in Amazon that draw statistical correlations and do A/B testing to figure out what annoying emails work. Just like any other marketing agency. So you can be sure that any annoying email you get is the probably the result of years of metrics.
That’s assuming those are the alternatives. If the alternative is instead to show you something you’ve already ordered or nothing, the odds of you ordering again if you see the thing you’ve ordered already are small, but nonzero.
That's a false dichotomy. There's a number of things they can show you, not just reorder or nothing.
In a perfect world for targeting sales, they will predict and show you exactly what it is you want to buy next. Repurchasing is just a very lazy way of predicting what you'll buy next.
If the chance I'm going to buy something that I've already bought again is close to 0, and it's not smart enough to find anything else for me that it thinks I should buy, then logically it should show me random things. That way I'm training the algorithm for them and maybe I'll find something I want.
Logically, they will show you whatever is most likely to generate revenue (possibly amortized over time). As long as the probability of you reordering is higher than that if you buying a random item out of millions (and those odds are tiny to begin with), the logical thing to do is to show you the item you already ordered.
People buy a lot of consumable products on Amazon. People buy a lot of things and realize how much they like and would like another for another room in the house or something. You aren't thinking broadly enough here.
I don't buy it, wrt recommending something already purchased. If the one thing you know about someone is a specific book (or other purchase people tend to only make one of) they bought, it's a decent assumption that they are more likely to buy literally anything else from you.
I think some people buy multiple copies or order over and over again, so the actual probability is high relative to other items (because as others have said, the total number of items is huge).
I think maybe the real mistake is that someone who is buying a bunch of copies doesn't really need the item to be recommended to them. The recommendations are more helpful for things that you can't find in the history.
It's bizarre to think amazon knows ONE thing about me when I exclusively shop using amazon. They know EVERYTHING about me, yet they still recommend things I purchased before.
This one is funny, and has a couple of obvious solutions that have been prevented due to internal politics. The short answer is that they have a couple of different recommender systems, all competing against each other internally for sales lift.
One is purely based off of pageviews. When you get recommendations for something you already bought, many times it is because you looked at it, but they don't know nor care if you already bought it. In their words, it works really well and accounting for sales brings in a lot of needless complexity.
Another is based off of sales. They also don't care if you already bought it because according to them, it works well. I remember trying to point out to them that for some types of products (specifically consumable products) this would work really well, but durables not so much. They claimed otherwise, that although they couldn't explain it, it was entirely common for people to rebuy things like vacuum cleaners and TVs and kitchen knives. I did a tiny bit of research to show them why they thought that, and proved with a small segment (vacuum cleaners, I believe), that after you filter for returns and replacements, that the probability of sequentially buying two of the same vacuum cleaner was effectively zero. They asked me to do it for the rest of their products, but I didn't have limitless time to spend on helping another team, especially one with a PM who was a complete dick to me for having the audacity to make a suggestion that he hadn't thought of.
In all, I believe there are a dozen or so recommender services, each with their own widget. There are tons of people that think all of the recommenders have merits in some areas and drawbacks in others, and the customer would be better off if they merged concepts into a single recommender system. But they all compete for sales lift, they all think their system is better than the other systems, and they refuse to merge concepts or incorporate outside ideas because they all believe they are fundamentally superior to the other recommenders. Just a small anecdotal glimpse at the hilariously counterproductive internal politics at Amazon.
"Dear Amazon, I bought a toilet seat because I needed one. Necessity, not desire. I do not collect them. I am not a toilet seat addict. No matter how temptingly you email me, I'm not going to think, oh go on then, just one more toilet seat, I'll treat myself."
From Amazon's perspective, someone who just bought a toilet seat is much more likely than the average person to buy another one. Maybe they're doing interior furnishing work and put loads of seats on toilets. Maybe they're replacing a broken crappy plastic seat on one of their toilets with a nicer one -- might they perhaps decide to upgrade their other toilets as well?
And sure, I know that these are all pretty weak signals and that the toilet seat recommendations will probably go un-clicked. But Amazon has to do the best it can with not very much information, and if they have to hawk toilet seats to a hundred people who've already bought them to find the one guy who's remodeling several bathrooms, that's probably a really good gamble for them to take.
Google has the same problem. I was searching for information about Apache Chainsaw the other day, and at one point forgot to put the word "apache" in as a search term. Ended up with adverts for actual chainsaws for like a week.
I've always been frustrated about this as well, but I realized that it's often not Google's intention to care about this error.
We're the product, willing or not, and the metric some chainsaw companies care about is simply 'searched "chainsaw"'. Google is happy to sell our eyes based on that, whether or not it converts into actual sales.
In fact, for Google to deduce further and use ML to say "this user actually isn't looking for a chainsaw despite that search," and the reverse of "this user is looking for a chainsaw despite not searching it," probably wouldn't go over well for an advertiser paying for views. It turns Google into more of a black box, and would appear less trustworthy or understandable.
Well it might be true for certain products. But I find it funny how they recommend me my phone or my laptop. Those aren't things you usually buy in bulk.
> Here's a true statement: People who will in 2018 give birth to a child named Abigail are at least 5X more likely to give birth to a child in 2019 than people chosen at random.
> "Naming your child Abigail can't make you more likely to get pregnant."
> Again, failure to do the math.
> (You get a 2X there, for free, from the observation "If you had a child named Abigail this year you are biologically capable of having children; this is not true of no less than half of humanity. Now apply same insight to childbearing age and you're already at 5X+.)
This is terrible math. Nursing a child severely inhibits getting pregnant; without use of formula (or a substitute like goat's milk or what have you), it is extremely rare for children to be spaced only one year apart. It's rare enough that there's a special term for such children -- "Irish twins".
If you want to swagger your conditional probability, you should know that you've got to account for that in the probability of giving birth one year given that you gave birth the year before. You can't just account for the positive adjustments, ignore the negative ones, and then say "the odds are at least as good as they would be if the existing anticorrelations didn't exist".
> What's a SWAG for how often a purchase immediately goes wrong? Not right color? Fridge DOA? Shoot I mismeasured my kitchen? Wife just hates it? Call that 2%. If I fix it within a week, then 2% / 7 = 2.9e-3 probability of purchasing a new fridge.
> That's a 10X relative risk.
But of those listed options for "immediately goes wrong", zero of them would result in re-buying the same fridge. Wrong color, I mismeasured my kitchen, and wife hates it would cause you to buy a different fridge. Fridge DOA would cause you to refund the fridge and get a replacement.
How many people, faced with the expense of a fridge that hasn't worked out, figure the best course is to just write it off and buy a second copy, hoping that it will work out better?
> This is terrible math. Nursing a child severely inhibits getting pregnant; without use of formula (or a substitute like goat's milk or what have you), it is extremely rare for children to be spaced only one year apart. It's rare enough that there's a special term for such children -- "Irish twins".
The timespan for two separate births in 2018 and 2019 isn't 12 months, it is anywhere between 12 and 24 months. While giving birth twice in 12 months is rare, I would assume that giving birth twice in 18 months or 24 months is not nearly as rare.
Yes, giving birth twice in 24 months is normal, but that gives you a very narrow window within 2018/2019.
If you give birth in July of 2018, an 18 month spacing won't let you give birth in any month of 2019. That's still a very significant negative adjustment to "demographic factors get you to 5X+". Demographic factors aren't the only factors there are.
This is the birth rate (for 2016, in the US) reflecting the odds that a randomly-selected person, male, female, infant, or menopausal, will give birth in a calendar year.
> Fertility rate: 62.0 births per 1,000 women aged 15-44
And this is the birth rate demographically adjusted for the ability to give birth. It's about five times higher.
Since siblings born in consecutive calendar years are rare, we can only conclude that patio11's "true statement" is false by a pretty large margin. For it to be true, every birth in 2016 would have to have been to a woman who also gave birth in 2015.
I wish that, before asserting something is true, people would think about whether it's true.
> we can only conclude that patio11's "true statement" is false by a pretty large margin. For it to be true, every birth in 2016 would have to have been to a woman who also gave birth in 2015.
So this has been bothering me, because if every birth in one year is to a woman who gave birth the previous year, and the years have equal birth rates, and we ignore aging in/out of the "fertile" demographic, then the odds of giving birth in one year given that you gave birth last year are 100%, well over 5x the population rate.
The claim is that the odds of giving birth this year given you gave birth last year are equal to the odds of giving birth this year given you're fertile, or in other words that, if you're fertile, whether you give birth this year is probabilistically independent of whether you gave birth last year. This is quite clearly false -- your odds of giving birth in one year are much lower, given you gave birth the year before, than your odds of giving birth in any randomly-selected year during which you're fertile. But my argument above is wrong.
For patio11's claim to be true, 6.1% of women who gave birth in one year would need to give birth again the next year. (Stated equivalently, 6.1% of people would need to have an older sibling born one calendar year before themselves.)
Good call with that sanity check. Patio11's assertion definitely had that paradoxical lightbulb moment that makes you want to believe it is true (a la Malcolm Gladwell), but two data points were all that was necessary to expose it.
The refrigerator example is stupid. I'm talking about buying something from Amazon, so if I have buyer's remorse or it's broken I won't ever buy that model of the same thing again from Amazon. I would at most return it, which the ad might remind me of.
There's a certain truth to you being in that buyer's class now, because there is a certain population that buys laptops. Yet as he stated, I'm not in the buying cycle. I would probably be in the buying cycle again in 1-2-3-4-5 years from now. So immediately advertising for something you buy rarely (refrigerators more, laptops less, phones maybe more often) is still wasted ad space.
David Scotten's comment on that thread is a bit more on point: "There's the opportunity cost of not showing another ad in that slot, though. So they're showing you what they think is the ad with the highest expected value for them."
I think they don't calculate the opportunity cost right. They will advertise right away for something that is unlikely in their best slot in my emails.
It would be better for them to advertise other related electronics (headphones that work with bluetooth if you just bought a new phone with bluetooth, loudspeakers, computer screen, external keyboard, docking station, etc.) Those would actually be much more likely buys for me.
Funny actually how babies are mentioned. I never had a baby, nor do I have a girlfriend (something Amazon should probably know with some basic data science), yet they always advertise baby diapers to me, even to the point of sending me some as advertisement. I still can't figure out how that happened.
For me Amazon is a store where I always look for something specific. Only the book recommendations have really made me buy something I didn't think of before. I think there's a lot of lost potential here for Amazon.
My guess is that they are really bad at de-duping products. Product ids vs skus and all that jazz is historically not handled well in recommender systems, and those product not recognized as variants of already purchased products end up being the closest in the "graph" to be recommended so shoot to the top.
But what really drives me nuts is textbooks. The data is practically handed to them with ISBNs. How hard could it possibly be for someone to tweak the algorithm so it doesn't suggest to me every edition of every text book I've ever purchased? Beyond the customer annoyance factor, this is a business issue. There is a virtually 0% chance that any textbook suggestion will be converted to a sale. That takes up valuable suggestion real estate on the user's screen and keeps me from seeing a suggestion that I might actually be interested in buying.
I know this is a joke, but you're a magnitude more likely to buy something if you've already bought it once. The whole retargeting industry is built on this, but I agree - it doesn't necessarily make it useful.
Depends on the something. If I buy a carton of milk, I'm pretty likely to buy another before too long. If I buy a new refrigerator, probably not.
Besides, the point of advertising is to get you to buy things you otherwise wouldn't. It would be pretty pointless to show me an ad that says "you love eggs, you eat them for breakfast every day, so buy some next time you're at the supermarket", because I was going to do that anyway.
Well of course there are outlier product categories which are not repeat purchases, like engagement rings, funeral caskets, refridgerators, etc. That goes without saying, and if someone is paying to retarget those they're probably wasting their budget.
Not the same product, similar products in form and function. It works great for books! Pretty good for movies! Alright for CDs! Generalize out to everything.
I loved AMZN's cloud certifications. Not a Data Scientist / ML Developer, nor am I trying to be, but I love sitting for these exams as I believe the amount of study gives you enough credibility to have thoughtful conversations with specialists.
Looking forward to reading sentiment to see if the curriculum is industry relevant and worthwhile!
We can agree to disagree on that. The certs are useful if your business requires them for whatever reason but my standard take is any cert that doesn’t require hands on learning, such as RHCSA/RHCE/CKA/CCIE, tend to be a pointless exercise in reading comprehension. I say this as someone with several AWS certs.
Agree. At the very least the cert needs to have at least a simulation of a real-world environment or processes. Textbook reading or listening to lectures for pure information is the lowest tier of knowledge IMO because it is easy to simply memorize and regurgitate. The real skill is taking that information, converting it to knowledge and being able to apply it to real-world problems to create solutions. The cert will only be able to take you so far but it should at least put you in that direction.
Do you believe that those certifications are solid investments for any IT professional (from developer and up) or only for those that are looking to become a SysAdmin/Network Engineer? (forgive me if my terminology isn't spot-on)
Agreed certs are for talking the talk, experience is for walking the walk. Consulting companies are root of majority of cert takers from my experience.
The "credibility" point is important. As I say in all data MOOC-related HN threads, these types of courses will not make you an expert, but they're very good at giving a background on the terminology so it's less arcane.
Most MOOCs from what I've seen are so basic to even take seriously. For example if you are learning some course called physics 101 which claims to be equivalent to college level. The real physics 101 in the classroom will be comprehensive in the sense that you learn all the material over 16 weeks by studying 15 or so textbook chapters - the whole subject in detail. The MOOC on the other hand will only cover the first 3 chapters or cover all 15 chapters but only the absolute basics of each chapter. The worse part is people take these courses and think they have a physics 101 understanding of the subject when in reality they only have pre-physics 101 introductory knowledge. It becomes really obvious what their knowledge level is.
I like the idea of MOOCs but there is a very serious quality issue with them at the moment, even the MOOCs from big name universities are a joke.
I took Cryptography I on Coursera. It is a very comprehensive and stringent course, and therefore stands out among all the other watered down MOOCs. Yet the discussion forum and comment fields are filled with students angrily complaining how hard the course is and how the teacher is condescending in saying things like “it should be obvious”, “as is known to everyone”.
I guess these students ruined MOOCs for everyone else. Schools now know that when they put out an advanced course, not only do they reap little reward, but also alienate lots of people who think their intelligence is insulted.
The best thing to do with the highest ROI is your own projects for a portfolio, ideally things which haven't been done before (e.g. in the context of data science/ML, not the Titanic dataset or sentiment analysis of Trump's tweets).
Everyone has their own learning style, but I stopped doing MOOCs once I realized that doing work in a similar domain is more practical.
Thanks for your response. I agree that portfolio projects show more about a person than their certifications. I never liked taking exams in school anyways. :p
Regardless, do you think that there are any MOOCs/certifications that are worth investing in, if you were going to get one anyways? Say, if your employer is paying you to get one, etc.
"The digital courses are now available at no charge at aws.training/machinelearning and you only pay for the services you use in labs and exams during your training."
Any one has an idea how much it might cost going through the training?
I bet on the order of tens of dollars. I know that when I do my demo of amazon machine learning (another ML service they offer) it costs about $0.50/demo, which processes about 25k records.
I haven't had a lot of time to look at the offering, but can anybody tell me if these courses have any theory, or are they just for promoting specific AWS ML services without going into the underlying complexity?
I looked at what I could about the courses on the "Developer" path (mostly just the titles of the different sections of each course), and there's obviously a very heavy focus on AWS ML services. Also my general impression is that the courses are too short to actually cover in depth what they say they're covering. Probably good for an overview.
I haven't looked in detail but the description strongly suggests an applied ML focus (which would be very typical of IT certifications in general). In AI/ML, there's some overlap between applied AI and research AI--e.g. you're going to want to know something about statistics in any case--but I'm not sure the distinction is always appreciated.
Some online courses or groups of online courses have certs associated with them. Some don't. Whether those certs are worth anything from a job hunting perspective is going to depend on the cert and who is looking at them. Very situationally dependent.
I was specifically speaking with regard to this offering. AWS has a pretty mature certification program (though I think was recently outsourced), so I figured they would have some sort of certification track to motivate people to consume their educational resources.
Checked out data science, there's a good amount of general stats in there. It's also in their interest to educate people so that they could become customers once they understand the basics/theory.
I just flipped through that course. It covers just vectors, matrices, probability, and derivatives. A better (and faster) resource is to use chapters 2-5 of Deep Learning by Goodfellow et al, but the mathematical detail in that book may be offputting for beginners.
One of the hardest to use online learning platforms ever. I thought it was going to be a structured course from beginner to expert.
Of course Amazon doesn't make it that easy.
And the Free Digital Training 'enroll now' button does absolutely nothing. You'll spend most of your time choosing which of the 90 Machine Learning courses to do.
Glad it wasn't just me... a sea of popups, 4 noreply emails, and 15 minutes later, and I'm impressed by how convoluted the process of browsing the classes is.
I'm excited for this - I have stitched together my own courseplan to study machine learning, but this will be great to workback from, complement, or supplant in certain cases.
Hi, sorry I missed this message. You probably won't see this, but my plan is to do the following (you can Google the course names):
1. Learn math (I majored in history and sociology)
Classes
-------------
Linear Algebra: Foundation to Frontiers (UT Austin)
Linear Algebra, MIT OpenCourseware
Single-Variable Calculus, MIT OpenCourseware
Multi-Variable Calculus, MIT OpenCourseware
Probability Theory, Stanford CS109
2. Learn machine learning
Machine Learning, Stanford CS229 (taught by Andrew Ng)
Classes @ deeplearning.ai (also taught by Andrew Ng)
