
Inside Amazon Go - atombender
https://www.nytimes.com/2018/01/21/technology/inside-amazon-go-a-store-of-the-future.html
======
samwillis
Waitrose in the UK has had something called "Quick Check" for maybe the last
10 years. On the way into the store, you swipe your "My Waitrose" card and
collect a barcode scanner. You then scan and bag everything on the way around
and return your scanner to a dedicated terminal (which generally have no
queue) and very quickly pay. More recently they added the ability to use your
smartphone as the scanner so that you don't even need to pick up a barcode
reader. They are only one step away from allowing you to pay with your
smartphone and walk out... although as it's at the terminal they ID you if you
have any restricted items that seem unlikely. Every once in a while (I have
had it once in 5 years of shopping there weekly) they will flag you for a
"rescan" to check you scanned everything.

It's a brilliant system and doesn't require hundreds of cameras and machine
learning.

~~~
raz32dust
That's something Walmart or anyone else could do. Doing it without
compromising on losing items to theft, that's where Amazon is trying to create
its moat, and that is where their competitive advantage lies. They have the
technology to go completely employee-free. So while it was enlightening and
impressive to read about the Waitrose system, it makes it sound like Amazon is
being naive and did not think about this simple solution, which I believe is
not the case. Amazon could easily do that, but then everyone else could easily
copy them. This technology though, Walmart is going to have a really hard time
replicating it.

~~~
tanilama
Someone needs to fulfill the items and chiming in under unexpected events, and
clean the store. Cashiers might go, but the rest of the crew that needed to
operate a store will still be there.

One advantage I would imagine for Amazon Go is going to be the experience for
the customers, that without the need to queue and even forget checkout at all,
is great. However I doubt how much they could save from going cashier free.

~~~
raz32dust
This is true, to be sure. And all this tech will also need expensive
maintenance. I am not sure how much they'll actually end up saving. I'd also
miss the happy experience that you sometimes get when dealing with a cheerful
crew. The ambience and crew is one reason I shop groceries almost exclusively
from Trader Joe's.

------
hn_throwaway_99
I know the standard reaction from a lot of HN folk will be to celebrate this
progress: it's faster, more efficient, and no one likes to waste time waiting
in line.

Still, though, I think Amazon's automation of the retail experience is helping
to create a society where there is even less real social interaction, and I
think it's a factor (if even a small one currently) in the increasing
tribalism in American society.

Throughout history, "the marketplace" had been one of the major centers of
interpersonal interaction, and these days for many people it's possibly to
completely avoid ever having to go to the store. I'm not sure that's a great
thing.

~~~
threatofrain
The marketplace today is hardly the place for meaningful interaction, whether
with employees or other shoppers. What it seems like to me is unpaid emotional
labor. To have interesting social time, we need less hours at work and more
public (non-commercial) spaces to meet. For example, where are children
supposed to meet?

~~~
dennisdamenace
I often learn a lot at the check out lines here in Oregon. People chat,
exchange information. Often banal, sometimes valuable. Always nice.

~~~
sotojuan
Most HN users live in urban areas. Here in NYC your average check out line is
people on their phones and/or with headphones on. And before you say NYC is an
outlier, it was the same when I lived in semi-rural suburban Texas.

~~~
rainbowmverse
>> _Most HN users live in urban areas._

I didn't know YC published HN's demographics. Do you have a link?

~~~
isitwhatitis
>80% of the American population lived in what is called an urban area in the
2010 census [1]. While their definition of urban is looser than you might use,
still >60% lived in 'cities'[2].

Unless the wealthier, more connected, more educated readership of Hacker News
strongly bucks every trend suggested by their demographics, it's pretty safe
to say we mostly live in urban areas.

[1][https://www.citylab.com/equity/2012/03/us-urban-
population-w...](https://www.citylab.com/equity/2012/03/us-urban-population-
what-does-urban-really-mean/1589/) [2][https://www.census.gov/newsroom/press-
releases/2015/cb15-33....](https://www.census.gov/newsroom/press-
releases/2015/cb15-33.html)

~~~
mulmen
How do you know HN is made up of only Americans?

~~~
chibg10
The demographic trends in the GP aren't unique to America. They're more or
less the rule in most developed countries, and I think it's safe to say most
HN users probably come from developed countries.

~~~
mulmen
Can you cite a source for that? How can you infer the demographics of HN in
the absence of data?

~~~
isitwhatitis
Where I come from, it's pretty poor form to ask people to cite sources for
obvious statements, without any other insight or commentary. After all, you're
asking them to do work for you, but not taking basic steps to suggest you
might appreciate their effort.

The preferred option is to do a little reading of your own beforehand, and to
share whatever it is you know that makes you think the obvious answer might
not be right.

------
tuna-piano
I love this - shows why Amazon keeps growing.

Seems like the total investment in this store/technology is probably in the
single digit digit millions (or maybe low double digit).

If it doesn't work? Lose, ~$10 million. But if it works there's likely huge
upside. It's just a smart bet for a big company. Sometimes your bets turn into
AWS, and sometimes they turn into the Fire phone. Most companies I've worked
at are very risk averse, and failure on startup type projects is not really
accepted the same way.

The checkout system was the failed promise of RFID[1], and maybe the future is
still with RFID. But maybe it's with vision, and if it is, then Amazon seems
well positioned.

BTW, the rudimentary math I'm doing. Let's say the store saves two checkout
people, working on average 20 hours a day for 365 days a year, who each
average cost of $15/hr. $15/hr _20 hours_ 365 days*2 people = $219k savings
per year. How much does the system cost to setup in a new store? How much
extra effort is there to keep food items organized? If you need someone
checking IDs, managing errors, etc, is there really savings? Maybe the only
way this really scales is if you can do this for full size stores?

[1][https://www.youtube.com/watch?v=eob532iEpqk](https://www.youtube.com/watch?v=eob532iEpqk)

~~~
bachaco
>Seems like the total investment in this store/technology is probably in the
single digit digit millions (or maybe low double digit).

It was much more than that. They bought Whole Foods which was worth 13
billion.

~~~
mft_
Amazon Go had very little to do with Whole Foods.

It may, one day, but to link the cost of buying Whole Foods to the cost of
development of Go is inaccurate.

------
dawhizkid
Anyone in NYC who regularly shops at Trader Joes or Whole Foods knows how
badly this is needed. There are literally two employees at Trader Joes whose
job it is to hold a sign that says "End of Line" because the line is always
almost out the door on weekends and weekday evenings.

~~~
iagooar
Why not use these people as cashiers?

I wonder if this endless line thing is something typical in the US. I've been
to many European countries and other than before Christmas, you never feel
like staying in line is something to even think of, because the longest you
usually wait is maybe 5 minutes.

In Switzerland, now that in many shops you get to scan the cart yourself
(after shopping or during it) there is just no congestion at all, even on a
busy Saturday.

How is it in your countries?

~~~
seangrant
Seattle area here. All my life I remember waiting in lines 10-20 minutes if
you go to a store on a busy day (fri/sat/sun). It's just simple math sometimes
where there's so many people with $400+ carts trying to check out and cashiers
can only scan so fast. Everyone with medium or small sized carts have to wait
in this.

I've seen great strides in recent years to advance self checkout technology
and its user flow. Walmart is my favorite example of this, my local one having
almost half of the area devoted to self checkout. It seems to have been
successful for them so far. I don't see anyone having a hard time operate them
and perhaps more surprising, it's filled me with the idea that I can pop in
and out of a big store on a busy day to just buy 1 item.

~~~
chrisseaton
> It's just simple math sometimes where there's so many people with $400+
> carts trying to check out and cashiers can only scan so fast. Everyone with
> medium or small sized carts have to wait in this.

Why don't the customers scan the items themselves as they put them into their
carts while at the shelves, rather than waiting until they get to the
checkout? That's what we do where I live in the UK.

~~~
lykr0n
Stop n Shop in the NYC Tri-State area has this, but they are the only store
that I've seen that does. I'd assume it has a technical barrier to
implementation, and large-ish up front cost.

------
makomk
"Amazon won’t say much about how the system works, other than to say it
involves sophisticated computer vision and machine learning software." Why do
I have the feeling this probably actually translates to humans staring at
video feeds a lot of the time?

~~~
Guest9812398
I imagine it's similar to the below video.

[https://www.youtube.com/watch?v=yeS8TJwBAFs](https://www.youtube.com/watch?v=yeS8TJwBAFs)

~~~
cm2187
I wonder how resilient it is to real life shoppers who pick things, then put
them back at the wrong place.

~~~
TheEzEzz
Jordan from Standard Cognition here (that's a video of our autonomous checkout
system). We don't depend on the shelf location an item came from. You can put
things back anywhere you like and the system behaves just fine. It's one of
the advantages of using a pure vision system as opposed to a sensor fusion
approach with sensors on the shelves.

------
janekm
There is a company opening similar types of supermarkets and corner stores in
China. But they have taken it a step further, there aren’t even any gates. The
experience feels really surreal... you just pick something off the shelf, scan
it with the app, pay in the app (helped by the seamless mobile payment
services in China), and walk out. The first time I tried it I couldn’t help
finding an employee to show them I paid. The second time I just walked out
after paying, but it actually felt uncomfortably like shoplifting (similar to
what’s described in this article).

I presume the shops are set up as a sort of VC-funded experiment that will
accept a high level of theft in return for a lot of useful data and early
positioning in this type of retail.

The other interesting aspect is that the larger of the stores is set up very
much as an experiential shopping environment, including a meat counter with
chefs that will cook a steak or burger for you, and the same for sea food (you
can buy shrimp or a lobster and they will cook it for you so you can eat it in
the store, it is very popular).

~~~
morgante
> But they have taken it a step further, there aren’t even any gates... scan
> it with the app

I consider that strictly inferior, not "one step forward." The genius of
Amazon Go is that _no_ scanning is involved. You just take things off the
shelf and they're automatically charged to you as you walk out.

~~~
janekm
That’s true, though one advantage for the consumer is that at least you get a
chance to confirm that you really did want to take that $40 lobster ;) For the
store an advantage is saving space, the Chinese ones are in really expensive
prime real estate areas.

~~~
aioprisan
You get to confirm it before checking out, there's an app listing items added
to the shopping cart.

------
heinrichf
Techcrunch article with additional technical details:
[https://techcrunch.com/2018/01/21/inside-amazons-
surveillanc...](https://techcrunch.com/2018/01/21/inside-amazons-surveillance-
powered-no-checkout-convenience-store/)

> ordinary RGB cameras, custom made with boards in the enclosure to do some
> basic grunt computer vision work, presumably things like motion detection,
> basic object identification, and so on. They’re augmented by separate depth-
> sensing cameras (using a time-of-flight technique, or so I understood from
> Kumar) that blend into the background like the rest, all matte black. The
> images captured from these cameras are sent to a central processing unit.

> In addition to the cameras, there are weight sensors in the shelves, and the
> system is aware of every item’s exact weight

~~~
srtjstjsj
Imagine when Amazon sells this system to Palantir and police departments for
use all over town.

~~~
callumjones
And does what with it?

------
Kiro
Am I the only one living in a country where most stores have self-checkout
already? The comments here make it sound like it's either Amazon Go or having
to pay through a cashier.

Jobs and social interaction have already been elminated by self-checkout. The
revolution with Amazon Go is that you don't need to scan anything yourself,
not that it removes the cashier.

~~~
seanmcdirmid
Many stores in the USA already have self checkout, and have had it for a
decade or so. Go just eliminates the checkout process all together.

~~~
pcurve
Maybe I'm weird, but I enjoy self checkout line. Picking up each item, looking
for barcode, scanning it, and bagging, is kind of fun activity to do every
once in awhile. The only lame part is, "please place the item in the bag"
mechanism.

~~~
4684499
Switching bags, finding barcodes and placing the goods at right angle so it
can be scanned, all of these makes me feel like being trained for unnecessary
skills, it's repetitive and time consuming, thus incredibly annoying.
Sometimes things went wrong, I had to ask for assistance (yeah, more
interactions please.) I just stood there like I'm a 70-year-old man who isn't
good at interacting with computers (well hello?), watching the assistant guy
input the barcode number manually. After all that, I had to do the whole card
payment process myself, while people stared at me waiting for free self
checkout slot. Ugh!

~~~
Kiro
> finding barcodes and placing the goods at right angle so it can be scanned

You should do that anyway to help out the cashier. It's common courtesy.
Amazon Go obviously gets rid of it though.

~~~
4684499
> You should do that anyway to help out the cashier. It's common courtesy.

O_O

To be honest, I've never done that, and I'm surprised that It's common
courtesy. I don't think I've ever been in a situation where I had a chance to
help the cashier to find the barcode. What I do myself is placing items on the
conveyor belt, putting items into the bags after checkout. Was I being
impolite? What's the cashier's job then? Pressing the scan button...?

~~~
morgante
It might be courtesy, but it's certainly not "common." Times like these are
where I feel like half of HN must be from a totally different planet.
Searching out barcodes is fun and common? What?

------
Flammy
When I installed the app it mentions you can let others buy stuff from your
account with a single phone on entry(kids, friend, etc). There is an animation
that shows you tap your phone in, send your friend in, then re-tap yourself
in.

Didn't see this mentioned anywhere, and I thought it was kinda neat.

~~~
rtkwe
Kind of confirms what I'd always thought, that they'd associate a face/person
with an account/shopping basket then track that through the rest of the store
instead of doing any phone based tracking.

------
eloff
No shopping carts? Maybe for a convenience store that's ok. I shop at Costco
without a car, and I bring two large carry bags. But I don't carry them around
inside the store unless I'm just buying one or two things. I buy a lot of
heavy things (oh god those watermelons, never again.) And I don't want my arms
getting tired in the store. It's bad enough for the 1.5km I have to walk back
to my place.

~~~
symlinkk
For the first time in history we have a grocery store without cashiers, and
you’re nitpicking that there’s no shopping carts?

~~~
dEnigma
What does one thing have to do with the other? Shopping carts are useful even
without cashiers.

------
blibble
not entirely serious, but not entirely joking: in a world where all stores are
like this; you drop your phone and it smashes, can you still buy food?

(what about people without phones?)

~~~
Tiktaalik
This is totally a real problem. I just celebrated my grandmother's 90th
birthday. She still lives independently and regularly goes shopping. She
doesn't own a mobile phone, let alone a smart phone.

In a world where services that require phones are dominant, choice narrows for
elderly and low income persons.

~~~
rtkwe
There's a lot of ways around the broken/lost/phone-less populations in a
system like this. The phone doesn't really do any more than provide the
initial authentication and receipt delivery. If these really took off and
started replacing normal systems it'd be really easy to add a receipt printer
and to add a card reader for a 'members card' that does the same thing as the
QR code on the phone.

------
FLUX-YOU
Attacks I'd like to try but would get picked up by the cops if I did:

\- Carry empty bags/containers of cheaper items in the store and place
products inside these before pulling them off of the shelf.

\- Pull products off of the shelf and give them to a friend. Place identical
empty bags/containers back to re-credit your account while your friend never
has the item debited and walks out.

\- Drape a blanket over yourself and remove products from the shelf and put
them in your bags so the system never sees items removed from the shelf

~~~
adrusi
Just save your containers from your last visit, bring them with you in your
shopping bags, pull an item off the shelf and replace it with an empty
container from your bag so that it looks like you changed your mind.

~~~
fooblitzky
That's a store selling food for public consumption. Stocking the shelves with
old containers seems like it might carry some health risk? Or Amazon might not
be very happy about what you are doing to the image of their store (and
they'll know who you are because of the cameras.) Seems like a big risk to
take to save a few dollars on lunch.

------
avg_dev
here I was thinking this was about bringing Go to AWS Lambda :)

~~~
jhwang5
They announced support for Go in AWS Lambda a week ago

~~~
avg_dev
That was my point; I was like "wow it's in the Times, programming really is
going mainstream." Of course I was wrong!

------
andrewseanryan
Welllll. Now that idea for a Smart Shopping Cart that I drew up 10 years is
completely obsolete hahaha.

Like it or not, this Amazon Go store is a model for all future brick and
mortar establishments. In 20 years we will recall the good old days when we
waited in line and used cash much like people look back at the time when
people sent cards and letters before email.

~~~
srtjstjsj
The smart-cart seeems a much simpler, cheaper solution than the smart-store,
doesn't it? Scanning cart items is trivial in comparison to tracking objects
moving through the whole store

~~~
sytelus
Yes, they also eliminate edge cases like customer taking item from shelf A and
putting it back on Shelf B. Also they are easily deployable in any stores
without putting in lots of cameras all over.

------
seanwilson
I haven't seen anyone discuss the technology...can this really be reliable? It
mentions cameras above the shop floor and machine learning. What do the
cameras look for? Are there special visual markets on the products to make
this task more robust? If not, there surely must be caveats?

Are RFID tags not a more elegant solution? Or are they too expensive?

~~~
rtkwe
RFID tags have a lot of problems among them that unless you get every
manufacturer on board for labeling all the products in your store you're going
to have to spend a lot of time labeling products before shelving. It's also a
continuous cost per item either from the manufacturer or from the in house
labeling process where a static system like Amazon Go can track without adding
a per item cost. It's also not good enough to be the only system in a store
because it's easy to block the signal using a foil lined bag so the store will
already need another system at least as good as RFID at tracking things coming
off the shelves.

As for what they look for that's the secret sauce but there's a few things
that are pretty clear from descriptions. They seem to be using mainly (only?)
cameras for the identification. Luckily branding is pretty prominent so they
have a pretty convenient thing for their item labeling to look for in the
feeds. For cart shopping there's some simple things like tracking people with
their faces from when they enter then adding items to their cart when they
move an item outside a defined shelf region.

------
yodon
Wondering if the grand opening will lead to a spontaneous HN and Deep Neural
Nets meetup at Top Pot Donuts on 5th tomorrow.

------
chiph
I wonder what would happen if you reached through a gap in the products
(perhaps one is a big seller and the shelf location is empty) and pulled an
item from the back of the next product's location. Like - reaching where the
strawberry Pop-Tarts are normally stocked, to remove a box of the cinnamon
flavored. (Why would you do this? Normally the older product is on the front
of the shelf so it gets sold first)

Your hand would be out of view of the ceiling-mounted cameras so it can't see
directly (covered up by the shelf above). Your arm would be in the strawberry
opening but you'd pull out a cinnamon. Is it's vision system good enough to
tell the difference from the packaging, or does it just go off position when
you pull it off the shelf?

------
panarky
_There were a little over 3.5 million cashiers in the United States in 2016 —
and some of their jobs may be in jeopardy if the technology behind Amazon Go
eventually spreads._

Why is this the knee-jerk response to technological progress?

True progress isn't forcing millions of people into low-wage routine work.
True progress would emancipate people from this drudgery entirely.

Isaac Asimov questioned why the goal of both the left and the right is always
"full employment":

 _" The goal of the future should be full unemployment, so we can play. That’s
why we have to destroy the present politico-economic system."_

~~~
resu_nimda
Because the politico-economic system Asimov referred to is still intact. Our
society does not use technological advances to emancipate the general
populace.

It would be amazing if McDonalds fully automated their stores and allowed
their current employees to work one day a week for the same pay they receive
now, but we both know that's not how it would work out.

~~~
icelancer
>> Because the politico-economic system Asimov referred to is still intact.

And it always will be until there is stimulus to change. Government - large
systems in general - don't change the other way.

------
akhilcacharya
I used it a few times a few months ago, amazingly cool technology. Closest
feeling to magic I’ve seen recently.

~~~
21
Next level magic: you enter the shop and a little robot hands you a bag with
what you were going to buy anyway.

------
chiefofgxbxl
Given Amazon's knowledge in warehouse operations and supply chain, I wouldn't
guess they plan to use this technology widespread in grocery stores. At that
point, they could take a simpler approach than all the cameras and computer
vision, by instead treating a grocery store like a warehouse or vending
machine (no human shoppers like other grocers today who offer delivery).
People enter their orders online, and stop by in person to collect their cart
at the front desk. All the food would be behind closed doors. Only downside to
that is customers who want to leisure shop and browse for new items.

~~~
wickawic
I don’t see this being viable. Most people shop with at least a small amount
of whimsy. This is why end-caps and samples and grocery store psychology
exist. And I think this is a feature, not a bug: there is something satisfying
about picking out the right bananas, realizing Brussels sprouts would go well
with dinner, substituting ingredients on the fly, etc.

the online shopping experience has none of this, which is why I think grocery
delivery has not yet taken over. Food is visual and visceral, and doesn’t lend
itself to drop-down menus.

~~~
taurath
The whole store is designed for you to run back and forth finding and
discovering items.

------
Markoff
this seem like really backward thinking, the future is to eliminate visiting
shop, not simplifying checkout process

I don't want to step in shop at all, I want my grocery delivered to my door
and the delivery guy/drone can on the way out throw my trash. I was doing this
in China (it helps to live in city with concentration of 20 million people),
they even offer the throwing trash service though didn't went further into
that step

------
ShotgunJed
Interesting. Though I would expect that it would be at least a decade or two
before this reaches other countries outside of the US.

------
Disruptive_Dave
Less human interaction, less discomfort, easier to spend money faster. Sounds
so wonderful! /s

------
jwiley
Next step in operation Foie gras: Amazon connects a tube to your mouth and
pushes corn through it, automatically deducting the cost from your credit
card. Totally effortless shopping

------
spking
No paywall: [https://outline.com/KhsPDb](https://outline.com/KhsPDb)

------
matte_black
Can’t wait to try this experience. I will be flying up to go see it myself and
purchase a few things.

------
dingo_bat
Get ready for some insanity in the next decade as this tech is adopted by
walmart.
[https://i.redd.it/shbv27xwmzoy.png](https://i.redd.it/shbv27xwmzoy.png)

~~~
djsumdog
Huh. Is there a year/source for that data?

It's interesting Boeing out employees Amazon still in Washington. I also
didn't realize Intel's Hillsborough office was that big! At least compared to
their office in Folsom.

~~~
dingo_bat
No source except a random whatsapp message ;)

------
aviv
The innovation in rethinking the convenience store is not only in the
technology, but also the fact that they're likely going to not sell nasty gas
station snacks.

~~~
phillipcarter
The latter point isn't an innovation. My local grocery store (not a big
franchise) doesn't sell nasty snacks, either. They've been in business a lot
longer than Amazon.

~~~
aviv
The innovation part is how this Go store has the power to bring healthy snacks
into general convenience stores and gas stations and getting the disease
causing ones to be pushed out.

~~~
pavel_lishin
Why are you patting Amazon so hard on the back over this hypothesis? The
article didn't mention anything about it; Walmart, Target, 7-11 all have the
exact same power, but don't exercise it. Why do you think Amazon Go will push
healthy snacks over deep-fried candybars?

~~~
aviv
Whole foods. Amazon is going upmarket, not down.

~~~
heimidal
There's no evidence that Amazon bought Whole Foods _because_ it's an upmarket
grocery store and avoided others. It's very likely that Amazon will eventually
buy a "normal" grocery chain and serve as large of a population as possible.

------
ckdarby
HN stop supporting sites that block information with paywalls

~~~
grzm
The HN policy on paywalled articles in general is clear. 'dang has commented
specifically on the NY Times as well.

[https://news.ycombinator.com/item?id=16003345](https://news.ycombinator.com/item?id=16003345)

~~~
ckdarby
I don't know of any work around with the recent changes they did.

~~~
grzm
It's fine to ask for workarounds. It looks like they still support a limited
number of articles for free. Others have already posted workarounds in this
thread.

------
mhb
No shopping carts? If that's not a signal that this is, at best, half-baked, I
don't know what is.

~~~
jvagner
It says right at the beginning the concept is like a 7-11. Are there carts in
a 7-11?

~~~
mhb
Hmm. You're right. Missed it.

------
ggm
If we're clever enough to empower people to be responsible to shop without
human oversight to charge, why don't we stop including money in the
transaction and instead of shopping, we're just taking what we need?

I dont understand what the role of money is in this situation if its taken to
its logical conclusion. I bet Jeff Buysus wants to keep it there, however..

~~~
nyolfen
galaxy brain

------
sAbakumoff
That is terrible. Not only it takes away jobs, it also gives Amazon the
unprecedented access to the sensitive information about customers. I would
never go to this store. Fortunately something like this will never happen in
our beautiful Europe.

~~~
ryanwaggoner
It definitely will. Self-checkout is widely available in Europe. The laws
about data and privacy will be different but this is a customer experience /
labor cost play, not a ploy to get data.

~~~
sAbakumoff
Self checkout is a way different from thousands of video cameras that track
all your moves, selection habits, etc.

~~~
criddell
There's a pretty good chance that stores in Europe have video cameras already
there tracking where customers move, what they look at, what expressions are
on their face when they read signs, etc...

~~~
sAbakumoff
I highly doubt so. The data privacy laws are pretty strict here.

~~~
21
They are pretty strict in who you share the data with.

