
Ocado's burning robotic warehouse - zeristor
https://www.bbc.co.uk/news/technology-47160448
======
davb
I found the fact that they're using modified 4G over unlicensed spectrum to
control the robots pretty interesting. Latency, scalability and consistency
seem to be the key drivers. There's more on that at:

[https://www.mobileeurope.co.uk/press-wire/ocado-uses-
unlicen...](https://www.mobileeurope.co.uk/press-wire/ocado-uses-unlicensed-
lte-to-enable-the-march-of-the-warehouse-robots)

[https://www.cambridgewireless.co.uk/media/uploads/files/UK5G...](https://www.cambridgewireless.co.uk/media/uploads/files/UK5G_-
_The_death_of_IP_Fraser_Edwards_-_Why_LTE_didnt_cut_it_for_Ocado.pdf)

[https://www.silicon.co.uk/networks/voip/ocado-4g-network-
unl...](https://www.silicon.co.uk/networks/voip/ocado-4g-network-unlicensed-
sprectrum-185777?inf_by=5c5cec74671db8e90d8b499b)

~~~
sterlind
Any clue what radio IC they used? I didn't see the hardware mentioned in those
links. Can you tune a vanilla LTE baseboard to unlicensed bands? Or did they
use a software-defined transmitter with a custom LTE stack?

~~~
abainbridge
I believe most LTE chipsets can be tuned into any band you like, as long as
you design your own RF front end (antenna and circuitry to/from the chip) and
persuade the chipset manufacturer to give you a firmware update in such a way
that they aren't liable for you distributing illegal devices. It's probable
quite difficult to find manufacturer who is prepared to work with you if
you're only going to buy 50,000 modules. I guess being Cambridge Consultants
gives you contacts.

~~~
proaralyst
Given that Cambridge Consultants spawned Cambridge Silicon Radio you're
probably right.

------
ucaetano
> But let's not forget there's a good reason why data centres full of computer
> servers are often built inside icy mountains or under oceans. Electronics
> get hot.

Gell-Mann amnesia effect applies here. This is pretty bad reporting.

~~~
ddeokbokki
You mean to tell me my ap-south-1 instances are not actually inside Mt.
Everest?!

~~~
ucaetano
Nope, they're in an orbiting datacenter. You can't beat the dead cold vacuum
of space.

PS: You can, the vacuum of space is actually terrible for cooling, I was
making a joke.

------
SmellyGeekBoy
I appreciate that "robotic warehouse" makes for a good headline but I just
wanted to point out that only the part the retrieves the goods from the
shelves is robotic. Humans are still packing and dispatching the orders.

It's amazing how many people in this thread are looking at the headline and
jumping to the conclusion that there were no humans present at all.

~~~
teh_klev
> Humans are still packing and dispatching the orders.

Perhaps not for long, according to the article they're now training robot arms
to perform the packing tasks.

~~~
IshKebab
That is a _much_ harder task than just picking up boxes that are all the same
size. It's essentially still a research task that many universities are
working on.

I'd say we're at least a decade away from a robot that can pick up irregular
items like a person can.

~~~
lnsru
Is it that hard when you know, what are you picking? I mean, each box contains
only one type of objects. Weight, shape and package type is known.

------
kleton
That's hilarious that a journalist thinks that datacenters are "often built
inside icy mountains or under oceans".

~~~
abainbridge
It'd be funny if it wasn't entirely normal for the BBC's technology output.

~~~
setquk
BBCs reporting is nothing short of dire on everything at the moment. Random
people on twitter are better journalists.

~~~
SmellyGeekBoy
The BBC do seem to have realised this, considering a lot of their "stories"
recently are literally just cobbled-together lists of tweets from these random
people.

~~~
pja
IIRC BBC Online was deliberately gutted by the top brass to keep the
(Conservative) government happy because it was perceived as competing too
effectively with commercial news providers.

~~~
setquk
So basically not lining up with the Daily Express?

~~~
pja
Not so much about political positioning, but simply that the quality of the
straight news reporting was high & there were no ads, so viewers were flocking
to the BBC website in preference to other commercial news providers. This was
deemed unacceptable by the Conservative government during a crucial time in
the negotiation of the renewal of the licence fee, so the website was quietly
gutted to appease them.

That’s the story I’ve been told at least.

~~~
setquk
If that is the case then it’s a sad outcome and it shows political overreach.

~~~
NeedMoreTea
Broadly speaking everywhere the BBC did well the Tories took as reason to see
them as impacting on free competition and commercial interests. There have
been periodic Tory calls to privatise them for decades.

BBC online has been significantly limited twice, and BBC news more generally
once (One relatively recent Tory change required the BBC to take on the costs
of World Service that had always been Foreign Office until that point. Whilst
editorially independent, the FO could ensure certain regions received output).

It is quite sad, as there were many areas they were doing well in.

------
aerophilic
It is an interesting question: how do you plan for the unforeseen?

We as humans are very good at coming up with solutions to ambiguous/messy real
world scenarios... robots less so. By definition, the current set of robots
are programmed for specific situations.

As we automate more and more, I suspect we will have more and more “one off”
catastrophic events. Yes, once we experience them, we can “test” against them.
However it almost seems to argue for one amount of human in the loop no matter
what, if only to give an occasional “that seems weird” check.

Am I missing something? Yes we can make an argument for machine learning to
learn “nominal” and report off nominal... but that only works if you gave the
right sensors being inputted in the first place.

~~~
entity345
Fire in buildings and warehouses is not an unforeseeable risk. It is a major,
well-known risk.

~~~
aerophilic
Correct, this particular case is obvious (and they should plan for it).
However what I am trying to dig into is the “less obvious”. An example, snow
showing up in an area that hasn’t seen snow in the last 100 years. Or perhaps
someone “crashing a car” into your factory. My point, and my questions has to
do with the following at the core: “How do we replicate/approximate the
‘general’ knowledge that a human will inherently have?”

------
pja
So their award winning sprinkler system clearly didn’t work.

Will be interesting to read the investigation report & see what actually went
wrong.

~~~
janekm
A sprinkler system... for a giant lithium batteries fire? If it was an actual
water sprinkler system that may have been an issue.

~~~
pja
One hopes that they had thought of that, but who knows?

Looking at the warehouse videos, it’s clear that under the robots is a vast
sea of individual towers of plastic boxes, all packed in tightly. I suspect
that once that plastic was burning merrily, it would be very hard to stop &
the heat would start cooking off the lithium batteries in the robots above.
Hard to see how a sprinkler system can prevent fire from taking hold deep
inside those towers of boxes, if that’s what happened.

~~~
kylegordon
A fire detection camera system could have picked out the heat plume in a short
amount of time, and parked the robots as far away as possible from it. The
article even comments on how cold the warehouse was, so any heat from fire in
'the hive' should have been easily detectable.

My bets are that it started somewhere totally different and unrelated to the
robot system. Will be interesting to see why their system failed.

~~~
smileypete
Could the robots bring buckets of water to dump on the fire? :)

I guess there's a lot of isocyanate foam insulation, once that gets going no
sprinkler is going to help.

------
zeristor
Isn’t this what the learning curve is about?

Although we’ve heard of hover boards, and the rare steals bursting into flames
it is odd that battery robots on rails weren’t thought to be a fire risk.

That added to the fact that it wouldn’t be simple or safe to get to a fire in
the middle of the grid.

------
verytrivial
For some reason I really felt for the engineers hearing about this. They've
presumably lived and dreamed this for years. A bit like watching say a model
railway go up in smoke: sure, no-one is dying, but what a _waste_ of best-
efforts.

------
zeristor
I wonder how much building fires are a source of pollution in urban areas.

Whilst commuting into London Liverpool St I’d quite often see huge plumes of
thick black smoke from fires in tyre dumps, which must be a huge source of
pollution even with all the diesel lorries and cabs.

------
setquk
As an Ocado smartpass customer who is expecting an order today, this appears
to just be manifesting itself as a few items out of stock.

Slightly disappointed though if I’m honest. This was the future. And now it’s
going to be looked down on by every semi critical person as a complete failure
even though the actual picking process was working very well.

For those who don’t know, Ocado was the only supplier in the UK that pretty
much have a 1:1 correlation with what you order and what you get and haven’t
been sitting on a shelf for days. All the other supermarkets are picked from
local stores by hand and are quite frankly terrible with 2-15 items (from
experience) missing on every order and some things very near expiry dates.

Sainsbury’s for example substituted me some bread which was out of stock with
a fresh loaf expiring on the day it was delivered. The delivery turned up at
22:00...

~~~
benj111
Its not that night and day.

Ocado will for example substitute some things that I don't really need, and
aren't really substitutable. but if they don't have the nappies you ordered?
No substitute, nadda. And it isn't like you can wait a week for nappies if
you're running out.

Sainsburys I've always found ok, even for our Christmas shop, there weren't
any substitutes.

Where Ocado is better is search. If I search for 'juice' in Sainsburys I very
quickly get 'quinoa with juice of jojoba'. Ocado gives me actual fruit juices.

~~~
vidarh
In case you don't realize: If they substitute something and you don't want it,
you can just ask the delivery person to take that item back. I've not done
that often, but when I have they've never had an issue doing it.

(the real big thing for me is that I have a weekly shop set up, and if I
forget to look on their site, they'll send me something based on a tolerably
ok prediction of what I usually buy; it far from perfect, but good enough that
I occasionally doesn't bother log in on purpose, because I know it will be
close enough to what I need to be ok)

~~~
benj111
You can reject things you ordered too, like the £40 of toilet roll added to
reserve the delivery...... So I've heard of course.

~~~
setquk
This does work. I know someone who has done it a couple of times. I suspect
that there’s an alarm somewhere that picks out regular abusers though.

------
baybal2
I have some criticism for Ocado. They were a freaking successful business in a
no brainer industry... and then they decided to make robots out of a sudden.

Not only their risky robot venture cost them a lot, but they ended up burning
more by operating them than regular warehouses.

Why?

------
arde
Next time their plans should include some firefighting robots.

~~~
Symbiote
The article says they had a world class sprinkler system.

[https://ocadoengineering.com/ocado-engineering-awarded-
prest...](https://ocadoengineering.com/ocado-engineering-awarded-prestigious-
highly-protected-risk-award/)

------
matthewfelgate
Was it arson?

------
Theodores
I don't really care about this story. I don't feel compelled to care as it is
not like other warehouse fires, e.g. the Asos one where you wondered about
people and their jobs. When it is just robots and when you know the customers
will just place their orders with one of the other click and pick outfits,
e.g. Waitrose, Sainsbury's or whatever there isn't that requirement to give a
damn.

In retail there are thieves who don't care about stealing from big companies
like how they might care about stealing from small businesses as there is an
assumption everything is insured. I am sure Ocado have insurance and I don't
care too much about the shareholders of the insurance companies as I am sure
they have their losses 'insured'.

File under 'expensive mistake' rather than 'human tragedy'.

~~~
Pharmakon
There are some holes in your argument. Most glaringly, this fire presents a
risk to everyone who has to fight it, and anyone else in the area.

~~~
bpye
If, and I know in this case it wasn't, your no-people warehouse was far from
homes and was on fire, wouldn't one option be to just let it burn itself out?
No risk fighting that fire...

~~~
Pharmakon
Then I’d agree, but when you think of how far burning cinders can be carried
on the wind, it would have to be very very far away. There would also need to
be some legal framework to allow firefighters to say, “No thanks, we’re not
risking our lives for your property, and we are 100% sure there isn’t a kid or
homeless guy who snuck in for the night.”

[http://johnsalvatier.org/blog/2017/reality-has-a-
surprising-...](http://johnsalvatier.org/blog/2017/reality-has-a-surprising-
amount-of-detail)

