
Luxonis wants to use computer vision to protect cyclists - seventytwo
https://www.siliconrepublic.com/start-ups/luxonis-computer-vision-bike-cycling-safer-technology
======
konker
"So the final product is a multi-camera, rear-facing bike-light, which uses
object detection and depth mapping to track the cars behind you - and know
their x, y, and z position relative to you (and the x, y, and z of their
edges, which is important).

What does it do with this information?

It's normally blinking/patterning like a normal bike light, and if it detects
danger, it does one of two actions, and sometimes both:

1\. Starts flashing an ultra-bright strobe - to try to get the driver's
attention

2\. Initiates haptic feedback - to let the person riding the bike that they're
in danger

3\. Honks a car horn. This is if all else fails, and only happens if you are
in certain danger"

Source: [https://discuss.luxonis.com/d/8-it-works-working-
prototype-o...](https://discuss.luxonis.com/d/8-it-works-working-prototype-of-
commute-guardian)

~~~
lnsru
I see in Germany older folks often using a mirror on the left handlebar. Once
I borrowed such bicycle, it was somehow weird, but I got used to that. Such
device identifies cars and dangerous situation very well. Zero false
positives, works pretty well in the night too. No batteries needed and costs
between 5 and 15€.

~~~
Luxonis-Brandon
Yes, mirrors are a fantastic solution. And for getting the attention of the
distracted driver (as sometimes the dynamics of the bike preclude maneuvering
out of the way in time), LOUD bicycle is a great solution:
[https://www.kickstarter.com/projects/lansey/loud-bicycle-
car...](https://www.kickstarter.com/projects/lansey/loud-bicycle-car-horns-
for-cyclists)

------
u801e
The real problem is that cyclists typically ride near the edge of the lane or
road rather than in the center of the lane. This puts the cyclist outside of
the foveal field of view for the motorist, which means that they're less
likely to see them in time to avoid a collision.

~~~
laurieg
Unfortunately, where I live bicycles keeping to the side of the lane is
required by law. Even in situations where this isn't required, drivers can
easily become aggressive towards slower vehicles such as bicycles.

Bicycles often fall through the cracks: not quite first class vehicles but not
afforded the same protections as pedestrians.

~~~
u801e
> Unfortunately, where I live bicycles keeping to the side of the lane is
> required by law.

In the US, most vehicle law is based on the UVC (uniform vehicle code)
template. While that does contain a clause that cyclists must keep as far
right as practicable, it lists a number of exceptions. One of those exceptions
is a "substandard width lane"[1]. The law defines it as a lane that's too
narrow for a bicycle and another vehicle to safely pass side by side within
the lane.

Most general purpose lanes are 10 to 12 feet wide. A cyclist can range from 2
to 3 feet wide (depending on the type of bicycle) and require a minimum of 4
feet of operating space[2] with 5 feet preferred.

Even with the cyclist operating at far right as practicable and taking the
minimum operating width into account, they take a little over 3 feet in
roadway width. Given that most states have a 3 foot minimum distance passing
law and the fact that most motor vehicles range from 7 to 8 feet in width
(including their side mirrors) and commercial vehicles are around 10 feet
wide, it's not possible to safely or legally pass a cyclist while remaining
within a typical 10 to 12 foot wide lane.

[1]
[https://law.lis.virginia.gov/vacode/title46.2/chapter8/secti...](https://law.lis.virginia.gov/vacode/title46.2/chapter8/section46.2-905/)

[2] [https://njdotlocalaidrc.com/perch/resources/aashto-
gbf-4-201...](https://njdotlocalaidrc.com/perch/resources/aashto-
gbf-4-2012-bicycle.pdf) (page 52)

------
djrobstep
This seems like a good example of the problems with Silicon Valley brain,
where we try to solve problems by hacking around them, rather than solving the
actual problem, which is poor cycling facilities, easily solved by installing
separated bike lanes.

~~~
sinieovercosie
But you can't expect every road to have a dedicated bike lane, right?

~~~
hannob
In Copenhagen or Amsterdam that's just the case. (Not literally _every_ road,
they don't have bike lanes in residential areas, but almost every major road
with significant traffic has a bike lane in these cities.)

~~~
flipchart
Not just Amsterdam, the whole of the Netherlands is like that. But more than
that, there is the _cultural_ aspect that cyclists and pedestrians are the
number 1 priority on the road, then cars. None of these fancy hi-tech
solutions work if the drivers don't care, and the laws don't protect cyclists.

------
hwnut78409
Seems like an overall well-intentioned founder story. Wondering if there are
Luxonis people out there who can address the below?

Would like to give the benefit of the doubt here to the CEO and Intel, but
being weary of Intel’s past “hacker friendly” blowharding and subsequent total
abandonment of the hacker community with the Edison, I do wonder whether Intel
is again playing the mafioso; the wording of this quote from the Luxonis CEO
comes off, to me at least, as kissing the ring,

> “DepthAI, powered by Intel Movidius Myriad technology, is making it
> tractable.”

Also wonder about the nature of the open source software the article refers
to,

> The Luxonis CEO said that the SoM works with a wealth of code that the
> company has open-sourced through an MIT license.

Is the software running on the Myriad X itself open source, or just the SDK
interfacing with it from the host machine? If the former, pretty amazing they
were able to get Intel to agree to this, as I imagine the chip details are
under strict NDA. If the latter, it seems pretty disingenuous to refer to the
SOM software as “open source,” at least not with a footnote that the most
interesting part (the spatial depth calculation) is not, presuming that’s done
on the Myriad X.

~~~
Luxonis-Brandon
Thanks for the kind words. Brandon the founder here. :-)

So although I could see how that quote could appear like that, the quote is
not kissing the ring. We scoured the whole semiconductor start up scene to
find a chip that could be used for this.

There are only 2 chips (as I'm aware, as of this writing) that can be used in
such a way: 1\. Intel Movidius Myriad X 2\. Inuitive NU4000AI.

And the Inuitive, until super recently (i.e. 1.5 years after we have already
built hardware off the Myriad X) was not ready to be used (had tape-out issues
at the fab).

So this made the Myriad X the only chip that (1) had the capabilities needed
and (2) was actually available and in production.

In terms of our open-source nature, the latter is what we've implemented -
where we have closed source binaries running on the Myriad X - which then have
a slew of open-source counterparts on the host. Sorry if this came off
disingenuous. Do you have advice on how to phrase it in a better way?

[https://github.com/luxonis/depthai-
hardware](https://github.com/luxonis/depthai-hardware) \- DepthAI hardware
designs themselves.
[https://github.com/luxonis/depthai](https://github.com/luxonis/depthai) \-
Python Interface and Examples [https://github.com/luxonis/depthai-
api](https://github.com/luxonis/depthai-api) \- C++ Core and C++ API
[https://github.com/luxonis/depthai-ml-
training](https://github.com/luxonis/depthai-ml-training) \- Online AI/ML
training leveraging Google Colab (so it’s free)
[https://github.com/luxonis/depthai-
experiments](https://github.com/luxonis/depthai-experiments) \- Experiments
showing how to use DepthAI.

The above includes open source hardware, core capabilities in C++ which can be
cross-compiled for various hosts, and open-source training notebooks and use-
case examples.

So the goal of open-sourcing all that we can (including hardware) is to enable
folks who have their own applications to leverage this, modify it, use it,
etc. w/out having to even talk to us. We cannot open-source the code that runs
on the Myriad X however, as then we wouldn't have a way to monetize, and would
have to give up on the mission. (That, and we're also not allowed to.)

Thoughts?

Thanks, Brandon

------
m0nty
So you're telling potentially distracted drivers that hey, it's OK to be
distracted, the AI will take care of that for you? Like with Tesla's auto-
pilot, it doesn't matter what the product manual says - too many drivers will
interpret it to mean you can pay less attention and the computer will take up
the slack for you.

~~~
Luxonis-Brandon
Luxonis-Brandon here. So the distracted driver actually wouldn't be aware that
the person riding the bike has this installed... unless the driver is paying
super-close attention, in which case the device is not needed.

So unlink the Tesla example, where the driver knows his vehicle has this
safety feature... the driver does not know the biker has this feature.

Thoughts?

~~~
m0nty
Sorry I missed your reply.

My experience of cycling in the UK is many cyclists already go to great
lengths to make themselves visible, and it makes no difference at all to close
passing, etc. Hi-viz clothing, bright strobing lights - drivers act like
you're not there.

It also makes drivers lazy - having nearly been knocked off my bike, the
driver stopped and berated me for not wearing hi-viz, despite the bright
lights I was using. Not impossible, therefore, that I also get berated for not
using your device to warn me about drivers. There have been attempts to do
this in the UK already with apps which notify drivers of cycists, with the app
maker saying it would be the cyclists' fault if they were in an accident and
weren't themselves using the app. (This was repudiated by the Advertising
Standards Authority, but still.) This is similar to the all-too-frequent
reporting about "A cyclist was crushed to death by a truck. Police said the
victim was not wearing a helmet." Even the helmet manufacturers don't say it
will stop a 20-ton vehicle from killing you :(

As Chris Boardman says about the helmet debate, it's the wrong emphasis. The
issue is safety on our roads and drivers being careful around cyclists. Also,
cyclists being predictable and riding safely. An electronic device will make
little difference without that, and I feel it could make things worse.

Also, I find a mirror on the handlebars, a life-saver check when turning, and
listening for engine noise to be pretty good at alerting me to drivers. Not
sure electronics would improve this.

------
miahi
I'm not sure I understand what the product is and what it does - it seems more
like a technology looking for a market.

Is this a product that end users (car owners) could buy? How would it be
integrated with the car? As a driver who doesn't care about cyclists (as those
are the ones causing most accidents), would you buy a specific device just for
avoiding cyclists?

Is it a product car manufacturers have to integrate in the car? Shouldn't this
have a wider target than cyclists? How does it compare to the existing
collision avoidance systems? If it's an optional safety system just for
detecting cyclists, how do you sell it to the car buyer?

~~~
pmontra
Yep, the article is very unclear about that. From the Luxonis site

[https://discuss.luxonis.com/d/8-it-works-working-
prototype-o...](https://discuss.luxonis.com/d/8-it-works-working-prototype-of-
commute-guardian)

> So the final product is a multi-camera, rear-facing bike-light, which uses
> object detection and depth mapping to track the cars behind you

It's for the cyclist.

I'm both a car driver and a cyclist. I wonder if I want something like this on
my bike and be unnecessarily scared by false positives (cars do get close to
bicycles all the times and what can I do anyway?) or on my car just in case I
don't notice a bicycle with no lights at night (plenty of them here.)

~~~
FridgeSeal
> I wonder if I want something like this on my bike and be unnecessarily
> scared by false positives

Right?

Also a cyclist, I can totally imagine this going (uselessly) wild when I do
something like lane-slip through peak-hour traffic in the city.

~~~
ljf
And also if the horn goes off, would that make agri drivers more so? I used to
have an air horn on my bike - it sounded like a lorry horn, and the issue was
that drivers heard it and would stop but would normally assume it wasn't from
me, and would carry on.

~~~
FridgeSeal
That's a really good point, it would definitely escalate aggressive drivers.

I've had situations where I rode past someone (they were stopped in traffic),
then they passed me again down the road (this repeated a couple of times) and
every time I rolled past them, they'd descend further into apoplectic rage, I
can't imagine flashing lights is going to help.

~~~
pmontra
I can imagine: they think they paid their car 100 times more than our bicycles
(actually maybe only 10), they know they could go 10 times as fast as a
bicycle (maybe only 5) and yet they are stuck in the traffic, go as slow as us
and won't be at home much earlier (or much later.)

When it happens to me to be a car driver I think that at least I'm enjoying
heating or air conditioning but I'd like to be on my bicycle at least until I
get out of the city.

------
sunpazed
Tech similar to this already exists. My rear bike light is also a radar that
informs me of oncoming traffic. [https://buy.garmin.com/en-
US/US/p/601468](https://buy.garmin.com/en-US/US/p/601468)

It works really well, but requires the cyclist to assess the risk and react.
Something like this, that could inform both myself and the driver of an
impending collision, would be (literally) a life saver.

~~~
ramzyo
This seems like a great future application of V2X.

[https://en.m.wikipedia.org/wiki/Vehicle-to-
everything](https://en.m.wikipedia.org/wiki/Vehicle-to-everything)

~~~
Luxonis-Brandon
Yes, totally agree on V2X. That’s another hard infrastructure problem though…
as it requires so many parties to adopt it (and implement it properly) for it
to really help. We did talk to a bunch of V2X folks on this though to see how
we could make an integration happen… but no traction so far.

------
aikinai
I couldn’t find in the article, what exactly is the cycling product and who
were their customers? Does it go in cars or on bikes?

~~~
Luxonis-Brandon
So the idea is a smart rear bike light with situational awareness. More info
on it is here: [https://discuss.luxonis.com/d/8-it-works-working-
prototype-o...](https://discuss.luxonis.com/d/8-it-works-working-prototype-of-
commute-guardian)

The hardware/firmware/software/AI necessary to make such a device wasn't
available when we started this though, so we had to make it first. And we're
releasing it with OpenCV. See here for details: [https://opencv.org/opencv-
spatial-ai-competition/](https://opencv.org/opencv-spatial-ai-competition/)

So the bike-safety idea is not a product yet, but the underlying platform is
now (and is going live on KickStarter soon).

Thoughts?

------
rorykoehler
The solution to this problem is protected bike lanes. Anything else is a half
measure.

~~~
ajuc
The solution is to change the driving culture.

Protected bike lanes still need to cross regular streets (and that's where
most accidents happen).

Technology won't solve people problem. Drivers need to expect bikers like they
expect other cars and adjust speed to that expectation.

~~~
rorykoehler
It's multi-step. Protected bike lanes will reduce traffic and in turn make it
viable to make wholesale changes to urban transportation infra.

~~~
Luxonis-Brandon
Agreed. And we view what we're developing as a stepping-stone to protected
bike lanes.

------
Davetron
Over-engineered solution to a problem better solved with segregated
infrastructure which will be "backwards compatible" with older cars

~~~
Luxonis-Brandon
We agree that segregated infrastructure is the ultimate solution. But we're
technologists and are 0% effective at lobbying for infrastructure. Having had
a bunch of friends hurt and killed by distracted drivers, it's us using the
skills and capabilities we have to try to make a difference.

And the core of the development is the computer vision technology which allows
a whole slew of other applications, including other safety solutions. The
sheet numbers of problems solvable with this sort of application-specific
human level perception is crazy.

So we are actually doing a competition now around it, here:
[https://opencv.org/opencv-spatial-ai-competition/](https://opencv.org/opencv-
spatial-ai-competition/)

We'll be releasing the winners of stage 1 soon (please feel free to apply!),
which will quantify how crazy it is to be able to approach human-level
perception (what an object is, and where) for specific myopic tasks (like
picking strawberries, or detecting when a car is going to run you over).

Anyway, back to it, I agree personally the solution is infrastructure (which
I've tried to write in every location that covers this... but maybe failed to
do in that interview). But I view this as a stepping stone to infrastructure.
If people are too scared to ride bikes because of the injury and death at the
hands of distracted people driving cars, then there will be no push for
infrastructure (as why build infrastructure for a thing no one does?).

So the goal is to allow people to ride bikes -safer- before the infrastructure
is there... and thereby help drive the infrastructure change (through supply
and demand relationship).

Thoughts?

Thanks, Brandon

