
Launch HN: Sterblue (YC S18) – Software for drones to inspect power lines - crubier
Hey HN,<p>I am Vincent of Sterblue (<a href="https:&#x2F;&#x2F;www.sterblue.com" rel="nofollow">https:&#x2F;&#x2F;www.sterblue.com</a>). We build software for drones to inspect power lines and wind turbines automatically.<p>We are three drone enthusiasts. I met Nicolas in 2006 during our Aerospace Engineering studies at ISAE, in France. Small drones were still in their infancy and we eagerly enrolled in the brand-new student micro-drone club. Nicolas later met Geoffrey when working in India for a large aerospace company, while I was working on my PhD in Computer Science.<p>In 2015, we shared a feeling of disappointment that the vision people had for drones back in 2006 had not fully happened yet. For example, utility companies need to inspect millions of power line pylons. They still mostly perform inspections using helicopters or rope access technicians. They sometimes use drones for short distance inspections, but they could not scale it up to inspect all of their grid.<p>We decided to try to make drone inspections happen in the renewable energy industry.<p>In the beginning, we were super optimistic: we started by designing and patenting actual drone hardware configurations, building and testing drone prototypes. Then we realised that hardware is really hard! and worse, that new drones are not what this industry needs right now. Even with super high-performance drones, you still need a top-gun-level pilot to capture precise photos 8 hours a day from a drone flying high in the wind. And then you need to spend ages trying to find small anomalies on thousands of captured images.<p>So our approach became to automate most of the inspection process, using off-the-shelf DJI drones. We built two core technologies in the last 2 years to try to get there.<p>The first one is a mission planning and execution engine based on ideas developed during my PhD. It allows drones to fly on complex 3D trajectories that wrap tightly around structures, flying the drone sometimes less than 3m away from the objects to inspect. Most other drones companies fly high above the objects to inspect, so their apps show trajectories on 2D maps. Our app shows trajectories on 3D views because our trajectories are intricated closely around objects. As an example, we had to take into account the fact that wind turbine blades bend under their own weight. Here is an image of power grid inspection with our software: <a href="https:&#x2F;&#x2F;imgur.com&#x2F;KRE1Oce" rel="nofollow">https:&#x2F;&#x2F;imgur.com&#x2F;KRE1Oce</a><p>The second is a deep learning framework built on top of Tensorflow, PyTorch and Caffe2. It allows to detect 130 classes of defects on images: corrosion, broken equipment, and other safety hazards. The typical use case for drones today is photogrammetry, in order to create 3D models from captured images. This is not super useful for wind turbine and power line inspection, so we focus on our thing: finding defects directly on images, just like human operators do now. For several classes of common defects, our software has 98% recall and 80% precision. We run up to 18 different neural networks on some images in order to find defects on all kinds of equipment. Here is an image of our algorithm detecting an anomaly on a wind turbine: <a href="https:&#x2F;&#x2F;imgur.com&#x2F;iGIxtAZ" rel="nofollow">https:&#x2F;&#x2F;imgur.com&#x2F;iGIxtAZ</a><p>These two technologies are linked to a GraphQL API and cloud platform. Our customers get to visualize inspections results on PDF reports and 3D interfaces online. Our UIs are made using React and React Native. Here is an image of an HTML report for distribution grid inspection: <a href="https:&#x2F;&#x2F;imgur.com&#x2F;dkznc6o" rel="nofollow">https:&#x2F;&#x2F;imgur.com&#x2F;dkznc6o</a><p>In the last 6 months we had successful pilot projects with utilities in 7 different countries, and some have even started to use our software in production mode now. This is exciting for us. But we still have a lot to learn, and we’d like to hear your feedback and ideas in this space!
======
nathancahill
I worked on a project in this industry with PG&E in California. The amount
that these companies spend to have humans fly power lines for inspection is
incredible. Besides the cost of the helo, they have a pilot and a second
person inspecting the lines and recording on a camera for review later.

Good luck, there's _a lot_ of cash up for the taking if you're able to
automate the process.

~~~
crubier
True. The helicopter inspection process is very productive, but also costly
and dangerous. Also their process is all about spotting defects from the
helicopter while flying, and only taking pictures of defects. This is pretty
epic, but not always the best way to make sure you don't miss anything.

We capture images of all structures exhaustively, which means that the
analysis is done offline, not in a rush. It also allow to scroll back in time
to see if a newly found anomaly was already present during a previous
inspection.

------
genemachinery
Have you tried linking transmission outages from EMS/SCADA to drone images?
Could you build ML for EMS training station to drone images? I worked with
Siemens EMS/SCADA and we always needed models of past failures to train
operators. Siemens has a microgrid simulator at SFCC, Santa Fe, NM. I am
writing python scripts and would like a scikit like set of examples for
students to simulate outages in NM, USA. Can you build some examples in
Jupyter notebook or Mu Python? How about Sql Server using Microsoft ML?

------
Dangeranger
Great work!

The electric utility world can be a challenging industry due to the reluctance
of companies to take risks on new technologies.

Another challenge can be the somewhat antiquated data delivery policies. When
I was working in the space around 2012 our vegetation violation deliverables
were required to be sent via Excel spreadsheets, then followed up using PDFs,
and GIS Shapefile formats.

I'm glad somebody is trying to push a more modern workflow into the industry
to make the process more efficient.

Good luck on your journey.

~~~
crubier
I see you have experience in the field, we noticed exactly the same pain
points as you did!

But we noticed that utilities are now getting more and more challenged on
different fronts, so many of them are opening to new technologies as a way to
improve their operational efficiency.

~~~
Dangeranger
That is excellent to hear. We experienced several different types of people
within utilities, with everything from "we've always done it this way" to "you
are the experts, tell us what to do".

The younger employees were always excited for us to work with them to make
their work easier. Many advocates within the companies had excellent
suggestions which were unfortunately sidelined by their managers.

------
PascLeRasc
Great idea. I worked on a contract doing pole replacement surveying for a few
months and it was incredibly monotonous - full-fledged engineers walking and
driving around neighborhoods taking photos of power lines and estimating
measurements with a rangefinder. There's a ton of room for automating and
eliminating hazardous jobs, and millions if not billions to be made. Good
luck!

------
ChuckMcM
This is a great idea and I've seen a number of companies offering 'drone
inspection as a service'. At the Commercial UAV conference in Las Vegas last
year there was a funny/sad video of a UAV that was inspecting high voltage
lines and it tried to fly between the wires. Apparently weather conditions
were such that this violated the minimum viable air gap and the two wires
arc'd through the UAV when fell to the ground in a smoking heap.

That said, when you take care, there are so many ways that this solution is
better than lines people in trucks inspecting the infrastructure!

~~~
crubier
Aha, every drone company has its own set of funny/sad moments as you call it!
We had a few ourselves in our beginnings, when we were still making drones.

However we never had a crash when inspecting power lines! Here is a video of
power line inspection using only satellite imagery as input, without giant
electric arc!

[https://www.linkedin.com/feed/update/urn:li:activity:6441158...](https://www.linkedin.com/feed/update/urn:li:activity:6441158446225199104)

------
roshanj
Awesome work Vincent! Glad to see more automation using drones in pilot-less
commercial scenarios.

If a customer had a structure without a preexisting 3d model how would you
create the flight plan to operate close enough for your defect detector to
work? Do you have issues with moving structures/obstacles (e.g. a wind turbine
rotor offset from its orientation in the model)?

Full Disclosure: I work on the SDK Platform for Skydio R1 (a fully autonomous
drone doing real-time visual SLAM and motion planning). If you're interested
in chatting more I'm at roshan [at] skydio.com

~~~
crubier
Many customers don’t have precise 3D models of their infrastructure. We deal
with that, and other real-world constraints such as wind turbine stopping
position, wind turbine blades bending under their own weight, power line
slacking low when temperature is high and others in the navigation algorithms.

We have been looking at Skydio for some time, your product is amazing! We'll
get in touch

------
rsp1984
Very cool stuff.

Are you doing anything special for 3D trajectory estimation (any visual
tracking / SLAM)? Or is it just GPS or whatever DJI puts in?

Also, would it help with defect detection and classification if you had LiDAR
data as well?

~~~
crubier
We are in a middle ground: we do more than pure GPS navigation around 3D
models using DJI inputs, there is some amount of computer vision going on, but
we do not do real time online SLAM either.

Some of our customers provide us with LIDAR data, which enhances both
navigation and accuracy of report generation.

------
semi-extrinsic
I'm in a research institution that's definitely interested in this topic; will
forward this to some colleagues.

Question: do you have any plans to offer self-hosted solutions? I imagine that
while many companies will be happy to let you guys do pilot projects on some
lines, once you start talking about "let's put the whole national grid in"
then national security concerns start popping up. Power distribution grids are
highly critical infrastructure. Probably wise to pre-empt any troubles on that
front; I'd guess a wise move would be partnering with a trusted entity in each
country where you want to operate, letting them run it as a "local, trusted
SaaS".

~~~
crubier
You are right, this is a concern for some of our customers. We had talks about
self-hosted solutions, but so far no customers required it. There a several
reasons for this:

\- We host data on AWS on data centers in appropriate regions for each
customer (read: in their own country, or a nearby one). AWS compliance
programs also help a lot in building credibility:
[https://aws.amazon.com/fr/compliance/programs/](https://aws.amazon.com/fr/compliance/programs/)

\- We use solutions such as [https://www.sqreen.io/](https://www.sqreen.io/)
to add additional layers of security on our infrastructure

\- Having data on our side allows to retrain neural networks more effectively,
which gives them great value that would be more difficult to get with on-
premise deployments. We are looking into edge computing and training however.

\- So far we perform visual inspection of publicly visible assets. We do not
host any data that would not be visible to anyone looking up to overhead lines
in the countryside. This might also change in the future, and the amount of
data we gather is also a concern, even if it is public.

In the end, we take this topic very seriously, and if you have any additional
info on easy ways to deploy with trusted entities in various geographical
areas, I would be happy to hear your experience about that!

~~~
semi-extrinsic
I think the amount of data is the main concern, yes.

For Europe, you probably want to follow closely the new “EU DSO Entity” which
is to be formed following the new Electricity Regulation, it's an entity
intended to enhance cooperation between Distribution System Operators. One
doesn't know if it will be just a paper tiger, though.

------
btown
Very cool to see these ideas coming out of academic work! Any chance you could
link any of your papers on the 3D mission planning? Having done 2D autonomous
robotics a while back, I'm very curious what new challenges and approaches
apply dealing with drones.

------
jacquesm
That's an awesome pick for an intersection between several different fields.
For an added challenge: run the drones from ships to inspect off-shore
installations, and possibly to expand into oil rig and crane inspections.

~~~
crubier
Inspecting off-shore wind turbine was actually the first application we wanted
to address. It was a bit difficult as a first step, so we came back to on-
shore inspections. But we are now able to perform off shore wind turbine
inspections

------
red154
Congrats on your progress! How do you guys deal with liability issues, such as
if your algorithms were to miss a potential defect and something were to
happen? Do your customers worry about that at all?

~~~
crubier
Hi and thanks ! There are multiple answers to this particular problem:

\- We tune the algorithms to give very high recall (up to 98%), at the cost of
lower precision (down to 70% in certain cases). This means that we miss very
few defects, but with sometimes 30% false positives.

\- We capture pictures of each point of the structure to inspect from at least
2 or 3 viewpoints. This means that the 98% recall for each image compounds: if
we missed a defect on a picture, we have chances of finding it on other
pictures from other angles.

\- All our software allows humans in the loop, from data capture to data
analysis. This means human experts can validate or correct results from the
AI, human drones operators can take back manual control of the drone, and so
on. Our automation is here to help humans perform their job, not fully replace
them.

\- Our customers worry about that a lot, and most of them aknowledge that our
method is more effective than traditional methods

------
Kagerjay
This is really interesting. Have you also considered oil & gas pipeline
inspection work as well, corrosion is a big issue here AFAIK. Or factory plant
inspections as well?

~~~
crubier
Thanks!

\- Factory plant inspections is on our roadmap. We have technical proof of
concept on this, especially on generating flight plans for arbitrary complex
3D structures, and detecting defects on images. It will be more of a
commercial effort than a technical effort for us.

\- Pipeline inspection is an interesting application, a bit less in our scope
right now because there is not so much of a "3D" aspect to the flight, it is
mostly linear assets, which can be dealt with existing mission planning
software and drones. But we see where we could add value to this, and will do
this if we have more time and resources one day!

------
asparagui
Very cool! Can you talk more about your ML pipeline? How are you managing the
process of running images through your pipeline? How are humans involved?

~~~
crubier
Our ML pipeline is deployed on AWS, using a mix of AWS Lambdas and Kubernetes
containers on the brand new EKS.

We also did some amount of training on our own GPU machines on premises.

But we recently started to work with our YC batchmates of
[https://snark.ai](https://snark.ai) , which enables us to speed up training
by A LOT

------
LeanderK
Why is your jobs-section not available in english?

I think I would not apply at a company where there's no english version
available. I don't think i'd be welcome there as a non-Frenchman.

In general, why is there still such a poor internationalization in everything
in france? From universities, to jobs in startups (!) etc. I recently tried
searching for a master in france and it was super difficult.

~~~
eldavido
I took french in high school and can read about 90% of this.

It's not _that_ hard. lol. Half of advanced math was done in France
anyway...Laplace, logarithmes, Cauchy, Riemann...

"analyser les résultats des algorithmes de manière quantitative" => analyze
the results of algorithms in a quantitative manner

~~~
gvancassel
We'll provide french lessons to any new Sterblue employees ;) (Geoffrey,
Sterblue CEO)

------
Space_Cube
Very exciting software. Do you plan to expand into other fields with a similar
process? Such as with crops and looking for defects?

~~~
crubier
Thanks a lot! Our software is really modular so yes we do have a few plans for
new verticals :-) At this point it’s more a matter of commercial effort than
technical development

------
hrsantiago
Is the demo working? I answered everything but got stuck at "Done! Your
information was sent perfectly."

~~~
crubier
Hi, yes we have a working demo but it is invite only so far. We received your
application and will give you access to the demo shortly!

------
veb
This is pretty cool. Maybe you guys can add English captions to the video, as
it's French right now :)

~~~
gvancassel
If you go on youtube, there are english subtitles ;). But thanks for the
point, we'll change this on the website

~~~
veb
No. Those are auto-generated. Quite terrible. It really makes a world of
difference when someone adds proper subtitles for the deaf/people without
audio.

As a deaf person, the way the auto-generated captions are shown is so much
harder to read than just your normal sync'd captions.

------
almost_usual
This is very cool, have you guys considered running inspection for fire risk
in wildfire prone areas?

------
msadowski
So exciting! A bit tangential to what you're doing but I strongly believe that
the first company to come up with single button press mapping with a drone
will capture a lot of the market.

Looking forward to see how your project evolves!

------
tron_job
Cool idea

------
aleem
Displacing millions of dollars with DJI drones + CV... can’t wait to see
what’s more in store.

Can hi-res cameras mitigate the need for close-proximity fly-bys?

Are there any companies doing home security drones? Like circling the property
routinely, auto docking to recharge, reporting human thermals around the
perimeter etc. Maybe even neighbourhood-watch, crime-fighting drone fleets in
the near future

So much potential, can’t wait for it to unleash.

~~~
crubier
You are right, a ombination of Hi-res cameras + Accurate GPS + Collision
avoidance on basic off-the-helf drones means that Software such as Sterblue
can now perform jobs that previously needed custom drones with custom sensors.

There are lots of companies in the drone security landscape. This is not our
market as we are not too much sold to the idea of drones tracking people and
potentially using weapons... We are more than happy with our mission of
helping the development of clean energies and a more energy-efficient future.

However I know a stealth company that uses drones to assist firefighters,
their solution is impressive and saves lives!

