AFAIK Comma.ai is the only open-source company in the self-driving cars space. It is very interesting that they are continuously launching consumer-products with a small team and in a highly competitive space where the other players are quite big.
They have a more focused approach that simplifies it a lot. Where others build whole self-driving cars Comma uses existing interfaces and sensors in the vehicle. That's far less complexity to worry about.
This does not in any shape or form compete in the self-driving cars space; the thing doesn't have a rear camera!
This is basically the "rsync as Dropbox" approach to augmenting your car with a hacky version of automatic cruise control, with a bonus lawsuit should you ever be in a crash significant enough to warrant the attention (think paralysis).
Comma is far and away the best version of "automatic cruise control" that exists in any consumer car. If you think Comma is hacky then wait until you see Tesla's version, which costs $10k and only works on one brand of car!
The stock "automatic cruise control" is most cars is far worse and far more dangerous than Comma, and those companies aren't getting taken down by lawsuits. At the end of the day, the driver is responsible for the operation of the vehicle. Comma is a tool the driver can use, and one that undoubtedly makes driving safer for themselves and other vehicles on the road.
> If you think Comma is hacky then wait until you see Tesla's version, which costs $10k and only works on one brand of car!
What does the cost and number of supported vehicles have to do with whether or not the software is hacky? (Hint: nothing; it has nothing to do with it.)
> The stock "automatic cruise control" is most cars is far worse and far more dangerous than Comma
That's a pretty extraordinary claim to make without presenting any evidence.
> one that undoubtedly makes driving safer for themselves and other vehicles on the road.
Ditto.
The front page of the Comma website claims "millions" of miles have been driven using it; given that in the US there's about 1 fatality per 100 million miles driven, I don't think we have anywhere near enough data to make any claims about Comma's safety. It may actually be safer, but I don't think we have a way to know that yet.
And also, given that Comma supports several different vehicles, I suspect that we'll have to consider its safety record on a per-make (and in some cases possibly per-model) basis, not on the system as a whole, in order to compare apples to apples.
"automatic cruise control" is free and comes with every Tesla. It is not $10k. It is what they call "autopilot" and is free. "Full self driving" is what is $10k and includes automatic navigation on highways when using "automatic cruise control"
Is that a downside? Anyone who's ever struggled with hardware driver problems knows that the more hardware configurations your software (notionally) supports, the more likely you are to have show-stopping bugs.
In a home PC, some reliability is arguably a fair trade-off for versatility. Not in a car.
Um... Have you driven a Mach E yet? No interest in its upcoming FSD, but the adaptive cruise control and lane assistance is wonderfully conservative and useful.
I've only driven my car, but Consumer Reports tested a bunch of cars and rated Comma the best [1]. Maybe another car since then is better, but of course Comma is always getting updated and you just download the new version to your device when it comes out.
After reading a description of the comma device and how it's being used it seems uncannily like how my Mach E drives. But also as both my first EV and my first semi self-driving car, perhaps I have a different set of standards. My TLDR was that the adaptive cruise control and automatic lane keeping extended my daily driving range by several hundred miles by reducing cognitive load significantly. It was the best 900 miles I've ever driven.
But watching people nitpick various variants of electric crossover SUVs, I realize I would probably like just about all of them because none of my cars have been anything like that up to now.
So I keyed in my other two cars to see if I could put the comma into it and alas I cannot. They are 2013 and 2016 models respectively.
If you haven't tried it, I would refrain from making that claim.
I used the v1 of this device for ~1 year, and since then driving has never felt the same. It was sooo much less annoying to do regular commutes when the device took control for even 90% of the trip. This was true even though I still paid attention to the road and had hands ready to take the wheel.
Sure, but your experience doesn't really tell us about the overall safety of the system. How many total miles have been driven by people using a Comma system? The front page of their website says "millions", which is several orders of magnitude less than Tesla Autopilot (several billion miles at this point).
Given that in the US there is around 1 fatality per 100 million miles driven, I don't think "millions" of miles driven with Comma is anywhere near enough to say anything about its safety record.
I'm not sure why you're being downvoted. This is an accurate, if colorful, assessment. It has all the same safety problems as a Tesla with even worse sensors, less compute, and practically zero real safety V&V work.
They cutely dodged NHTSA's regulatory ire by shipping a "dash cam" that happens to run software you can install which makes it semi-autonomous... even though said software is made by the same company that ships the "dash cam". NHTSA isn't stupid, so I'm curious what it will take for this loophole to eventually be properly closed.
Sadly, safety is a lot like security. When it's done well, it's invisible, but when it's done poorly...
Not actually open source. The source code that’s available is a thin wrapper around a closed-source ML model. If George really cared about Open Source he would release the training architecture and training data.
As somebody who works with video and photo user data for machine learning, I'd say they are much better off not releasing the majority of their data. Unfortunately there are fairly severe legal consequences for commercial use of certain kinds of unconsented user data in various US states (consent of the people filmed, not the people collecting the data), and even companies in complete compliance have to deal with a huge potential legal headache to prove that their datasets follow every local regulation.
Has anyone here tried using one of these? On the surface they seem to be relatively reliable and safe. I am tempted to get try one out since their driver cam gives me confidence that I won't become complacent while using one of these self-driving systems.
I have a Comma Two and have driven maybe 4000 miles on it around the SF Bay area. It is somewhat limited in that they can only steer as hard as the car's lanekeeping system is willing to steer. The driver monitoring helps although I am middle aged and pretty careful anyway so it's only actually beeped at me on a handful of occasions. Honestly, it is not the type of device where you want to take your eyes off the road, I think the DM is actually a little more lenient than it should be. In generally, you are basically as alert as if you are driving. The advantage is that you aren't spending mental energy keeping the car centered in the lane or maintaining a good following distance, or straining your arms holding onto a steering wheel. Long car trips are much more comfortable and less as a result.
It's not very useful on city streets or curvy mountain roads, the places where it shines are freeways and stop and go traffic -- the boring driving.
It has been quite reliable, most of the kinks have been worked out and they have enough users that there are not any serious bugs on the standard releases. I have reported minor bugs and had them fixed in a matter of days, which is always amazing for an OSS project. If you have an uncommon car there can be tuning issues from time to time. There is also a fairly strong user community on Discord that can help you troubleshoot things.
It works best on Toyotas and Hyundais, Honda and Subarus are a little worse, and the other manufacturers is really hit or miss--there may be individual models where someone got it working.
It is not a consumer product, you need to be somewhat capable of troubleshooting the occasional issue, reading documentation, dealing with occasional quirks etc.
Installation is not consumer friendly (though I'm sure any auto shop would happily install it), but functionality certainly is. My wife who is not particularly tech savvy has no problem using Comma.
I have a car that isn't even the best for Comma because of steering limits (Honda CRV), and on road trips I've been able to not touch the steering wheel or accelerator/brake for 30+ minutes on open stretches of the 101. You'd think it doesn't matter all that much, but it saves so much energy. A 4 hour drive feels like a 1 hour drive. Monitoring that the car is driving sensibly uses far less energy than actually driving.
And as you say, in bumper to bumper traffic it is just amazing. I was in a 2.5 (!) hour backup between LA and San Diego a few weeks ago and Comma did 99% of the driving. And the only times I had to take over were when other frustrated drivers were doing insane things out of boredom or anger that endangered me. Computers don't get frustrated or angry! The aggravation saved on that one trip paid for the cost.
I use my comma two everyday. Works great. Support from the company is non-existent.
I'm kind of pissed because I bought it in the last few months and now they've come out with another one. There should be a trade-in and upgrade program.
It has better cameras. Higher dynamic range which means it can handle tunnels, shade, etc. better. The cameras can also see further which means the possibility for traffic light behavior in the future.
I'm not anywhere near a specialist when it comes to car software, CAN, or self-driving, but I'm concerned by some of the code that (appears to) control some of the supported cars: I pulled up a random file[1], and it's littered with undocumented bytestrings that are apparently sent over the CAN bus, as magic constants (with accompanying comments that mention "aggressive" and "laggy" behavior).
I have no insight into any of the proprietary work in this space, so I have no idea whether this is unique or commonplace. But it certainly isn't confidence inspiring, particularly as a cyclist.
The file you referenced is related to Ford, which is not a supported car/brand. The code for Ford is provided as a reference and is forced into “read-only” both by openpilot and panda (the bridge between openpilot and the car).
On one hand I agree with you wholeheartedly (I also think python is a weird language choice for something that probably has some hard real-time requirements), but also consider that stuff like that may very well be present in the code bases of the closed-source commercial solutions, but unless someone who has worked on one of them speaks up, we'll never know.
There's openpilot code, and then there's panda code that runs on a separate real time micro chip. All the safety relavent code runs on the panda, and it's written in C following misra c guidelines. They also comply with iso 26262.
See more info about their safety model in the safety architecture section of this blog post.
Yeah, that's why I couched my critique a little bit: I have no insight whatsoever into the commercial side of this and, for all I know, it could be even worse!
But that alone is bloodchilling: it terrifies me, as a pedestrian and cyclist, to think that stuff like this is probably controlling the pieces of machinery that could kill me in a split second.
In their system, the actual real-time safety critical code is elsewhere, on a dedicated processor. The python code sends messages to the safety code, which applies limits and sends commands to the car hardware. And other safety systems like AEB are totally untouched and still functional.
The safety code has a limit on steering ramp rate and a max torque limit. The driver can easily overpower the torque limit, and the ramp rate limit means it won't suddenly jerk.
The python code I think is temporary, they ultimately want to do end-to-end models that do both perception and control (rather than a model to do perception and traditional controller to do control). If that effort fails I think they would probably replace the python code, the current stuff is definitely amateurish.
George just said today during a Q+A session that they do not intend to do learned control because traditional control is already greatly exceeds human ability.
I know the identifiers etc are probably hard to catalog, but that is all the more reason to give things symbolic names and maybe even avoid having to specify those IDs repeatedly.
Thanks for the reference! That certainly does look much nicer.
That being said, I do still see some "magical" looking constants and dictionary keys here[1], as well as lots of small numbers later in the file. I think it would be confidence-inspiring to have those better documented as well.
Edit: I found some more magic-looking bytestrings in the Volkswagen support here[2].
At some point the values have no deeper meaning and are just whatever VW decided on as their API. DBC files have concepts of enums, but that’s really diminishing returns and wouldn’t make the code that much easier to understand.
Also sometimes there are fixed values in the message that we’ve only observed as static data. Ironically that makes it impossible to reverse engineer what they mean. So those just have to be labeled as hardcoded value.
Those are actually magic. Those are the ECU firmware identifiers from the manufacturer that are used by openpilot to recognize which car it’s connected to.
There’s some messing around involved in building the interfaces to the cars. Comma’s excuse for why you shouldn’t be concerned is that both Open Pilot and the cars themselves have limits against erratic movements.
> Comma’s excuse for why you shouldn’t be concerned is that both Open Pilot and the cars themselves have limits against erratic movements.
Are these limits baked into the cars at some level beneath Comma, or do they rely on the same physical layers and networks? It makes perfect sense to me that the car has its own limits; my concern is that installing Comma takes the car outside of its expected operating parameters and that I have no real way of verifying whether those limits are still in place.
Far more devices running Android than iOS in the world. And unlike consumer tech, the average car on the road is over 12 years old.
So there's a long window of opportunity where we might see more people choose to upgrade their current vehicles to self-driving, versus those choosing to buy new vehicles with inbuilt self-driving hardware.
I think a big barrier to adoption here is that your Android phone most likely won't kill you if it has a bad bug[0].
Comma almost certainly won't get widespread adoption until it has enough miles driven on it (at least a billion?) for people to have enough data to start talking about its safety record. So that means it'll be stuck in the realm of enthusiasts for quite a long time.
If it does prove itself, though, I agree with you that it could be a game-changer for people who have recent enough cars to be supported, but not have their own driver-assist systems. And that's probably a lot of cars, and a lot of people.
[0] Although, with the exploding batteries debacle...
No OEM tests their ADAS for a billion miles before release. It's a level 2 system that requires human attention, not a 100% self driving level 5 system. Ford claims they'll release their new adas, Blue Cruise, when they reach 500,000 miles of testing. (doubt it was that much for their current co-pilot 360) Openpilot so far has 50 million miles.
I think safety has already been proven, and I don't think that's preventing widespread adoption at all. It's not like most people even know about this product. It's currently being purposely kept down-low in order to only attract a certain type of tech savvy person who is more likely contribute to the project in some way (pull requests, bug reports) or at a type of person who at least won't need heavy hand holding. A large mainstream consumer audience for a dev kit is too much for them to handle, and would just be a distraction.
Yeah definitely. Though the timeframe is in the 10-30 year range. Safe bet that 10 years from now most vehicles on the road will still be human-driven. Openpilot with 10 more years of development will be impressive.
"The comma three devkit does not ship with any software. Once you have the device you will be able to install any software you choose at your own risk."
Makes sense, if they shipped with the software already preinstalled, it would mean that they open themselves up to liability and other legal issues. So they don't have it installed as it's pretty easy to download and install from their Github
I’m generally not a fan of that kind of fig-leaf (say “we are not sharing weapons, just information” when encouraging people to print a gun-shaped bobby trap that is more likely to blow your hand than change anyone’s mind about gun control) but in this case, the veneer of technical engagement seems to filter for, or encourage people who are excited enough by self-driving that they actually pay attention and send feedback.
It's very likely a deliberate and precise execution of the legal requirements to shield themselves from legal liability, under the advice of lawyers. At each step, from the coding to the hardware development to installation and configuration the responsibility is clear.
The installer of both hardware and software ends up being responsible for the use of the system in a way that is much more legally clear than other automation systems or even consumer software. Deliberate and informed choices are made by entities completely outside the plausible influence of developers.
It could get tricky in cases where the installer is not the end user.
> deliberate and precise execution of the legal requirements
I really want to agree with you, all evidence says you are right — but then I see any 30 seconds of geohot, speaking a mile a minute and saying about 40 things that would give any lawyer a heart attack, assertively antagonising _everyone_ and piling on the least politically correct way of seeing things. Either he’s the Top-Gun of walking the fine legal line, or he’s incredibly lucky. Inexplicably, lucky seems more likely.
In the one I saw he spent 5 minutes (of 30) figuring out a mistake caused by a simple typo, that a regular IDE would wiggle for him the second after he typed it in.
Is there a significant number of people who use vim without any sort of syntax highlighting and linting? I'm still a newb programmer but I started trying to learn neovim this weekend, I can't imagine using it plain...
