Hi! I am the guy behind this, awesome to see it posted here. Happy to answer any questions.
To answer a question I saw pop up, I am using the FAA aircraft registration data to pull up the extra information about the planes. It lists who actually owns the plane, vs who is operating it. That is why you see so many banks. Places like Flightaware have much better data, but a pricy to look up.
oh yea! great point, I forgot about that. I am using PiAware to handle the decoding so I do have an Enterprise Account. I will have to check if I can make API calls for free.
Unfortunately FlightAware Enterprise doesn't get discounted API access.
The FlightAware API I was using (they have multiple) didn't provide all the data I wanted either. The ADSB Exchange Rapid API was more comprehensive, but came at a cost.
The aircraft don't broadcast everything over ADSB - so these sites are all pulling from a database somewhere else. It appears ADSB Exchange and some of the pi based UIs use the tar1090-db. This was easy to work with and free - just need to update it periodically due to registration changes.
This is absolutely incredible. I have been wanting to do this as well, but I was going to approach it from the computer vision approach (blueiris and deepstack ai). Or a dedicated camera and pythoncv. I never even thought of having cameras move to match flightaware data.
A number of planes fly low and when I check flightaware, they're not on it. Or there's a plane that could be it, but it's flight path is heading East to West 30 miles North of me, while this plane came from the South heading North. Also, pretty much once a month or so, a pair of A10 Warthogs fly over or near our house. Last month, we had a real treat with 4 of them directly over the house, better than an air show.
So I'd like to setup 4 cameras on my tower focused skyward and have them try to catch clips of planes. I'll be looking at your work to see if I can also match those with flightaware.
Doing a vision only approach would be really cool. A lot of the interesting planes do not have their ADS-B transponder on. The two systems could actually work well together. My system ends up generating a really nice, labeled dataset. I put some initial scripts / notebooks together for building ML models that can classify the different types of aircraft.
https://github.com/IQTLabs/SkyScan/tree/main/ml-model/script...
Since 2020-01-01, ADS-B is a requirement in a lot of US airspace (FAR 91.225). Turning it off is allowed for govt / intelligence / law enforcement / military only. Many countries have similar requirements.
People often think an aircraft is flying with transponder off because it doesn't appear on FR24/FlightAware. In fact these companies allow blocking aircraft from their service; this is why ADS-B Exchange is better.
Not a pilot, but played with ADS-B tracking several years ago, and my impression was below something like 4-5000' the transponder wasn't required to transmit location. This meant that surveillance aircraft (usually single engine propeller craft and helicopters) didn't send GPS, only the fact that it was in the air.
Yeah. It's a requirement in a lot of US airspace, but not all airspace. Even at low altitudes it's required near major airports though.
Surveillance aircraft, if operated by a government entity, would not be required to transmit anyway. (Though they may choose to for safety purposes if it doesn't compromise the mission, just as the military often do in busy airspace.)
If turning ADSB off is only allowed for government / military / intelligence airplanes, would using a CV algorithm to capture these airplanes be against the law?
Recently there was a post of Google satellite cameras capturing a B-2 Stealth Bomber showing up on Google Maps [1]. Would such planes be captured by a vision system only, and would that be of concern to the government that might be trying to keep it undetected?
Back in my day, everyone’s address was listed in the phone book. If you wanted to remain anonymous, you had to pay a service charge to keep yourself from being listed!
When I was a kid, nobody thought it was anything unusual to have a reporter from the local newspaper come out to your house and write an article with your home address and phone number.
And now those articles are searchable on newspapers.com!
I do have to admit it was fun finding my mom's old Red Cross activities and my dad's chess tournaments. Any chess players remember Sammy Reshevsky? He was a friend of my dad's and would sometimes stay with us when he visited Eugene for a chess tournament.
Here in Finland the companies operating the phone books and directory services still exist, and you can still search people's phone numbers and addresses online or by calling/texting directory services - just no printed books anymore.
(and you can of course unlist yourself)
I'm somewhat interested, did it go down differently in e.g. U.S., i.e. are directory services gone, too?
In Iceland you can put whatever you want as your job title (or degree) so you have people calling themselves beefcakes, lion tamers, tetris players etc.
It helps to see if you have the right person.
Norway is one such country (https://www.bbc.com/news/magazine-40669239) but I think some others in the Nordics also does the same, not sure exactly which though and Google is not being very helpful this evening.
It's not exactly true that your salary is public -- what's public is your taxable income, taxable wealth, total estimated tax, birth year and postal code. Various deductions apply to the income which makes it non-straightforward to deduce a person's actual income, and as for taxable wealth the minimums are high enough and deductions are substantial enough (especially if you own a home) that a lot of people with wealth much greater than zero will be listed with a taxable wealth as zero.
Given that the image is cropped and it's hard to tell which part of the frame the cropping is for, I think you can easily guess which airport this is nearby, but definitely not at the house or even street precision.
The thing about unbiased random noise is that it cancels itself out! And for the most part, we can compute exactly how many data points are needed to achieve a particular level of precision in the result.
I don't think so. You only need 1 flight to locate him pretty accurately. If you do the thing you said, you'd need 2 flights along different axis to get a pretty good idea, with a larger amounts, you'd just improve accuracy a bit. Regardless, I think it doesn't matter. So what if they know where he lives, it's not like this sort of information is hard to come by anyway, and the risk profile of this is... not interesting, IMO.
Your site got hugged to saturation but this is something I've been wanting to do for a while. I just got a sort-of ridiculous telescope (Celestron 11" SCT on alt/az mount, 2800mm focal length) and have been tinkering with ADSB for years. Can't wait to check out your work!
That would be awesome! How fast can those mounts move? Would they be able to follow an aircraft? I have found that more zoom actually makes things a little tricker because you have to make sure it is leveled and aligned precisely. When my camera is fully zoomed, it has about 1 degree vertical field of view.
I've got a 6" Celestron, and I think it would be capable of following a plane that's sufficiently high up. It can slew a max of 5°/sec, so that would be ~500mph at 10000' or well over 1500mph at 30000'.
Because it's a telescope, it's designed to be aligned at night. It's more of a pain to do during the day (such as for an eclipse or I guess tracking planes). But there's no reason you couldn't align it to stars at night and it will its alignment pretty precisely. I'm not sure how much drift there is in that over say 24+ hours though. It's also smart enough with the Celestron controller to never point at the sun even during a slew. I'm not sure if it has that same safety feature when controlled from a computer, which would be a major concern with daytime use.
But a 6" f/10 (so ~1500mm lens) telescope with a full frame camera is still a 1° field of view so the same as you have now. Smaller sensor would be tighter field of view.
And of course the obvious drawback: None of this setup would be weatherproof
That is interesting! There is this cool piece of software that looked like it helped with a lot of this. It did visual object tracking with telescopes... but it is no longer available: http://www.optictracker.com/What_is_it.html
Alignment has been a big pain for me. I might actually have to look at how to use stars to correctly position everything.
Instead you should just feed ADSBExchange, which doesn't require premium business plans or other nonsense. They also do not censor the data and will show you planes that the owners don't want you to see.
Back when I first discovered flightradar24 it seemed like every plane I was curious about came up on there. The curious ones are mostly military planes as they don't follow standard routes and schedules. Then more and more I noticed stuff not appearing. Later I discovered ADSBExchange and things seemed to be on there. But now, again, I regular hear military planes right over my house with not a thing shown on ADSBExchange. I don't think it's a coverage issue as I always see the regular boring cargo planes that come this way. Is it really "uncensored"?
Yes it is uncensored. Starting in 2019, aircraft conducting operations related to homeland security, law enforcement, national defense or intelligence can turn off ADSB out and communicate their location to the FAA out of band.
Feed ADSBExchange does not have a good return, ADSBExchange have way less data available, they don’t even have a free iOS client. A single Raspberry pi feeds both FlightAware and Flightradar24, they give you free business plan with apps with good user experience.
From a normal user’s point, don’t see a good justification to do so.
If you set up an ADB, Flight Aware will give you an Enterprise account. FlightRadar24 also will give you a Business account. There is a docker image out there that will let you pump data to both services.
For aircraft registration data, I think offline database download should be available without (paid) API access on some smaller services. I thought ADSBX was providing the aircraft database but cannot find it right now. Quick googling found opensky-network, but not sure how much is the database maintained. (Monthly snapshot keep being published, though.)
Hey I wanted to add myself to the list of people that always wanted to build this but never actually did. I wanted to build my own servo gimbal with an old telescope though, you choosing a 30x ptz makes it much more reasonable. Well done!
What did that camera set you back (or did you get it some other way)? Their page only says "Call for pricing", which usually means it's going to be outside my hobby budget hah.
I managed to get it for $500 on ebay... but I just checked and it looks like similar ones are around $1000 now, so I guess I got lucky. An Axis PTZ dome camera turned upside down works great too though and there are a lot of those available on Ebay.
In theory it should work with any camera the supports ONVIF, which is a standard for security cameras... actually, as long as the camera has an API you could probably get it to work. Since precision and image quality are important though, it could end up being more trouble that it is worth. You may have better luck trying to find a used one.
I had this exact idea and have been working on it. Have almost exactly your bill of materials. Glad to know there are others out there who find this interesting, great work!
I was always worried about making it a public data source. Kinda seems like it could be potentially dangerous, but that’s completely paranoid speculation. Thanks for sharing!
I got all the camera gear on ebay, which was lucky.
I figure since it is all based on ADS-B data, there isn't anything too crazy about making it public. I am also still trying to figure out if it is actually useful for anything... besides getting unlimited plane photos!
Do some airlines not provide details, or is that some other issue? I'm seeing a flight 3C7601 that shows -1 passengers and -1 engines. Same for 4007f3 from British airways and 394a06 from air France. A8060C seems to have something in the way, it's just gray with a black stripe
You're the kind of hacker I would love to work with some day.
This is seriously one of the most fun and inventive projects I've seen. Such ingenuity at every stage of this pipeline. I didn't even realize they made cameras like this.
Awesome! I live north of Austin-Bergstrom and I quite often see planes on final approach. I've thought about doing this, and also trying to find a physical site for a telescope or telescopes to watch take-offs and landings.
Let me know if you ever go down this rabbit-hole. Happy to give you pointers! You can get some pretty decent cameras off ebay for <$500. Since you are close you do not need much zoom.
I live under KTEB’s most common departure procedure and very much want to integrate this with my current ADS-B exchange setup. Super excited to give this a shot, hopefully won’t freak out the neighbors too much.
would it be possible to pinpoint location of your house? Since we know the paths each of these planes have taken, public record, and although we dont know what time the photos were take we can still come up with range of locations vs time, where these angles would be possible.
so find intersection of all these planes' paths, then using geometry figure out possible locations, then find their intersections..
It should work for helicopters. In DC, alot of them do not have ADS-B and the flight paths are a little different, so I haven't managed to capture one yet though.
Correct, GP asked about military so I mentioned that it's legal for them to turn it off. It's also legal for intelligence, law enforcement, etc -- anything government-related with a good reason pretty much.
Phantom planes are also technically possible. One could even do it themselves using the "Havok" firmware for HackRF One. I won't link to it, but to quote its wiki, "Yes, it works. For real. If you want to try it out, DO NOT transmit on restricted bands and ESPECIALLY NOT on 1090MHz."
Yes, 1090MHz is probably the stupidest bad to illegally transmit on.
Not just because of the potential safety issue, but because of how easy it is to track you.
In many areas, they have Wide Area Multilateration equipment installed that will show the real location of the transmitter, instead of the fake GPS coordinates you transmit. Normally it's there to double-check the ADS-B coordinates are correct, to track older planes without ADS-B, and to give true altitude readings, but it will track illegal transmitters as a side effect.
Even in areas where they haven't installed a permanent MLAT system, they usually have some sort of equipment that can track down a 1090MHz transponder. For-example, search and rescue planes often carry one.
Feels like the website-about page: http://skybot.cam/about is a much better link than a tweet.. loaded with details. For example:
- Most planes are passing at over 30,000'
- It's using a Costar RISE 4260 camera /w built in heater
- It leverages SkyScan by IQT labs[0]
- It leverages Azure custom vision API[1]
You could get away with a pretty cheap hosting bill, I think a $10 VPC would be more than enough. My current Azure forecast is around $65, but that is for a couple other things too. Half the reason I went big with the different features is that I just started at Microsoft and wanted to learn the different things... plus you get a $150 monthly credit.
All of the capture stuff just runs on a Raspberry Pi.
A year ago today I launched something similar, but for Great Lakes boats passing my house on the Detroit River (uses AIS rather than ADS-B, and passively tracks the boat crossing the fixed camera rather than pointing the camera):
Useful idea for usage:
Point a WiFi-dish/antenna towards the plane and gather WiFi & Bluetooth MAC addresses for all the passengers. Now you can make statistics about the number of passengers, how long /often they travel ++ good information if you want to trade stocks in airplane companies.
You can direct the antenna precisely because you have the image recognition in addition to GPS.
You can also provide the plane with WiFi access this way. A passenger can easily activate WiFi in combination with Airplane Mode on most phones (including iPhone)
About 5 years ago I started trying to build this exact thing. I wrote some code, printed some gears and attached them to servos, assembled some hall effect sensors onto the whole thing, and then stopped.
I had a camera that I could command to an azimuth and an elevation. I think it was a PlayStation3 pseye camera.
It certainly would not have been able to zoom in to the point that this one seems to be able to.
I was going to use my RTL SDR to listen to the ADSB and mount the whole thing on my roof under a clear plastic dome.
Easiest today would be to use the project this project uses. I don't think it existed at the time I was trying, though. And, yes, that would be easier, but not in the vein of "doing it myself" that I was aiming for. :D
ha! yea.... The camera itself doesn't know when a building gets in the way. I have some general stop angles, but it is not perfect. There is some ML in there to get rid of photos like this... but I guess the model things that air conditioners look like airplanes.
Any interest in sharing the plane dataset you used to train Azure Custom Vision with others? Happy to host it on Roboflow Universe along with an API to a trained model others can hit to detect planes in images.
Definitely - Let me build a little better dataset. I need to augment a bit because it was false detecting on some of the contrails.
I built some scripts to also start building a model that can classify the different aircraft models. One of the interesting things with this, is that you get really accurate labels... probably better than what a human could do for images of this size.
Did they ever indicate using a model like that? Just plug in an ADS-B receiver and they'll broadcast their tail number and location to you. Everything else can be looked up.
This is really neat. I've thought about doing something similar, but feared that this is enough information to deduce where one lives. Kudos to the author for not only making this, but also publishing it
You’re not wrong. In 2017, 4chan trolls (in)famously deduced the location of (and defaced) Shia Labeouf’s “HWNDU” flag art/protest project in 37 hours [1] from only a webcam pointed at the sky. Never underestimate the capabilities of bored people on the Internet with an ax to grind.
As someone living directly below a flight path, I want one of two
things: to make enough money to move out, or lobby the FAA to spread the
pain more widely. At some point, the so-called NextGen technology
enabled the FAA to channel all air traffic along a hyper narrow corridor
-- one that's unfortunately right over my street.
I've been thinking about building a laser graphics system to write on clouds on nights when there are stratus clouds.
I'm worried about lasing an airplane (I am not too close to the airport but not that far either) and was thinking about hooking up an ADS-B receiver to tell the system to keep the beam away from airplanes, even if they are on the other side of the cloud deck.
A system like that, funny enough, is supposed to get approval from the FDA (Food & Drug Administration) because the FDA regulates lasers. If aircraft are involved the FDA asks the FAA what they think about it. I'd hope that they think that protective system would be good enough.
Keep in mind that not all aircraft are required to transmit an ADS-B signal.
Not to say it couldn’t be done, given the right approvals. You’d possibly be asked to publish a NOTAM to tell aircraft to stay away. The FAA has a publication on this very topic:
>I'd hope that they think that protective system would be good enough.
They most certainly won't. Peruse this site if interested. Long story short, the kind of wattage you're talking about (writing on clouds visible from the ground) is incredibly dangerous, not just to airplanes but to those on the ground, and you would have to demonstrate through licenses and permits that you know what you're doing. I don't think $50k will get you even halfway there, if possible at all in any kind of urban center.
Wow there is a plane roughly every 2-3 minutes even at 1 am. You should be living pretty close to the airport for that kind of traffic? I really don't know how about the traffic density is in various places. Since they are mostly over 30k i assume noise is not an issue ? Would it be possible to mount a omnidirectional mic along with the camera to see if you capture any sounds ?
Amazing idea. I always wanted to make something like ths. We live about 3 miles (vector distance) from a local airport and this airport is middle of a 200 year old town, used pretty much for teaching how to fly! In summers, when we sit outside, it has rather become a big nuisance as there's a plane that creates sound pollution. I meant to gather facts as to how many "learn to fly" planes fly over the residential areas with inherent risk associated and many residents are also worried about the same.
Are you able to open source this setup and guide us thru some of the details? Your idea/implementation would actually help change the world into a better place.
A thought experiment leapt to mind on how one would triangulate (septangulate!) your location based on heading, time, flight information, and their intersections. I suppose you could get within a 100 meters? Very cool photos!
When I lived in Macau a while back, I used to listen to the Hong Kong Airport tower on my SDR and try to link it to flight data, but at the time (2011/2012 or so)my connection and the data was not that great, but I could see and hear who was taking off and landing.
Totally - I think you could work back where the camera is based. I have the bearing / azimuth of the camera for each photo, so you could look up where each plane was at that timestamp and then project backwards.
I bet you had much better aircraft flying out of Hong Kong!!
Fascinating: from a very quick scroll it is surprising to see how much traffic is to do with banks.
Wilmington Trust, Bank of Utah, Wells Fargo, “UMB Bank” (?), US Bank. Is it really the case that a quarter of US air traffic is rich people in suits who are too good for Zoom? :)
Isn’t the operator of a flight what shows up in e.g. Flightradar and also what you’d typically be most interested in, rather than the owner of the aircraft?
It's to hide the owners identity. I don't know why people are arguing with me about this. Trust's are a wealth management tool that provide a number of services including hiding a persons identity.
Look at a normal vehicle title it will say {Bank} financing division. NEVER "trust"
I did some digging to try to find out. I found the owner's (well, registrant's) house, and it was reasonably average for someone flying their own private jet.
That's incredible, what a great idea! I read some instructable a long time ago, but forgot to bookmark it, where someone was using an ADS-B receiver and raspberry pi to display a faux radar screen image of azimuth and altitude of planes nearby.
Ubiquiti’s UniFi Protect app uses a similar UI for ‘smart detections’ and it’s really very nicely done / looks like you’ve achieved the same quality of cropping and layout in the snapshots. Bravo. So much potential for this. Good luck with it!
I feel like FBI is gonna be knocking on the door any time now. It'd be cool to calculate the velocity and display that. It looks like one of the planes is specially going fast.
This is one of those things I always wanted to work on, but never got around to. Though, the execution on this is probably a lot more slick than I would have done. Well done!
So, if I put one of these on top of an RV and parked it near Groom Lake, does anyone know if the flying saucers leaving Area 51 are transmitting ADS-B?
Also, what if many folks set up the same so that we have full crowdsourced surveillance of aircrafts over various regions, how could this info be used i wonder?
Looking at the planes that fly over your house? Seriously, I am interested in this and do it already "manually" when I can. What more reason do you need?
From my experience you are either under a flight lane and a lot of planes fly over, or none do. There may be some places further north where they are more spread out, but for example I used to live just south of the approach to one of the runways at my city's airport (maybe 10 miles away so the planes were not super low but were already lined up with the runway) and we saw a constant stream of them go by. I really enjoyed it to, there were lots i could identify readily and sometimes an obscure one that I'd jump on flightaware to see what it was. I think this is a cool project and wish I had something similar.
Edit: related, I regularly commuted by plane between two cities, and my parents live out in the country part way between the two. I could regularly see their house out the window of my plane, because by chance one of the flight lanes was just north of them. So they would have have a lot of traffic going overhead, despite being nowhere near an airport. They also have less frequent planes go by on great circle routes to europe, which is probably what you're imagining thinking about seeing planes infrequently.
Good question - I just checked and my forecast for the month is $65... but I have a few other things running. One of the perks of working at Microsoft is that you get $150 monthly credit, so I didn't work too hard to keep the costs low. I could save a little by combining the API and App server.
If you add a laser pointer you'll effectively get yourself arrested within a few hours.
Aircraft get laserd relatively often and it's dangerous to the pilots. We report it to air traffic control with an estimated location, they then coordinate with the police.
Common practice is to turn off exterior lights when lasered, because it's just bored people with a laser pointer and they can't aim when the plane has no lights on. But if you link it to ADS-B that would require switching off the transponder which will cause issues for ATC.
I am not trying to be negative about what seems like Ana amazing project but I am genuinely curious: why? Is it the cool factor of being able to identify aircraft out of the sky with a camera? Being able to show the photos to someone (who?)? Asserting your rights to capture this data? Why would someone want to do this?
The way you wanted to phrase that question is, "Cool! Was there any other reason you wanted to build this beyond the technical challenge of it?" Otherwise it sounds like you're saying, "You need to get laid more, bro."
This question is also handy at hackathons. Sometimes you get an answer like, "nope", other times you get anti-government screeds, sometimes dreams of saving the world once they've scaled up.
Personally, I find it better not to ask and to just enjoy the hack.
The reason I asked is because I have a unique opportunity to do the same where I live but I want to know if there is any utility in it other than the build itself and the learning experience, and if so, what it is. This reminds me of the thing that takes photos of the ISS every time it flies overhead, except I get no understanding of why someone wouldn’t just use the many many high resolution photos of it from much more up close or the publicly available data. But let’s say that it turned out that every time it passes by you could use your decoder ring to get another piece of a neat message then I would get into it for my kids’ sake.
