
Ask HN: Are there any projects to do historical archiving of RF spectrum? - adamgamble
Something like archive.org but for rf spectrum. I&#x27;m not even sure if its legal, but I was playing with my SDR the other day and thinking of all the historical information that passes over the airwaves that is ephemeral. Being able to view this data after a major event would be amazing. Imagine listening to all the radio information as 9&#x2F;11 transpired or during the coup attempt in turkey.<p>Obviously this couldn&#x27;t be done everywhere all the time, but it would be interesting to be able to archive radio spectrum during major events.<p>A quick google search didn&#x27;t turn up anything.
======
kastnerkyle
As vitovito mentions, wideband data in high resolution is incredibly expensive
from a storage perspective. There are entities that do it, but this _also_ has
the same privacy concerns as saving every packet that comes over a wire - in
fact if you record 2.4 MHz and 5 MHz you _will_ be picking up someone's
(hopefully encrypted) private data. I don't think it is illegal, but the
technical aspects of this are pretty daunting.

You would be better off decoding and storing that, but that gets into v&
territory pretty quickly, depending on who and what you decode (definitely
don't decrypt/crack).

~~~
elbrownos
Google was deemed to have broken wiretapping laws when it scooped up unsecured
wifi data while collecting images for Streetview.

[http://www.bbc.co.uk/news/technology-24047235](http://www.bbc.co.uk/news/technology-24047235)

~~~
throwaway7767
In my country back in the eighties, the TV networks broadcasting scrambled
content over the airwaves sued some people who were making a profit modifying
receivers to unscramble the broadcasts without entering the numeric key (it
was a very weak scheme).

The court ruled that no law was broken as the signals were being sent over the
air, and therefore had no expectation of privacy. And since at that time the
descramblers were sold to the user, there was no problem with modifying the
hardware.

This is why they switched to leasing the descrambler boxes when they upgraded
their system (and chose a system for which descramblers were not easily
publicly available).

It always seemed a sensible ruling to me. Though I don't think anything has
changed legally, I suspect the ruling might be different if such a case were
tried today. :/

------
pigeons
Hard enough to get good coverage, and very hard to get the storage resources.
There was a CCC talk about setting up coverage, but that was ambitious enough,
archiving was mostly out of scope:

[http://www.cw-complex.com/rfarray/](http://www.cw-complex.com/rfarray/)

[http://www.rtl-sdr.com/talk-monitoring-spectrum-building-
dis...](http://www.rtl-sdr.com/talk-monitoring-spectrum-building-distributed-
rf-scanner-array/)

------
derptacious
This is super interesting! And if anyone else is curious about creating a
project that monitors the range of RF amplitude and frequency at many data
points around the world, then send me your contact info! I want to see a
record of this spectrum (human and non-human made) in frequency and amplitude
to see how much it is (and hopefully "has") changed in physical space. I think
interesting things would come out of this data.

~~~
adamgamble
Any idea if its legal? I assume it is but I have no idea.

~~~
gravypod
Listening is 100% fine in the USA. Transmission without a ham license, and in
certain bands, will net you some fines and maybe jail time.

~~~
vitovito
Any spectrum recordings would almost be guaranteed to include copyrighted
content, and in the US distributing those recordings would fall afoul of the
DMCA.

~~~
gravypod
That's not how RF works. The courts have heard cases like this. Look up the
talk on CreepyDOL from defcon.

------
vitovito
From my notes on a radio archiving project I'm doing:

> Could you record the entire radio spectrum and extract stations and
> broadcasts later?

> In the USA, the AM radio band is 540-1710 kHz, a spread of 1170 kHZ,

> [http://rtl-sdr.better-than.tv/?page_id=237](http://rtl-sdr.better-
> than.tv/?page_id=237) states that spectrum recordings are also a function of
> how many samples per second you choose to record:

> 2.8msps - 44.8mbps = 5.6 MB/sec.

> 2msps - 32mbps = 4 MB/sec.

> 1msps - 16mbps 2 MB/sec.

> [http://www.myradiobase.de/perseus/](http://www.myradiobase.de/perseus/) has
> sample files. 3.5 minutes is ~360MB. 60 seconds of 1500 kHZ is a gig. All
> too much. 24 hours at 1MB/min is 1.5GB, but 17MB/sec. is 1.4TB. Spectrum
> recordings are out.

~~~
x1798DE
Assuming most of this is dead air, don't you think this would be a highly
compressible data set (even lossless)?

~~~
niftich
Highly is perhaps an overstatement, but compressible like most clear signal
data, behaving similar to PCM audio.

I compressed '20090922_950_GLFS_outdoor.wav' (150 kHz - 1750 kHz) from the
Perseus site [1] with some lossless compressors. In each case, I independently
round-tripped the de/compression and the results were confirmed to be
lossless.

The compression ratios are in line or slightly (but not overwhelmingly) better
than what you'd expect [2] from audio:

    
    
      original:		 1.000
      7zip-lzma2-xz-ultra:	 0.820 (not a signal compressor)
      wavpack-normal:	 0.724
      wavpack-high:		 0.691
      wavpack-hhx6:		 0.593
    

I would've liked to test additional compressors. In addition to WavPack, FLAC,
TAK, ALAC, OptimFROG, TTA, WMA Lossless all purport to have some level of
high-resolution support, but I'm not sure how many kHz they go up to. FLAC
wouldn't take the file, and the rest I don't have access to right this moment.

[1] [http://www.myradiobase.de/perseus/](http://www.myradiobase.de/perseus/)

[2]
[http://wiki.hydrogenaud.io/index.php?title=Lossless_comparis...](http://wiki.hydrogenaud.io/index.php?title=Lossless_comparison)

------
jf
The Internet Archive is archiving some of the broadcast TV part of the RF
spectrum, which is where this comes from:
[https://archive.org/details/tv](https://archive.org/details/tv)

------
privong
Most of the major radio astronomy observatories maintain archives (e.g., [0]).
Of course, they mostly use highly directional dishes pointed at the sky, but
local radio signals can create interference[1]. So, those archives provide
some probe/record of terrestrial RF spectrum. Of course, those telescopes are
generally placed in radio-quiet locations.

[0]
[https://archive.nrao.edu/archive/advquery.jsp](https://archive.nrao.edu/archive/advquery.jsp)

[1] [http://www.gb.nrao.edu/IPG/](http://www.gb.nrao.edu/IPG/)

------
gravypod
Archiving the entire spectrum can range anywhere from infeasible to
impossible. First you'd need to specify the frequency ranges you want to
listen to.

Then you've got a data storage problem which has been covered.

After that you've got the propagation problem. Not everything is seen from
everywhere.

In any event you might want to check out
[http://www.reversebeacon.net/](http://www.reversebeacon.net/)

It's a project that's got a different "simpler" archiving goal. It is just to
track spots of people calling CQ in the CW bands.

------
vesche
As vitovito and pigeons mentioned storage and collection locations are the
main issues. Things like this exist, but they don't archive data and are
designed for military use- Google: Wideband Recording System . They typically
are mobile units that focus on fast capture rate and decently large storage (a
few terabytes).

sigidwiki is worth looking at, but it's more for classification:
[http://www.sigidwiki.com/wiki/Signal_Identification_Guide](http://www.sigidwiki.com/wiki/Signal_Identification_Guide)

------
superuser2
>all the radio information as 9/11 transpired

I know it's not everything, but if you are interested in this the New York
Times has an excellent feature using the ATC tapes and some recorded phone
lines in the military/air defense system.

[http://www.nytimes.com/interactive/2011/09/08/nyregion/911-t...](http://www.nytimes.com/interactive/2011/09/08/nyregion/911-tapes.html?_r=0)

~~~
adamgamble
Also here is beeper messages that somehow wikileaks got a hold of.

[https://911.wikileaks.org/](https://911.wikileaks.org/)

------
niftich
I find this fascinating because the context of transmissions occurring
simultaneously is indeed ephemeral.

This data gets lost with traditional archiving, which is squarely considers
context (including temporal simultaneity across multiple objects) to be
metadata.

These sorts of efforts can be partially retrofitted/approximated with
timecodes on existing archived material, which may also be a separate,
worthwhile endeavor to pursue.

------
theknarf
The closest thing I know about is websdr
([http://websdr.org](http://websdr.org))

------
barrystaes
Likely there are top-secret satellites doing exactly that for specific
geographic regions. These likely wont retain all gathered data indefinitely
however, and even if data is shared its tainted politically.

------
wyldfire
No but that sounds like a good idea!

------
rfw1z
there are commercial solutions that do this

~~~
shawn81
Such as... ?

