
Using the Web Audio API to Make a Modem - maaaats
https://martinmelhus.com/web-audio-modem/
======
ravenstine
Having worked on a similar project, my experience is that signal-to-noise is a
very challenging problem when you have an air gap. Inevitably, it opens up
room for minute errors caused by accoustics. The only "reliable" workarounds I
devised were to increase broadcast duration and rebroadcasting the same data
with simple checksums.

This was what I came up with:
[https://github.com/Ravenstine/airsocket](https://github.com/Ravenstine/airsocket)

Note it's extreme crappiness is due to a combination of my inexperience and
also my desire to build as many parts from scratch as possible. The Web Audio
API is only used to play audio, but the oscillator and decoding functions were
written by me. It barely works. It's probably better to go full Web Audio all
the way since I'm sure there's hardware acceleration(particularly with
AnalyserNode). The author's method of encoding bytes is also far superior to
what I came up with.

I wish such an article was published when I was trying to devise my own audio
modem. He does an excellent job going over every step and explaining what's
going on. Documentation and tutorials around Web Audio used to be too
convoluted, with basic examples too few and far between.

~~~
brian-armstrong
I have a WebAudio modem that works reasonably well, and I went a different
direction with it. Mine uses a C DSP/SDR library with good error correction,
run through emscripten to generate the samples.

You can try it here [https://quiet.github.io/quiet-
js](https://quiet.github.io/quiet-js)

~~~
ravenstine
Actually, I think it was your project that gave me the idea to attempt near-
ultrasonic transmission. I could only get to ~15000hz before it would just not
work and I never figured out why that was. Maybe there's a flaw in how I used
the Goertzel algorithm. (reason I went with Goertzel is because I built it for
a previous project and wanted to see how far I could take it)

Is that example page new? That's brilliant. Your ultrasonic transmission seems
to work pretty consistently.

~~~
Arve
Someone pointed out low-pass filters to you, but that's not the entire story.
Built-in speakers in phones and laptops generally don't have a naturally
extended frequency response.

A while ago, I made audio measurements of the built-in speakers on the iPhone
7 Plus - [https://imgur.com/gallery/DRbu5](https://imgur.com/gallery/DRbu5)

The iPhone's extension is not entirely atypical of such devices, and leaves
you with a reliable passband of 500 Hz to 10 kHz - above and below those
frequencies, you're getting too much noise.

If you're airgapping with audio cables, you'll have a reliable passband of 20
- 20 000 Hz or thereabouts, with some caveats for built-in audio on devices
such as a Raspberry Pi that fizzle out at 15 kHz.

As noted by brian-armstrong's comment, you'll only get reliable near-
ultrasonic airgapping over a wired connection.

~~~
brian-armstrong
This isn't correct, by the way. Quiet's near-ultrasonic mode (~19kHz) works
fine from several feet away using phone hardware. As long as you're using
relatively narrowband communication, it works fine.

------
martinmelhus
I created this, so please let me explain myself.

After stumbling over a really cool Web Audio API synthesizer I wanted to play
around with the API myself. I used my work situation for inspiration, but I
just wanted a fun pet project and didn't really bother to deep dive into the
theory of modems. Hence the naive implementation and lack of error correction.

So, do I actually use this at work? Truth be told, I've only ever used it to
show of to my collegues. USB-drives are way more practical in every situation
where I might want to use this. But the modem is able to provide the tool I
set out to make: to copy ascii text between two computers!

Glad to see that you guys liked it!

~~~
ivionday
What was the cool synthesizer you found?

~~~
martinmelhus
This one! [https://webaudiodemos.appspot.com/midi-
synth/](https://webaudiodemos.appspot.com/midi-synth/)

------
pavlov
A discussion about the Web Audio API from a couple of weeks ago with the title
"I don't know who the Web Audio API is designed for":
[https://news.ycombinator.com/item?id=15240762](https://news.ycombinator.com/item?id=15240762)

I'm still not convinced that this API is actually useful for any application
that needs to play sound or music, but it definitely seems to have the
"interesting synthesizer experiments" segment covered.

~~~
fasteddy
[https://inverted3.gitlab.io/drum-machine/](https://inverted3.gitlab.io/drum-
machine/) is a working multi channel drum machine using Web Audio.

~~~
pavlov
Yes, the API lets you build 1979-style synths out of a toybox of generators
and filters. However it lacks support for the most common task in real-world
digital audio: computing samples into a buffer and queueing it for playback.
(There is ScriptProcessorNode but it’s deprecated and performance is not up to
audio rendering standards.)

~~~
BillyWM
>There is ScriptProcessorNode but it’s deprecated and performance is not up to
audio rendering standards.

On the web, "deprecated" doesn't mean you can't use it. It can take a long
time for "officially deprecated by the standards" to become "I can't use it
anymore" (obsoleted).

The replacement is the "AudioWorklet" interface and the purpose is to fix
those exact synchronization issues. I wouldn't expect ScriptProcessorNode to
disappear from the browser until the replacement is actually rolled out.

The Web Audio API started as the Audio Data API in Firefox, which was nothing
but a buffer to blast samples into, then the Web Audio API was a
refinement/competing standard above that extremely simplistic API (I adapted a
Javascript-based .mod player to use AudioData instead of sticking an <audio>
tag in the page circa 2011).

You've been able to generate and play back samples in browser (without Flash)
for ~6-7 years now, they're simply making official changes to the underlying
API in the hopes of improving it. "Deprecated" is just bureaucracy.

~~~
kevingadd
It's not reasonable to call it a refinement when it removed the central
feature of the Audio Data API (feeding samples). Web Audio was a _competing_
API, designed by a former developer of Core Audio (at Apple) once he moved to
Google. They shipped it without a standardization process, at which point it
became the de-facto audio API for the web.

~~~
BillyWM
> it removed the central feature of the Audio Data API (feeding samples)

Again, I've written actual code with both. With all due respect, I'm not sure
what parallel universe you're from. (Do they spell it Berenstein on your
side?)

I can pull up my Github commit from 2014 and look at where I changed my moz-
compatible > dynamicAudio.write(modPlayer.getSamples(bufferLength));

to

>var processor = audioContext.createScriptProcessor(bufferSize, 0, 2); >
[......] >processor.connect(audioContext.destination);

etc, and it still played sound.

If you're unaware how Amiga .mod/.xm works, it is a REQUIREMENT that you can
provide raw samples to the output source, because the samples are embedded
into the file.

You can feed samples to the browser and have it play them, right here, right
now, just like you've been able to for years. If you don't like the status of
ScriptProcessor as deprecated and think AudioWorklet sucks, that's a whole
separate issue, but you can feed samples now and you'll STILL be able to feed
samples when AudioWorklet obsoletes ScriptProcessor.

>Web Audio was a competing API

I already mentioned it was a competing standard in my comment.

>They shipped it without a standardization process

The Audio Data API was one dude hacking on Firefox (David Humphreys) and then
other people got interested. One of the requirements of the standards process
is that there are multiple implementations in the wild. You need to throw it
in your browser if you want it to ever be standardized.

And they DID standardize it early on. You can go read old copies of the W3C
standard submitted by Google.

I was sad to see Mozilla's version go away but browser vendors eventually
decided it was the losing implementation - including Mozilla.

Mozilla didn't HAVE to submit to the whims of others, they could have dug in
their heels if they thought their version was better (see: The Video tag, and
everyone who just so happens to be part of the MPEG-LA patent pool saying no
to a patent-unencumbered, open video standard baked in to the browser)

------
foreigner
A friend and I once entered a competition to see who could transmit a text
message first between two computers using DAQ boards. The judges walked around
the room giving every team two wires. By the time they got to us they had run
out, so we had to use paperclips. Our implementation did single-bit
handshaking. Most of the other teams were trying to use a clock-based
solution. Or throughout was super slow, but because our solution was stupid-
simple we got it working first and won!

~~~
a_t48
Could you elaborate? Did you use something like Morse Code?

------
pgeorgi
coreboot has something similar to provide an early boot console in a device
without "simple" output devices, using the PC speaker[0][1]. I doubt many
people use it, but it's one of the things that can save the day.

Although in case of "this corporate computer is air gapped", I wonder what
corp policies that little project violated.

[0] Sender (highly hardware specific):
[https://review.coreboot.org/cgit/coreboot.git/tree/src/drive...](https://review.coreboot.org/cgit/coreboot.git/tree/src/drivers/pc80/pc/spkmodem.c)

[1] Receiver:
[https://review.coreboot.org/cgit/coreboot.git/tree/util/spkm...](https://review.coreboot.org/cgit/coreboot.git/tree/util/spkmodem_recv/spkmodem-
recv.c)

~~~
tuxxy
Woah, I had no idea this existed. This is pretty neat!

------
mikeflynn
That's a cool project.

For others to experiment with, there is also quiet-js that is similar:
[https://github.com/quiet/quiet-js](https://github.com/quiet/quiet-js)

~~~
brian-armstrong
Ah hey, I made this. I was going to post it but it looks like I don't have to
:)

Here's the live demo of it [https://quiet.github.io/quiet-
js](https://quiet.github.io/quiet-js)

------
sgt
That's cool. I managed to whistle:  00
@@`@@@@@@```@@@ @@`@` ` `

------
westurner
While we're talking about Air Gaps, it's probably worth mentioning GSMem (an
{x86,} internal bus as a GSM cellular transceiver (modem)); from Wikipedia:

[https://en.wikipedia.org/wiki/Air_gap_malware](https://en.wikipedia.org/wiki/Air_gap_malware)

------
staunch
Maybe an "acoustic coupler" of some kind would help? They look so cool, I've
always wanted an excuse to use one for something.

[https://en.wikipedia.org/wiki/Acoustic_coupler](https://en.wikipedia.org/wiki/Acoustic_coupler)

~~~
beezle
Ah yes..my fond memories of 300 baud and being able to read my program listing
as it printed on the teletype or when lucky, video terminal. Thankfully I
missed 110 baud.

------
mkagenius
Google Tone[1] is a similar work.

1\. [https://chrome.google.com/webstore/detail/google-
tone/nnckeh...](https://chrome.google.com/webstore/detail/google-
tone/nnckehldicaciogcbchegobnafnjkcne?hl=en)

------
nayuki
Cool demo! From the title, I mistakenly thought it might be to make a dial-up
compliant modem in JavaScript. But nope, it's a custom toy protocol instead.

The encoder (sound playback) works in Chrome and Firefox, but the decoder
(microphone recording) works in Chrome but not Firefox. Also, the
encoder/decoder can't handle non-ASCII text; maybe it is only grabbing the low
8 bits of each UTF-16 code unit.

------
amigoingtodie
Lots of weird anecdotes.

Why is the computer unable to be connected to the internet?

Why are you pasting so much code from stack overflow to visual studio?

Cool implementation - should try ultrasonic...

~~~
empath75
It seems like it was just a story. Like tethering his cell phone would have
been a more realistic option also.

~~~
tyingq
Guessing some contractual clause that the device not be connected to any other
network.

~~~
jacobush
So he connected it to another network.

~~~
tyingq
One limited to 7 bit ASCII and cut and paste.

~~~
krallja
That's sufficient: [https://en.m.wikipedia.org/wiki/Quoted-
printable](https://en.m.wikipedia.org/wiki/Quoted-printable)

------
perpetualcrayon
Would the signal from the headphones need to be boosted to make the connection
more reliable?

It might be useful / fun to implement a 2-wire-protocol or other IoT protocol.
I'm thinking you should be able to implement up to 2-wire given the R and L
channels of the headphones.

------
JorgeGT
Also worth checking: minimodem
[http://www.whence.com/minimodem/](http://www.whence.com/minimodem/) (and an
audio cable if your laptop has a line-in minijack!)

------
beached_whale
All I can think of with this is another way to bridge air gaps but accessible
to more people.

------
adrianN
So did the author manage to successfully load a website using this modem? How
did they teach their operating system to use the modem?

------
mschuster91
Can you please add an export function to the encoder? It's a nice way to
create electronic music!

------
tyingq
Very cool. Funny that a Norwegian would settle for an implementation limited
to 7 bit ascii though.

~~~
tyingq
Er, because many Nordic characters wouldn't pass, if that wasn't clear.

------
jonknee
Interesting security implications for air gapped systems, gotta yank the 3.5mm
too!

~~~
krallja
And the bus...
[https://news.ycombinator.com/item?id=15473481](https://news.ycombinator.com/item?id=15473481)

Good luck patching THAT hole.

------
ape4
Fun project. But what about an ethernet cable, bluetooth, NFC, wifi,...

~~~
krallja
...USB drive, USB keyboard HID driven by the laptop, QR code to webcam, CD-
ROM, floppy...

~~~
JetSpiegel
IP over Avian Carriers

~~~
ravenstine
For those who didn't know...
[https://en.wikipedia.org/wiki/IP_over_Avian_Carriers](https://en.wikipedia.org/wiki/IP_over_Avian_Carriers)

