
Transcribed police scanners in real-time - illuminated
https://murph.live/
======
VWWHFSfQ
The speech-to-text transcription is so incredibly wrong that it's almost
dangerous to publish it like this.

For instance:

> at the beach view new screen for assistance there is a needle in his hand
> he's foaming from his mouth throwing off this item

What the officer actually said on the radio:

> He was going to Rainier Beach area. A request for assistance to approach two
> people with needles. <operator>: Call was from a neighbor in the area.

~~~
MengYuanLong
I don't have familiarity with speech-to-text but wouldn't it be possible to
weight words based on their probability in this application to help resolve
this. For example, suspect is probably a low frequency word in normal speech
but very high in radio chatter.

~~~
amluto
This needs care. Imagine if you accidentally trained a model that added racism
when none was present in the audio.

~~~
VWWHFSfQ
Or used a weird model like this one which does some sort of Markov chaining to
complete sentences that weren't even present in the original audio or
transcription.

"foaming at the mouth" was never even close to being uttered on the radio. I'm
guessing the (flawed) model inserted that part because of the proximity to the
word "needle" and "assistance".

Maybe? No idea.. this website it totally fucked.

~~~
optimumsuave
Hello!

The quality is currently limited by Google's API. I am working on getting some
pre-trained models implemented, but voice processing is not my speciality as a
software engineer.

I do NOT want to spread misinformation nor do we want to unjustly slander
anyone. Tonight I will be adding a disclaimer mentioning the limitations of
our service and will make sure it is forefront on the website.

Hopefully we can create a model which can deliver better results.

------
blantonl
This is very impressive.

I'm the owner of Broadcastify.com, where presumably these streams are being
transcribed from. We've dabbled in this space and looked at real-world
approaches to taking something like this to market, but transcribing 7000+
streams to text seems like an expensive (computational) and ($$) effort that
needs a lot of investigation.

Note to mention that the individual lexicons between streams are drastically
different.

I wonder how the developer has done the integration to our streams... because
I never heard from them :)

~~~
lunixbochs
I prototyped this concept too, at
[https://feeds.talonvoice.com](https://feeds.talonvoice.com) with
prohibitively expensive Google speech recognition, but also have a feature for
users to listen and fix transcriptions. If murph was anything like me they
probably paid for broadcastify and tailed a couple of the static mp3 feeds.

My plan was to collect user transcription corrections on my site then train my
own inexpensive models on them. The open-source speech tech I work on can do
passable transcription at close to 100x faster than realtime on a quad core
desktop CPU (or 200 simultaneous streams per 4-core box at 50% activity). With
higher quality transcription it's closer to 10-20x faster than realtime.

For your case you could also try to push some of the computation down to the
uploading machine. These models can run on a raspberry pi.

I think the biggest work for a new effort here is going to be building local
language models and collecting transcribed audio to train on. However, there
have been a couple of incredible advances in the last year for semi-supervised
speech recognition learning, where we can probably leverage your 1 year
backlog as "unsupervised training data" while only having a small portion of
it properly transcribed.

The current state-of-the-art paper uses around 100 hours of transcribed audio
and 60,000 hours of unlabeled audio, and I bet you could push the 100h
requirement down with a good language model and mixing in existing training
data from non-radio sources.

~~~
blantonl
Our new project, Broadcastify Calls, might be a better fit for this. Instead
of 24x7 live streams, we capture and ingest every individual call as a
compressed audio file from SDRs (software defined receivers) We can then
ingest and present back to consumers playback, rewind, playlist, of those
calls. We're now capturing over 100 systems and 800-900 calls a minute... as
we solidify the architecture it will be our new direction for how we capture
and disseminate public safety audio (Police Scanners)

[https://www.broadcastify.com/calls](https://www.broadcastify.com/calls)

~~~
p0sixlang
Hey Blatoni, big fan, and software engineer here. Any way you could add
Rochester, NY (Monroe County Sheriff, and RPD) to the list of supported calls?
I have an RTL SDR, but haven't been able to spend the time figuring out how to
decrypt the Phase II trunking.

~~~
blantonl
You can get started as a calls ingest provider here:

[https://wiki.radioreference.com/index.php/Broadcastify-
Calls](https://wiki.radioreference.com/index.php/Broadcastify-Calls)

------
rachelbythebay
Example audio:
[https://www.broadcastify.com/webPlayer/29351](https://www.broadcastify.com/webPlayer/29351)

This is close enough to the Seattle feed that you can do a compare & contrast.

Heard: "clear my first call ocean nora 470" On the site: "charlie my first
call"

So, yeah, this still has a long long way to go. I considered and discarded
this in 2011 because it was pure insanity, and as another comment suggests,
it's highly context-sensitive.

"ECR" is El Camino Real. "Vets" is Veterans Blvd.

But...

"Code 99" is the emergency button... for one department... and it means
something else entirely for another, just 20 miles apart.

I'd love to have it, but it still seems out of reach.

~~~
jcims
Your scanner project was a marvel to me at the time.

Until a system can train right down to the officer/dispatcher I don't see how
this gets beyond 50-60% accurate. I can't even transcribe half the calls, not
sure why I should expect a computer to (at this time anyway).

------
cddotdotslash
I really like this idea - it also has the side effect of making all of these
streams easily searchable later on.

That being said, transcription still isn't at the point where it can
understand radio garble.

> because I'm going to take that group that someone over the Willing. Can you
> raise Lieutenant lady and having semi-open Buddhist month

> you know the carrot for you and Hudson no accident

Reminds me of the early days of Google Voice voicemail transcription.

~~~
bberenberg
Absolutely agree. I feel like the next step here would be to:

* Put the audio clip next to the transcribed text

* Flag transcribed text which fails some heuristics

* Allows users to suggest edits to the transcription

* Train the transcription models on contributed edits + police specific lingo

~~~
gfosco
Cooperative community editing, genius-style, would be perfect. Big events
would lead to incredibly accurate transcripts.

------
lunixbochs
I built something similar to this, open-source, here:
[https://feeds.talonvoice.com](https://feeds.talonvoice.com)

It's not currently streaming any feeds because Google Speech is pretty
expensive, but I have the expertise and plan to train my own speech models
that would be less expensive to run and more accurate than Google at this task
as well.

The main difference between this and murph is my `feeds` site has UI for users
to listen to the audio and fix/vote on transcriptions, and corrections are
propagated quickly to other users.

------
optimumsuave
Developer of murph.live here again - after reading this thread I have some
ideas I'd like to vet with everyone here.

1\. We need to post links to the source of the stream. I neglected to do that
for fear of cease and desist, but now I realize we need to create
accountability on our own platform. I will be contacting broadcastify.com to
ensure we can direct users to a source.

2\. We need a disclaimer on the site directly in your face. I agree with
everyone here - this could potentially spread misinformation and do more harm
than intended. These transcripts should be read with caution. Additional
messaging from us is a must.

3\. We need a better acoustic model. Google is too much $$$ and although I'm
an engineer, I'm not a linguistics machine learning expert. Can anyone help me
with this please?

Our mission was to create transparency into our government - not cause harm.
There is a lot of responsibility creating a tool like this and we want to get
it right.

With that being said, this site blew up in a few hours. I'm overwhelmed.
Please let me know if you'd like to help. Thank you to everyone for the
feedback so far - it all helps immensely.

~~~
akreal
I have a generic English ASR model for ESPnet
([https://github.com/espnet/espnet](https://github.com/espnet/espnet)) trained
on multiple various datasets and would be happy to provide it. If you send me
few audio samples, I can give it a try. You can contact me
pavel.denisov@gmx.de.

------
neilv
When I was studying photojournalism on the side, for a while I ran a vintage
radio scanner over the various local police/fire/etc. public simulcast
frequencies. It didn't pay off with any "breaking news" leads, but one of the
surprise benefits was the strong impression of professionalism and good nature
by many.

I should be clear that I know there's tragic racism and other injustices
everywhere, and that problems tend to be worse than is visible to most people,
and I think now is the time to finally right some wrongs that we should've a
long time ago. Perhaps relevant to that, it was really reassuring to observe
signs of _goodness_ in the institutions that many people never have occasion
to. Problems need to be solved, but there's hope from multiple directions.

------
belorn
Automatic recording, transcribing and publish. A recipe to force them to
change equipment and start using encryption.

I don't know the law in the US, but here in Sweden any police investgiation is
kept under strict secrecy until completed. When a crime occur and journalist
ask for details the answer is always the same. While investigation is ongoing
no details may be given. That would not work if there is a searchable
transcription of the communication online.

So an nice technical achievement, but the more successful it is the faster it
will be made obsolete in terms of getting information out of police scanners.

------
optimumsuave
Hello Hacker News!!!

I am the developer of murph.live - I just want to thank all of you for taking
the time to check it out and give us excellent feedback. I stumbled upon this
post and now have goosebumps.

This started by listening to police scanners throughout the night during
recent protests in Seattle, WA. I wanted to help and I immediately put my
credit card down for Google's Speech to Text API.

As for the inbound streams, @blantonl is spot on - we use the streams from a
premium account on broadcastify.com (thank you for not sending a cease and
desist yet!).

A few dockerized ffmpeg processes segment the streamed audio into 30 second
wav files. Subsequently, sox removes silence from the audio files as police
scanners have quite a bit of downtime between transmissions. The performance
is very scalable using docker containers to record and trim.

Currently, we pipe these trimmed wav files to the Google Speech API - as
others have mentioned this is $$$. We are receiving donations, but this
dependency on Google will eventually need to be eliminated.

I have started looking into possible solutions using NLP and other acoustic
models to bring the costs down. Honestly, speech processing is not my forte so
I'm kind of shooting in the dark here. I am currently testing pre-trained
models for wav2letter++, kaldi, vosk, and maybe deepspeech.

We can all agree the quality of the transcripts is something to be desired and
improved upon. Potentially dangerous if transcribed incorrectly, but
nonetheless we wanted to launch to give citizens a platform to provide
transparency into our government. The idea is what counts right now.

Thanks again and I will be responding to a bunch of comments on here! You all
rock!

~~~
rasz
[https://news.ycombinator.com/item?id=23322321](https://news.ycombinator.com/item?id=23322321)
first comment might be of interest

------
petee
Nice but mostly incoherent - _" Google Launcher new job and I want to play
better third-party colder or does the people from the vegetable okay"_

------
starpilot
> got another call from the zombie shelter for 24th Avenue got two males
> waiting inside

Oh shit.

------
solarengineer
Given the current global (and US) social situation of mistrust of the police,
and of people running wild with information, it may be wise to shut down this
website and fix technical issues first.

Otherwise, you do run the risk of "See the screenshots - the police
transcriptions have been changed to become police friendly now!"

------
silveraxe93
I was reading a post yesterday about how someone found difficult to gauge the
level of violence in the current protests. As media is so polarised that you
either get shots of peaceful protests or complete mayhem, depending on the
paper's bias.

This got me thinking: What if we get the police radio transcripts, and extract
every mention of an address. Then have a central hub of drones that get
dispatched automatically to any address of "interest" filming.

It's a bit crazy, most likely illegal. But definitely would be cool to see!
Arguably truly unbiased "reporting". If you want to know the situation, just
plug in to the feeds and get a decent overview of how it looks.

------
tapmap
Would be great to link to the real recording, so you can hear it for yourself.

------
yellow_postit
Neat! Wonder what service they are using for the transcription as some of the
results are pretty rough probably from both poor quality audio and a unique
lexicon. Custom speech models could probably go a long way here to clean it up
and I would bet there’s some public interest in contributing transcriptions
for training if the site allowed user submitted corrections. Having this
indexed and searchable could be powerful.

------
pathhandwaving
I didn't see where anyone pointed this out but scanners are only one way
emergency services communicate and they also know people are listening in.
More often than not the important conversations are communicated over the
phone. It is also important to note that since these agencies know that
civilians listen in it can become a way to intentionally misinform the
listeners.

------
LVDOVICVS
Thanks for the encouragement in continuing with my court reporting classes.
It's still going to be a good career for the foreseeable future.

------
larrywright
This actually would solve one of my use cases that broadcastify itself does
not. I don't listen to the scanner often, and often when I do it's in response
to hearing sirens or seeing something. By then, the call has already gone out
so tuning in to the scanner isn't all that helpful. Having transcripts of
recent calls available would be quite useful.

------
stuartaxelowen
Very cool! Is this yours, OP? I built a text classifier platform
([https://www.taggit.io/](https://www.taggit.io/)) and I'd love to explore
building something on top of this text stream to notify people of important
happenings around the protests in real time.

~~~
illuminated
Not mine, I've got the URL over IM few minutes before posting it here

~~~
stuartaxelowen
Thanks!

------
mhh__
Is it legal to listen to these in the states?

I know OFCOM (UK FCC) state it's illegal to listen to certain frequencies
without permission, which seems delightfully unenforceable (for legal reasons
I most definitely do not possess a wideband software defined radio or a
similarly sensitive antenna)

~~~
jauer
(not a lawyer, if one shows up, listen to them) Generally speaking, you can
listen to anything, as long as it's not for commercial gain. The exception to
this is mobile phones, because it was trivial to listen to them using
children's toy radios.

There's apparently some uncertainty around handling of encrypted emergency
services communications:
[https://www.rtdna.org/content/scanners](https://www.rtdna.org/content/scanners)

[https://www.fcc.gov/consumers/guides/interception-and-
divulg...](https://www.fcc.gov/consumers/guides/interception-and-divulgence-
radio-communications)

------
pugworthy
Some of the text though are pretty weak. I have no idea if this is just
garbled transcription, or dreamy cop talk...

"movie up and around pain and dreamy I'm not"

or

"happy person that you can just so may I help you"

~~~
lunixbochs
It's the transcription, I think this kind of site should allow you to fall
back to listening to the audio. Even with high accuracy, if there's no human
curation, there will be pretty awkward mistakes every once in a while in the
text.

------
atarian
Apart from the police and people listening out of interest/curiosity, who else
taps into police scanners? It seems like a really narrow and competitive niche
to be in for developers.

~~~
blantonl
It is ridiculously competitive in the mobile app store market, because there
is a lot of revenue out there. All of the apps source their streams from
Broadcastify, and they license Broadcastify's API and content to do so. The
apps are enormously popular, and the market generates hundreds of thousands of
dollars _a month_ in revenue.

I deal constantly with nefarious developers who often feel like Broadcastify
has an obligation to grant them a license. We had a terrible dust up this
weekend with a developer that was upset we wouldn't grant him a license to
develop "yet another police scanner app" and he went on a warpath with me
personally.

The app store environment is a major pain and exhausting...

~~~
ketzo
Seems like y'all are definitely positioned right in the market, though, if
everyone's using you! Do you do any marketing aimed specifically at developers
of these apps?

------
nogbit
This is so wrong it needs to be taken offline. The Translations aren’t even
close and that’s dangerous.

I seriously looked into doing this myself, but for this reason and costs I
bailed.

------
inamberclad
The emergency-service voice transmissions in the clear around me are fire and
medical. I am presuming that the busy digital signals in the same VHF band are
police.

------
abrichr
This is great. Is it open source? I would love to know more about how it
works. Where do you get the source data? What is required in order to add more
cities?

Thanks for sharing!

~~~
yasoob
I would assume that its piping the audio output from a radio receiver to some
live transcription API. Would also love to find out how it actually works.

------
whoisstan
Very useful mapped version in NYC
[https://scanmap.mobi/NY/](https://scanmap.mobi/NY/)

~~~
smokelegend
This is awesome! This should be crowdsourced like piaware...

------
m0zg
Speaking of which, does anyone know the _frequencies_ for police scanners in
Seattle? I have an SDR box, but Googling yields nothing of value.

~~~
smokelegend
Radio reference [dot] com

Listed frequencies by federal/state/county/local departments

Searchable for all.

------
busterarm
Which channel though? In each of these cities there are several frequencies...

------
vondur
Our local PD is going to be encrypting their communications soon I believe.

------
flaque
I’m so glad someone did this

------
quickthrower2
Looks like DNN speak.

