
Show HN: WebRTC signalling data using QR codes - AquiGorka
https://github.com/AquiGorka/webrtc-qr
======
phiresky
Similar project of mine that connects a phone to a PC browser using a WebRTC
QR code so you can use a pen / touch on the phone to create drawings on the
desktop:

[https://www.youtube.com/watch?v=Gvsm84xL9Sk](https://www.youtube.com/watch?v=Gvsm84xL9Sk)
([https://github.com/phiresky/webrtc-remote-touch-pen-
input](https://github.com/phiresky/webrtc-remote-touch-pen-input))

Sadly it needs a (tiny) server component to exchange the initial connection
information - using a QR code in both directions is smart, but also pretty
annoying for the user.

~~~
AquiGorka
I hear you. It is not the best user experience I agree, but it does take away
the (tiny) server component. I will try to improve the experience.

And by the way, cool idea to use the pen on the smartphone.

------
TomasHubelbauer
This is great, thanks for posting it. I've had the same idea myself and
started and abandoned this exact same project a few times. Same down to
cutting the SDP up into multiple QR codes. I am glad to see this brought to
life.

I am also surprised to see @nilknarf has done that in 2014! (As per their
comment.) Don't know how I didn't come across his post during my research into
this, but checking it now, someone in the comments posted a link to a patent
which captures this idea.

[http://www.freepatentsonline.com/20160021148.pdf](http://www.freepatentsonline.com/20160021148.pdf)

The patent must be invalid, because it's from 2016 and there is clearly prior
art, but it kind of sucks, because it basically means polishing something like
e.g. @phiresky has done with the remote pen just means if you succeed in
bringing down the time to connect and make it something usable enough for
users, you're opening yourself up to a claim if you decide to package it up
into an app.

~~~
AquiGorka
> opening yourself up to a claim if you decide to package it up into an app

Even though there is public domain tech from 2014 using it? (@nilknarf's).

I was under the impression when something is part of the public domain there
can't be a patent claim on it.

~~~
TomasHubelbauer
I think this patent has been mistakenly granted this year:

[https://patents.google.com/patent/US20160021148A1](https://patents.google.com/patent/US20160021148A1)
(see the "grant" status line)

I think it should not have been granted there being prior art, but for it to
be turned over, I think it would have to be challenged in court. That's my
limited understanding as a non-US person without any experience with patents
whatsoever.

~~~
AquiGorka
Oh wow, I am shocked to see the patent was granted and I'm very interested in
understanding the repercussions. These might be big.

------
garlicfish
Nice!

I also now of this Android App that uses WebRTC Audio/Video without server (no
Internet or DHCP needed):
[https://github.com/dakhnod/Meshenger](https://github.com/dakhnod/Meshenger)

I made a submission, in case somebody likes to discuss the need for serverless
WebRTC:
[https://news.ycombinator.com/item?id=18203495](https://news.ycombinator.com/item?id=18203495)

(Disclaimer: I am not the author of the App, but I helped out a bit)

~~~
AquiGorka
> Contacts are found by taking the devices own IPv6 addresses and replacing
> the devices own MAC address by the contacts MAC address.

You require the devices to be on the same network for this to work right?

~~~
garlicfish
Yes, that is (mostly) true.

The App uses ipv6 link local addresses (fe80::.. + mac address). But it also
tries to make use the received IPv6 network prefixes. Also, the last contact
address is stored. These approaches improves the probability a bit to find the
contacts in other networks.

As a side note, many mesh networks are emulating a layer 2 network. With this,
everybody can build up their own decentralized telephone network.

~~~
AquiGorka
> everybody can build up their own decentralized telephone network

Nice. I'm very interested in being able to connect devices in areas where
there is no Internet and using a mobile router, like in the mountains or in a
bus trip through deserted areas.

------
kodablah
I've wondered about other signalling mechanisms. I figure you could use a DHT
(js-ipfs/js-libp2p or webtorrent BT mainline) and put the offer directly on
the DHT (AES'd?) or if it's too big to be a key, seed it as the stored value
for its hash using those systems.

~~~
AquiGorka
I've worked with `js-ipfs` and their `js-libp2p`, they use a signalling server
(actually two types of them):

\- [https://github.com/libp2p/js-libp2p-webrtc-
star](https://github.com/libp2p/js-libp2p-webrtc-star)

\- [https://github.com/libp2p/js-libp2p-websocket-
star](https://github.com/libp2p/js-libp2p-websocket-star)

That would be going back to using a server for the handshake.

In regards to using a DHT, that's something I have not researched. I am not
sure how long an offer is valid, were you thinking of being able to update
them as needed whenever you want to establish a connection?

~~~
kodablah
> In regards to using a DHT, that's something I have not researched. I am not
> sure how long an offer is valid, were you thinking of being able to update
> them as needed whenever you want to establish a connection?

Yes. I have toyed with js-libp2p to obtain "providers" based on a key, but I
believe the part of js-libp2p that allows you to participate in the DHT isn't
quite done [0]. I imagine you'd make a CID that would be a simple hash of some
shared string. Then you'd tell the network you are a "provider" with that CID,
and when requested, you give the offer (AES'd w/ shared string or other shared
secret?). The client can just connect to the same DHT and give the hash and
get the (encrypted?) offer. Not sure if IPFS has a way for me to prevent other
people from serving/replicating the CID I am serving instead of forcing it to
be a hash of the content. A similar approach can happen on BT mainline w/ an
infohash maybe.

0 - [https://github.com/ipfs/js-ipfs/pull/856](https://github.com/ipfs/js-
ipfs/pull/856)

~~~
smaddock
I created swarm-peer-server [1] as a way to connect to specific peers over a
DHT using a public key known ahead of time. I use this in a networked app I've
built to perform WebRTC signaling between peers [2]. The networking and crypto
works using modules popular in the Dat Project community.

The challenge is that multiple peers could announce under the same hash on the
DHT. Thus I have code to perform a handshake to verify peers upon connecting.

Another project worth mentioning is hyperswarm [3], also created by the Dat
Project folks. It's planned to be their new method of connecting peers on a
DHT in Beaker Browser.

[1] [https://www.npmjs.com/package/swarm-peer-
server](https://www.npmjs.com/package/swarm-peer-server)

[2]
[https://github.com/samuelmaddock/metastream/blob/master/app/...](https://github.com/samuelmaddock/metastream/blob/master/app/browser/platform/swarm/index.ts#L62)

[3]
[https://pfrazee.hashbase.io/blog/hyperswarm](https://pfrazee.hashbase.io/blog/hyperswarm)

~~~
AquiGorka
Thank you for sharing. Metastream is very nice! I thought I was following Paul
Frazee's blog but I wasn't (updated my feeds now) so, thanks for the link to
the `hyperswarm` post too.

------
ggerganov
Cool. In case you are interested in a similar approach using signaling through
audio (you might say an "audio QR code"): [https://github.com/ggerganov/wave-
share](https://github.com/ggerganov/wave-share)

~~~
AquiGorka
Yeah! I was familiar with it, very cool.

------
nilknarf
Did you happen to see my post on this?
[https://franklinta.com/2014/10/19/serverless-webrtc-using-
qr...](https://franklinta.com/2014/10/19/serverless-webrtc-using-qr-codes/)
(The demo unfortunately doesn't work anymore since it's from 2014)

The flow is very similar to yours, down to the flashing QR codes!
[https://youtu.be/rw4f9LfqqpI?t=8](https://youtu.be/rw4f9LfqqpI?t=8)

~~~
AquiGorka
Hey there, I wasn't aware you had worked on the same thing (and 4 years ago!).
Amazing, my hat is off for you.

Did you continue to experiment with it? Improve the experience? Try out
different approaches?

~~~
nilknarf
There were a few interesting ideas by others in the original comment thread.
Mostly on how to compress the sdp so you can get away with just one large QR
code without flashing (for example by presharing a dictionary or stripping the
sdp down to the bare minimum: [https://webrtchacks.com/the-minimum-viable-
sdp/](https://webrtchacks.com/the-minimum-viable-sdp/)).

I lost interest in this approach because it was way more practical to give up
on serverless in exchange for only scanning in one direction instead of both
ways (e.g., scan a QR code once on mobile to both open the webpage and join a
websocket channel, then use it to upgrade to a webrtc connection). It's a neat
trick for use cases like AR.js where this is natural to do:
[https://github.com/jeromeetienne/AR.js](https://github.com/jeromeetienne/AR.js)

------
arcbyte
Doesn't work in Edge because createDataChannel isn't supported:
[https://developer.mozilla.org/en-
US/docs/Web/API/RTCPeerConn...](https://developer.mozilla.org/en-
US/docs/Web/API/RTCPeerConnection/createDataChannel#Browser_compatibility)

~~~
AquiGorka
Thank you for pointing that out I wasn't aware Edge has no support for sending
data over WebRTC.

------
n-gauge
Cool idea, however I could not get it to work with Chrome on a laptop as a
host (with the moving qr codes) and a S5 phone (join) also running Chrome
pointing at the laptop (it could see the qr codes on the transparent video).
It didn't connect.

~~~
AquiGorka
I'm sorry to hear you could not get it to work, this means I have to improve
the flow. I tested this using Chrome and both Android and iOS devices (chrome
and safari). The catch is that you gotta point the series of qr codes towards
the center of the camera.

------
webrtcbt
I am the author of the Android, iOS and web based Webrtc application and
developed our own media and signaling servers (C++) and customized XMPP for
chat. Currently it is in private beta but I'll let you know soon.

~~~
webrtcbt
APIs are in Go, Node.js and Java!

------
ricardobeat
Seems like you can only host, not join, from Mobile Safari?

~~~
AquiGorka
Both options work in Mobile Safari.

------
webrtcbt
I think industries are ready to adopt real-time voice, texts, and video
communication solutions to improve customer experience.

------
gardnr
Actual "serverless" webrtc. It's very unfortunate that faas products use that
misnomer.

~~~
AquiGorka
Yes! And I plan on experimenting more with this, like sharing image files and
maybe even html.

------
k__
Nice idea :D

When I played around with WebRTC for the first time I used copy-and-paste
signaling.

~~~
AquiGorka
I know! I wanted to use sound but that has been done, so tried with a
different part of the spectrum..

------
cphoover
Can I get a better description/instructions in the README?

~~~
AquiGorka
Sure, I'll update it with a better description of the flow, check back later
on today.

------
hamandcheese
Why does the QR code constantly change?

~~~
AquiGorka
I wasn't able to get all the info into one QR code, so I used a series of them
(looping) to send all the data. I figure I can use this approach to send files
too.

~~~
ggerganov
What is the capacity of those QR codes?

When I played with WebRTC, I was able to pack the relevant SDP data to just 80
bytes:

\- 1 Type of the SDP - Offer or Answer

\- 1 Packet size in bytes (not including ECC bytes)

\- 4 IP address of the transmitting peer

\- 2 Network port that will be used for the communication

\- 32 SHA-256 fingerprint of the session data

\- 40 ICE Credentials - 16 bytes username + 24 bytes password

~~~
AquiGorka
I found a similar limit (although I still want to try with different options
for the qr codes). I send the data as is by splitting it into smaller segments
and looping through them with more than one qr code.

------
adzm
Will this work across NATs?

~~~
AquiGorka
This is still WebRTC, so the same limitations apply. The QR codes are used to
share the `signalling data` for the WebRTC connection.

~~~
cjbprime
More detail: Yes, WebRTC contains several methods for routing through NATs --
ICE, STUN by default (using your browser's default STUN server), optionally
TURN.

~~~
deathanatos
My browser manufacturer has a STUN server? Where can I find that? (e.g., for
Chrome or Firefox)

Edit: AFAICT, this isn't correct. Your browser has no notion of a "default"
(or any other) STUN/TURN server. The JavaScript doing WebRTC has to specify
one.

That said _Google_ runs one, at stun.l.google.com:19302; but it isn't clear if
that's something third-parties could or should make use of (or if Google even
permits such activity), though it seems like plenty of examples (and perhaps
even real-world uses) make use of it.

~~~
cjbprime
In the past, I've seen this Google STUN server used on Chrome when none is
supplied by the user JS. This may have changed in recent years.

------
brian_herman__
This is amazing! Good job!

~~~
AquiGorka
Thanks

