
Show HN: WebRTC implementation for Python using Asyncio - jlaine
https://github.com/jlaine/aiortc/
======
znpy
OMG this is a godsend! I've wanted to do webrtc outside of a browser in python
for a while now but there was no available implementations (besides the native
one, and I didn't want to go down the swig route).

Some questions:

Can it handle the signalling on its own?

Can it substitute any part of a webrtc session? That is, can I have either
browser-aiortc, aiortc-browser or aiortc-aiortc?

Can it be used to build, say, a desktop application doing webrtc
communication?

~~~
tw1010
Naive question: what's cool about doing webrtc outside a browser?

~~~
Immortalin
If you are trying to build e.g. Twitch or Discord, you want to have a server
backend to handle processing, hide user IPs from each other for privacy etc.
(the term you are looking for is single forwarding unit). If you just want to
get something up quickly, I recommend OpenVidu.

~~~
pmalynin
If you're hiding everything behind a server, couldn't you just use WebSockets?
I thought the whole appeal of WebRTC is the whole peer to peer setup.

~~~
gfodor
WebRTC provides audio and video streaming, as well as unreliable/unordered
data transport through data channels, which is really great for games.

------
Orphis
Nice work! As a WebRTC implementer (both native and in Chrome), I applaud the
work!

But I can only notice quite a bit of mismatch between your APIs and the
current spec. Do you plan on updating your implementation to match it?

~~~
jlaine
Is there any specific area you spotted which would need some love? Modern spec
compliance is most definitely a goal (e.g. RTCRtpTransceiver has been there
from the start, aiortc uses the "modern" SDP form for data-channels), so any
pointers are greatly appreciated!

I've had some great interactions with the Mozilla crowd (hey Lennart, hey
Nils) - looking forward to more of the same!

~~~
Orphis
Anything related to RTCRtpSendParameters (previously RTCRtpEncodingParameters)
really.

For example, RTCRtpSender.getParameters is missing and the setParameters seems
to be missing (unless that's the send function). And as far as I can tell, you
are not supposed to call setParameters without getParameters first, so that's
not spec compliant ;)

And of course, the sendEncodings property of the init parameters in
addTransceiver is missing. It is required to setup simulcast (and later SVC
when we standardize it).

~~~
jlaine
Duly noted, I've opened an issue to track this:

[https://github.com/jlaine/aiortc/issues/76](https://github.com/jlaine/aiortc/issues/76)

aiortc's RTCPeerConnection indeed uses RTCRtpSender.send to apply parameters.
This is directly inspired by ORTC:

[http://draft.ortc.org/#dom-rtcrtpsender-send](http://draft.ortc.org/#dom-
rtcrtpsender-send)

~~~
Orphis
Also, my take on this is that you should probably not mix WebRTC and ORTC
APIs. Those are quite different (even if there are similarities).

If you want to implement ORTC, fine, but you should probably do it in another
package and make sure you don't expose fields in WebRTC that aren't part of
the standard. Some fields (like the SSRC field) is not exposed for good
reasons.

------
shazow
This is super cool, great work! It's the second (mostly-)language-native
implementation[1] outside of a browser that I'm aware of, both started this
year despite the WebRTC spec being finalized about 5 years ago.

It's really important for libraries like this to exist so more people can
build apps without binding to massive hard-to-compile browser codebases. Thank
you for working on it!

Also kudos for going asyncio and Python 3 right from the gate.

\--

[1] At least semi-functioning with DataChannel support. Lots of started
attempts that have never gotten this far. The other implementation is in Go:
[https://github.com/pions/webrtc](https://github.com/pions/webrtc)

------
gfodor
Since a lot of WebRTC devs may be in this thread (and I personally didn't know
of this project nor
[https://github.com/pions/webrtc](https://github.com/pions/webrtc)) -- any
erlang/elixir implementations out there?

~~~
evadne
The members of Slack‘s Calls team have written their own and they have spoken
at SF HTML5 meet-up but nothing else publicised AFAIK.

Also somebody has forked erlyvideo to deal with WebRTC

------
ndesaulniers
Wow! This is hard to do! I got most of an ICE implementation done in Node.js
then got completely hung up on DTLS. Kudos!

------
pranaysharma
Hi can the author or someone point to a hello world code using end-to-end of
the above code?

~~~
jlaine
There are several examples available from the project's github repo:

[https://github.com/jlaine/aiortc/tree/master/examples](https://github.com/jlaine/aiortc/tree/master/examples)

I personally like the "server" and "apprtc" demos as you can easily talk to a
browser. They illustrate slightly different things:

\- "apprtc" relies on a third-party signaling service, and shows you how you
can play media from a file / record it to a file, or generate video frame-by-
frame.

\- "server" shows you how you can combine media and signaling into a single
Python-based server, and how you can apply image processing on the fly to the
received video.

------
simmons
Great work! I can really appreciate the effort you likely put into re-
implementing SCTP, since I'm currently working on implementing SCTP in Rust
for the same reasons (WebRTC support). :)

~~~
jswrenn
I'm really eager for a SCTP/WebRTC implementation in Rust! I took a stab at
it, and re-implementing SCTP was the obstacle that stalled me.

Is your work on github/gitlab?

~~~
simmons
Yes [1], but it's incomplete, needs a lot of TODO resolution and cleanup,
(insert other scary disclaimers here), etc. I also have some slides from a
presentation [2].

[1] [https://github.com/simmons/webrtc-
sctp](https://github.com/simmons/webrtc-sctp) [2]
[https://cafbit.com/post/rust_webrtc_data_channels/](https://cafbit.com/post/rust_webrtc_data_channels/)

------
gfodor
This is really impressive! Seems like a great way to learn WebRTC and
potentially is going to be much more portable than the reference C++
implementation.

~~~
stochastic_monk
Isn’t C++ generally more portable than python? All you need is a sufficient
compiler, as opposed to a correctly set up Python installation of the correct
version.

~~~
Longhanks
Well, CPython is C89, that level of compatibility is hard to beat.

~~~
todd8
I suppose many readers of HN already know this, but CPython is the name of the
original and most popular implementation of Python; it is the implementation
found at python.org. It is written in C and hence is called CPython.

------
ArtWomb
Great work! Thanks for this. Scripting WebRTC is always a hassle. And the CV
integration will make things easier ;)

~~~
jlaine
Absolutely, might as well enjoy the rich Python ecosystem!

Starting with aiortc 0.9.9, there is now also deep integration with PyAV for
all your FFmpeg needs.

\- At the lowest level : aiortc uses PyAV's AudioFrame and VideoFrame class,
and PyAV is soon to gain more ndarray converters:
[https://github.com/mikeboers/PyAV/pull/415](https://github.com/mikeboers/PyAV/pull/415)

\- At a higher level : aiortc provides MediaPlayer and MediaRecorder classes
to read or write audio/video

------
bkovacev
Amazing stuff! Since I mainly do python, I'll definitely dig into this over
the weekend.

Can this library in the current state record audio of both local and remote
streams at the same time and could you mute a user for every participant on
the call?

------
Dowwie
Can you comment on any performance hotspots due to Python (are there any)?
What wouldn't this library be suitable for?

~~~
jlaine
So far the main CPU hog is (unsurprisingly) media encoding, especially video.
Luckily this occurs off the main thread, inside C function calls. Unless I'm
mistaken the GIL is released, so there is no Python-specific downside on this
specific point.

I've found heavy usage of asyncio.Queue to bridge consumers / producers does
not lead to amazing performance, so I'm trying to cut down on that pattern in
favour of.. callbacks (I know - not very asyncio-ish). A good testcase for
testing performance without media encoding is transferring large amounts of
data over datachannels (see datachannel-filexfer example). I usually hit ~
70Mbps on the loopback interface, which is more than I've managed to get out
of browsers.

I can't say I have experience with using aiortc at scale yet, so there are
almost certainly so hotspots I haven't seen.

------
rjsw
Would have thought that using kernel SCTP would be much faster.

~~~
detaro
WebRTC afaik uses SCTP wrapped in UDP, not SCTP directly.

~~~
rjsw
Doesn't have to wrap it in UDP. Seems to be just an expectation that firewalls
can't handle SCTP directly so you will need to wrap it in something else.
Extending a firewall to understand SCTP isn't particularly difficult.

In any case, the SCTP reference implementation in FreeBSD can do RFC 6951
encapsulation, someone could add it to Linux too.

~~~
detaro
It's specified as wrapping in DTLS/UDP, so implementations have to do that if
they want to be interopable. The "just an expectation" is sadly a well-studied
fact.

That FreeBSD can do that is neat, but just it doing that is pretty much the
answer why a library wouldn't implement it, or at least not as a priority.

------
comesee
This is super cool. For what are you using it?

------
londt8
Is ORTC used by something else than Skype?

~~~
jlaine
ORTC is what Microsoft has chosen to implement in Edge, so if you use WebRTC
in Edge, you're most likely using the WebRTC-over-ORTC adaptor.

In short ORTC is: break WebRTC into objects you manipulate directly, instead
of everything being instanciated via RTCPeerConnection. There is significant
overlap between the two specs, for instance RTCRtpReceiver / RTCRtpSender.

------
jgh
Awesome work!

