
Twilio Video – Real-time WebRTC video infrastructure - timdorr
https://www.twilio.com/video
======
jc4p
I was really excited by the title but this seems a bit lacking.

I've done a lot of WebRTC work -- we currently use Google Hangouts everywhere
at Stack Exchange for internal team communication, but the 15 person cap hurts
some bigger all-hands calls or our weekly remote parties. I don't think the
hardest part of WebRTC is transcoding or TURN servers (what Twilio seems to be
offering), multiplexing is where all the complexity comes from.

It's very easy to set up a video chat interface via WebRTC small-scale, and
good on Twilio for focusing on making it easier to integrate mobile clients
too, but I can't wait until someone (most likely the Talky.io team seeing how
their betas are going) makes it a bit easier to have large N connections
(right now on Hangouts N is 15, on my own stack N is around 10).

~~~
hootener
I think you've hit the nail on the head, here. WebRTC is really great for a
handful of people. Implementation using some third party was never the
difficult part, but scale was and is still a total nightmare.

I'd love to see WebRTC done by a service provider in a highly scalable way, so
I can stop relying on RTMP (and typically embedding a flash player to support
it) to deliver live streaming audio/video on a large scale.

As someone who has only leveraged WebRTC through third party providers (e.g.,
OpenTOK, etc.), I have no idea the complexity of what I'm asking for. Perhaps
it's boil the ocean difficult, but it sure would be nice to have!

~~~
jgrowl
Yeah, I don't think that it is boil the ocean difficult but there is
definitely some complexity there. I have been working on a scala WebRTC
server, but I am not sure if it would really work at a large scale

Here was my basic approach:

An end user creates a PeerConnection with a Publisher node and starts sending
a MediaStream using a string identifier.

Then another user can create a PeerConnection with a Subscriber node using the
same identifier.

The Subscriber node then makes a request to the Publisher to make another
PeerConnection to a Registry that exists on the same node as the Subscriber.

The replicated MediaStream can then be attached to the Subscriber.

Since the replicated MediaStream is in a Registry, any additional subscribers
can attach the MediaStream on the same node as well.

The code is a huge mess but it is here (The scala server is in the media
directory):
[https://github.com/jgrowl/livehq](https://github.com/jgrowl/livehq)

There is a vagrant file that brings up the whole system at the root of the
project (using docker). I have not tested it recently on anything other than
ubuntu.

------
dirtyaura
I did a lot of testing of a few WebRTC platforms last summer for a product
idea that we had.

I came to the conclusion that on mobile phones, WebRTC video is not yet
usable. Even with just audio on a cellular network, call quality started
deteriorating after a few minutes on every platform that I tried. Video
without a WiFi signal was hopeless.

I discussed this with an WebRTC expert, and the problems are partly with
WebRTC, partly with how mobile operators shape their data traffic.

Also all platform implementations that were considered the best of the breed
were bought by mobile chat companies (both SnapChat and OK Hello did
acquisitions on this front)

~~~
gz5
The issue is mostly due to the nature of the mobile IP, rather than inherent
WebRTC variables.

However, ORTC does seek to improve with simulcast/SVC, and attributes of ORTC
will merge into WebRTC as well, ultimately providing more hooks, finer-grained
control and better instrumentation/visibility to the upper layers.

All that said, you can do acceptable quality WebRTC voice and video over 4G
LTE today. But you do need a rock solid signal and be prepared to with a fully
charged battery if you are not plugged in.

~~~
pthatcherg
ORTC doesn't really have anything to do here, except nicer control over some
configurations that you can already control with SDP munging.

Simulcast/SVC, for example, are irrelevant outside of multiway video. And you
can already do simulcast with WebRTC.

------
lucaspiller
Although it's great to see Twilio branching out into this space, with their
recent product releases they do seem to be losing focus on their original
Twilio Voice product. Compared to other prodividers their prices really aren't
conpetitive, but Twiml is so much more flexible and easier to use than
VoiceXML. What to do? :/

~~~
RobSpectre
Rob from Twilio here - appreciate the feedback.

Have you had an opportunity to check out TaskRouter yet? We released it a few
weeks ago - definitely my favorite recent product that works with Twilio
Voice:

[https://www.twilio.com/taskrouter](https://www.twilio.com/taskrouter)

~~~
tjbiddle
Interesting; however in my opinion I'd redesign that page. Having a lot of the
information behind the interactive scrolling will cause it to be lost.
Personally, and I know a lot of others, just scroll the page and look for the
relevant information in paragraph form. Much easier to digest. This
implementation forces me to stay on the page and is a bit frustrating.

~~~
ayushgta
I just posted the exact opposite of what you said :) Usually these "scroll
hacks" are annoying. In this particular case the "flow" of calls is captured
really well this way, IMHO.

~~~
tjbiddle
Heheh, to each their own :)

------
anthony_franco
First thing this reminded me of was
[https://tokbox.com/](https://tokbox.com/). Hopefully the pricing is along the
same lines, it's good to see some competition in the space.

------
talltofu
Awesome. Will try it soon. What would HN readers recommendfor near zero
latency audio only? Our client needs to connect two devices within a LAN (no
throughput issue) but near zero latency. We have tried Twilio PSTN, other
webrtc technologies but they all have noticeable (half a second) delay from
device to device. Any thoughts?

~~~
joeyspn
You could try a SIP client with a low latency codec like CELT...

~~~
talltofu
Thanks for your reply. Do you have any recommendations?

~~~
joeyspn
Yes, jitsi for desktop and if you need mobile, csipsimple is a nice option

~~~
talltofu
Thanks Joeyspn. I am a noob when it comes to SIP and have some specific
questions. For example, which of the jitsi product I should be using? If this
is not the right place to ask such questions, can we please take this offline?
I could totally use your knowledge

------
Animats
How available is IPv6 handset to handset connectivity? I understand that
T-Mobile offers it, and Verizon can but it's usually disabled. If you have a
true peer to peer connection without NAT, more options become possible.

~~~
pthatcherg
If you can get a peer-to-peer connection, it's irrelevant whether you're
behind a NAT or not. That's what ICE is for: to get a peer-to-peer connection
even if you're behind a NAT. Having publicly routable IPv6 addresses may
increase the likelihood of getting a peer-to-peer connection, but it's not a
prerequisite.

~~~
Animats
Having a publicly routable IPv6 address means you don't need some paid service
just to get connected.

------
gz5
The support for H.264 is interesting.

It of course makes sense for iOS and devices that can support hardware
acceleration, but device mix can change during a call. So, what happens if two
iOS users start, but then they add a VP8 only device? How Twilio handles that
use case will be instructive for understanding what use cases we can support
with this service.

It isn't likely they are transcoding since this intro suggests a P2P
architecture without distributed media servers, and we know H.264/VP8 is very
expensive video transcoding - e.g. many media server vendors don't get more
than one transcode per core at full HD.

~~~
jumpchat
I just added h264 to jumpchat on ios and android. Say A(iOS) talks to
B(Android). They both support h264, they negotiate to use h264. C(chrome)
joins. A triangle connection is setup where A will talk to B using h264. A & B
will talk to C in VP8. It's kind of taxing on mobile when this happens.

------
kolencherry
Are there any plans for Twilio to integrate this with the SIP side of things
(e.g. Twilio providing a media server to enable WebRTC clients to communicate
with SIP clients and also call out to traditional phone clients?)

~~~
andrewkirk
I wrote this [http://www.siptowebrtc.com/](http://www.siptowebrtc.com/) that
uses our SIP interfaces and PSTN to connect to our WebRTC client. I'm going to
put the code up on Github after I clean it up but I'd be happy to talk through
how it works. andrew at twilio dot com.

------
dskae
I'm pretty new to using WebRTC, but I was able to implement it really easily
using icecomm.io's wrapper. I'd have to look into twilio some more, but
icecomm has been more than effective enough.

~~~
gg_hacker
Agreed. Out of all the WebRTC products I've tried, icecomm.io was the easiest
one for me to use.

------
marquis
A tech question for Twilio: can you manipulate the SDP object?

~~~
RobSpectre
Good question - one of Twilio Video's aim is to take a lot of the signaling
headache away. We do expose the PeerConnection, but not the SDP object.

What would you want to use it for?

~~~
JoeAltmaier
Extra authentication; new roles; address filtering (recognize VPN path etc);
bundling control...

~~~
RobSpectre
Check. Good feedback.

------
jumpchat
There goes my project. Back to the drawing board. Haha!

------
andreyf
> Provides multi-party authentication, registration and signaling, which can
> orchestrate up to 4-way calling using peer to peer mesh topology, across any
> combination of supported devices. Calls can be video or voice only.

Peer to peer mesh topology? Anyone know of further reading on this?

~~~
jc4p
"up to 4-way calling using peer to peer mesh topology" sounds like fancy names
for regular WebRTC connections.

~~~
RobSpectre
Rob from Twilio here - confirming this is an accurate description. Each peer
in the conversation has a connection to every other peer.

~~~
andreyf
Is there any chance of scaling WebRTC any better than that?

~~~
RobSpectre
Absolutely - we have a lot of ambition around where we want to take Twilio
Video in the future.

This is only the first step.

------
Everhusk
It would be useful to know which browsers are supported for javascript
developers. I am using OpenTOK for Real-time WebRTC at the moment which works
great, but it doesn't have safari support. Would definitely consider switching
if this worked on all browsers!

~~~
kolencherry
Chrome and Firefox are the only browsers supported WITHOUT plugins. Look into
the plugin that Temasys makes, which enables support for IE and Safari. It's
relatively trivial to write a shim for said plugin.

IE will bring native WebRTC support in the next version of IE.

~~~
jumpchat
WebRTC on Opera works without plugins also.

------
fit2rule
Anyone know how this compares with icecomm?
([http://icecomm.io/](http://icecomm.io/))

I was quite impressed with how easy it was to set up icecomm .. are twilio
upping the ante here in any way that I should know about?

~~~
skotzko
Was gonna ask the same — had a really good experience w/ IceComm when I tested
it out a few weeks ago

------
anonbanker
WebRTC needs p2p for group chat, and it'd be unstoppable.

Currently using Hello for one-on-one conferencing, but need something that
both scales and doesn't require a centralized server. Considering Skype had
this ten years ago, it's obviously doable.

~~~
TD-Linux
WebRTC already has p2p support for group chat. appear.in is one example of an
implementation.

The main limitation is that upload bandwidth scales linearly with the number
of people in the group chat. But that's often OK for up to 10 or so people.

~~~
tommoor
The limit is more like 3 or 4 max. Anything about that and a modern machine
starts choking on the video encoding anyway :)

~~~
JoeAltmaier
I think the OP is correct. The video is encoded only once for all viewers. Its
the uplink bandwidth, traditionally a choke point, that holds the practical
limit at 3 or 4.

------
bretthellman
"up to 4-way calling" .. If it said up to 16 that would be more of a game
changer.

~~~
matthewaveryusa
MVP, got to start somewhere right?

------
higherpurpose
Is ORTC still being implemented in the next version of WebRTC? I've read that
ORTC needs a greatly simplified implementation (if you forget about all the
other legacy stuff in WebRTC, such as support for the phone networks).

~~~
kolencherry
I believe that ORTC has been added to the WebRTC 1.1 specification.

~~~
pthatcherg
There is no WebRTC 1.1 yet. 1.0 isn't even done yet. Once 1.0 is done, then
work on 1.1 will start. ORTC maybe be adopted in whole or in parts as 1.1, but
the future isn't written yet.

~~~
kolencherry
Good point. You're correct on both areas.

------
zk00006
Would be nice if they support some video tracking capabilities. In the demo
video, the arrow pointing to the slot is "asking" to be stack to the slot
independent of the camera motion.

------
JoeAltmaier
Doesn't CliqMeet do much of this? But it can go to 100+ because it provides an
MCU service (multiplexing node), plus auditorium models as well.

Is the difference that this is SaaS, and CliqMeet was an app?

~~~
kolencherry
CliqMeet is a video conferencing product. Twilio is offering an API to allow
you to build your own products with WebRTC functionality built-in.

------
treme
damn twilio. building them pipelines.

------
curiously
can you use this to build something like twitch.tv?

~~~
joeyspn
For twitch.tv clones you're better off researching about streaming media
servers... (and cheap bandwidth hosting)

[http://en.wikipedia.org/wiki/List_of_streaming_media_systems](http://en.wikipedia.org/wiki/List_of_streaming_media_systems)

~~~
curiously
is there an open source streaming media server?

could you host this on digitalocean?

------
aylarov
We had this in VoxImplant for a few months already :)

