1: Did they re-encode the input? Or just repackage it?
2: I'm unfamiliar with EvoStream, but if it is ingesting RTMP and outputting HLS, why did Globo need to bother generating an HLS manifest? Couldn't they just use the one EvoStream created?
1: We just repackage it.
2: The problem is that EvoStream stores the manifests and chunks locally and we needed high availability. That's why we use an external data storage. We have had up to 30 simultaneous streams, with 7 bitrates each and 2 hours of DVR.
When our CDN was very crowded, one tag on the HLS playlist was able to direct users to a lower quality, preventing all the users to fight for the same bandwidth and avoiding rebuffering events.
What is the benefit of EvoStream over nginx-rtmp?
I use nginx-rtmp as well and it creates HLS files just fine.
Also, how did you handle the network load? How many edge servers were actually delivering the content to users?
Did you use 10G ethernets, link bonding, or did you use cloud services (eg cloudfront) to deliver the content?
Thanks a lot.
* We used 20G ethernet, we could get 19G from each machine.
* All the content is hosted by us.
Cassandra response time was increasing with load to a certain point where clients started to timeout and the video playback completely stopped.
After these changes, we were able to achieve a latency in the order of 10ms for our 99% percentile.
You can send metrics from Cassandra to graphite
Since the streaming is similar to HTTP page flow, it's not that hard.
The architecture up from what was described is almost purely to provide caching capabilities, aka bunch of edge servers caching content to final users.
Why not just use Akamai for RTMP ingest and delivery? From my experience they can do all that was described and at scale.
About why chose HLS over RTMP, in our experiments HLS showed to be easier to scale than RTMP (maybe because the latter is stateful + the current players don't have an optimized adaptive bitrate algorithm + it's easier to scale http over rtmp)
* We were only allowed to broadcast to Brazil.
* We already have all the servers and needed bandwidth, why would you pay when you already have all the servers?
This documentary is still banned in Brazil and can't be broadcast offline anywhere.
There's no denying the extent of Globo's influence. But, compared to the 80's, it's merely a shadow of it's former self. Probably because of the internet. Also, there are other communication vehicles fighting for eyeballs now.
Politics aside, its IT arm 'globo.com' is pretty modern and open.
The ire lingers, though. Sometimes Globo shows up to report on something and they essentially get kicked out of the area (by the people near the event). This may be a combination of how it tends to report on certain subjects and leftover distrust from the past decades.