

Benchmarking cloud video encoding: Zencoder, Encoding.com, Sorenson, & Panda - jon_dahl
http://blog.cloudharmony.com/2011/10/encoding-performance-comparing-zencoder.html?

======
gtaylor
Bit of a shameless plug, but we wrote our own EC2 + ffmpeg encoder, since we
didn't want to pay out the ears for encoding:

<http://media-nommer.readthedocs.org/>

Open source (BSD), beta'ish, but very much functional. Save some money! Pull
requests are welcome and greatly appreciated, as is feedback on the docs.

~~~
lobster_johnson
A shameless plug on top of your shameless plugin: We also wrote our own
encoding system, called Tootsie (think Dustin Hoffman), which is in production
use:

<https://github.com/alexstaubo/tootsie>

Written in Ruby, based on FFmpeg + ImageMagick, handles both video, audio and
photos (we use it for photo/thumbnail scaling). JSON API, easy to use. It's
designed to run anywhere, EC2 in particular. MIT license.

~~~
gtaylor
Looks cool, but are there any real central docs to look through (aside from
that README)?

~~~
lobster_johnson
No. The README covers pretty much everything you need to know. I would be
happy to explain (and rectify the readme) if there is anything that is
unclear, however.

------
Jabbles
This only measures the speed, not the quality. Encodings may vary enormously
in their complexity, you can't treat 2 encoders that produce the same
"resolution" as being equal.

~~~
DarkShikari
They (almost) all use the same encoder.

Albeit most likely with different settings, for a different speed/quality
tradeoff. And some might be using ancient versions from the dawn of the Bronze
Age.

------
spaznode
Where's the shout out to ffmpeg? Don't tell me that almost all services
involved aren't built on top of it.

~~~
zimbatm
Big shout out to ffmpeg from Panda.

As far as I know, we are the only ones to give you full access to the command-
line options in case you want to do funny stuff. The disadvantage is that we
have been stuck with ffmpeg 0.6 on custom commands to not break backward-
compatibility (which is what they used).

If they had used the presets, the encoding would have run with the latest
ffmpeg which is also a lot faster.

~~~
spaznode
Ouch! (regarding getting fsck-ed with command line options)

Before current job I built a fairly large more general media management system
which included tons of video related work (and encoding ;) ) using ffmpeg for
some fairly prominent (something something geographic something / etc) so I
definitely appreciate how powerfully awesome ffmpeg is once you get over the
initial overwhelming hump of having to learn all the more general video
concepts involved outside of ffmpeg.

Which is why seeing a "performance comparison" blog post was immediately
extremely suspicious sounding for the very valid point that everyone has
brought up that speed isn't all that matters. From my experience speed is one
of the lowest concerns anyone doing anything professionally with video has.
Quality / reliability / tweak-ability are all much more important.

If you're writing software you intend to sell for money for anything other
than cat videos uploaded to youtube - do yourself a favor and make sure you're
informed about all the core concepts before randomly picking someone. You'd be
surprised at all the hiccups and encoding issues you run across once you have
a large enough sample set of "problem videos" to test against.

------
rorrr
Sponsored by Zencoder.

As much as I want to trust this benchmark, I'm doubtful.

If Zencorer knew which account you used, they could've assigned the best
dedicated servers to your jobs.

~~~
jon_dahl
Sure, it could be a scam. But for what it's worth, it's not - we absolutely
didn't cheat. Cheating in a benchmark might work some places, but it certainly
doesn't work for a company that sells to hackers.

If you want to verify the results yourself, we're going to post the
benchmarking source code to Github this afternoon. You can review the
methodology that CloudHarmony used and run the tests yourself, with your own
account, without us having any knowledge that you're testing us. That way, we
have no way of cheating. You can even change up the files and use different
encoding profiles to be sure that we didn't pick specific files or settings
that happen to be faster at Zencoder.

~~~
sologoub
Would be interesting if someone independent could corroborate the results. The
sponsorship disclosure is appreciated, but I agree with the original comment -
raises red flags.

~~~
brandonarbini
Here's the source code that anyone can use to independently test and
corroborate the results: <https://github.com/zencoder/cloudharmony-benchmark>.
We chose Cloud Harmony to run the benchmark on the basis of their independent
reputation. However, we'd happily sponsor (pay for encoding fees) another
third-party to run the tests as well. Feel free to suggest an additional,
neutral person/company that could do this. Though the results will be the
same, no matter who runs it. :)

