

Ideas for Justin.tv RFS from JTV - justin
http://blog.justin.tv/2009/10/y-combinator-wants-justintv-api-ideas.html

======
Keyframe
I worked as a director on (don't shoot me) Big Brother TV show. We had
numerous live feeds (40+) with capability of recording only two channels which
consisted of two cameras each at the time. It was of premium importance to me,
as a director, to choose which streams we would record. When I delegated to my
switchers which cameras and persons should be on streams I would bounce myself
between other "stories" developing in the house.

How is this relevant to justin.tv? Live television/streaming would be
impossible for me to do without a 'tool' I had, which was loggers. Loggers
were basically two persons, one delegated to each stream that would vaguely
transcribe what was happening on the stream via simple logging software.

Logger would watch two cameras (master plan and close ups that were chosen by
me and operated by switcher and remote camera operator, or cameraman behind
mirrors) from the stream and listen to it and then would hit Alt-m in a
textarea sort of control to enter current SMTP timestamp and start writing
what was going on or transcribing a dialog.

I think this kind of primitive, yet powerful feature for logging what is going
on can be of use for justin.tv in more ways then one - lectures, spiders could
crawl through text, one could skim through a transcription and have SMTP links
into the prerecorded video.

Of course, not every video will have willing logger to do so - but premium
content would most probably have. Look at avid pirates transcribing subtitles
for pirated movies and stuff. I can only imagine someone transcribing a
webcast lecture, or a company sponsored live event being transcribed.

just my 2c.

edit: Now that I think about it - justin.tv could be ideal platform for
casting. Private one to one sessions with casting apprentices or even director
should not be a problem with remote cam, since casting is usually one person
with lame camera and one person in front of that camera acting out directions
from the person behind camera or making poses, headshots and stuff like that.
You could have an option to make it somewhat open, so people can vote who they
like most and stuff like that. This is more of a product idea than API/tool. I
have much more, I'll stop now since this was not a question. But, you smart
guys here - someone should build something like this on top of justin.tv (or
them themselves) and offer someone like shots.net a percentage for marketing.

~~~
Keyframe
SMPTE, not SMTP(!) - what is wrong with me :) sorry

------
DarkShikari
_Real interactive gaming_

 _Customer service application_

 _Video dating_

The real problem here is always latency. Even if you're not doing Onlive-style
stuff, videoconferencing is hard. I doubt the standard Flash toolchain (webcam
+ Flash + etc) can do it all with acceptable latency. Current Justin.TV only
works IMO because you're communicating via chat in one direction and via video
in the other; a real two-way communication link gets really awkward even at
500ms latency. When people do videoconferencing, they expect it to be like
their cell phone, not like a call to the moon.

We at x264 have been working on this for some time for... a certain company
that wishes to stay anonymous. Right now you can get down to about 9
millisecond total end-to-end latency for 800x600 video (not counting client
and network latency, just server-side), which works pretty damn well.

But dropping encoder latency isn't enough; you need to have a system built
around it that has tolerable latency. VLC, for example, doesn't; it's not made
for two-way communication, and so it has a 5-15 second internal buffer. It
ends up requiring a lot of custom-rolled toolchains, even if the basic
libraries already do what is necessary.

~~~
jwecker
I think you slightly underestimate JTV's performance, as well as the level to
which it can be customized...

EDIT: concede the point to DarkShikari, though I still think you could make
very viable companies based on those ideas and JTV technology.

~~~
DarkShikari
I think you underestimate the difficulty of getting low latency video
connections.

For example, Flash by default buffers _at least 8 frames_ for decoding. This
means you instantly have 1/3 of a second of latency for a 24fps stream.* Most
live-streaming encoders use a VBV buffer size of about 1-2 seconds--very few
are even designed to go below about half a second.

When you need to go low latency, every millisecond adds up, and it becomes
painful fast.

*There is a (very ugly) hack to somewhat reduce this. I don't think I'm allowed to talk about it due to my current contractual obligations, but if you're experienced in video streaming, perhaps you can figure it out.

EDIT: I agree, it's possible, but JTV might have to modify their internals to
make it possible.

~~~
Retric
First thought "Decouple sound from video."

Next thought "send video directly between computers."

But for a truly ugly hack, double each frame on the client side so flash
thinks it's 48FPS video.

~~~
DarkShikari
_But for a truly ugly hack, double each frame on the client side so flash
thinks it's 48FPS video._

You're getting close ;)

------
rabidsnail
Here are some useful links for people thinking about building stuff with the
justin.tv api:

<http://apiwiki.justin.tv>

<http://www.xbmc.org/trac/browser/trunk/xbmc/lib/libRTMP>

<http://ffmpeg.org>

<http://www.videolan.org/developers/x264.html>

<http://linux.bytesex.org/v4l2/>

<http://videolan.org>

I work at Justin.tv on the api, btw.

------
evansolomon
These are from a list we made over the summer while getting ready to launch
our API. Now that the RFS is official, it seemed like a good time to share
them with everyone and solicit some more ideas.

------
pclark
I'd love to see live gigs use this stuff.

------
chaosprophet
The one about "real video games" is extremely interesting (and enticing).
However, most of the game ideas I have now are set in AUs or require
supernatural abilities. I'm going to think about a good game idea for this
kind of gameplay.

Perhaps something like Augmented Reality games using the video feed from your
opponent??? Imagine your character having to parkour through your opponents
living room...

------
abstractbill
The hardware broadcaster was my suggestion. It should be dead-simple, with
inputs for video, power, and ethernet - configurable through a web interface,
just like a router.

If you can make it cheap enough (< $100 would be my guess), we have a lot of
gamers who I believe would buy one. You wouldn't have to write much software -
for example vlc can already stream to jtv.

~~~
grinich
_"vlc can already stream to jtv"_

Are you guys looking for hardware or software encoding? I thought you were
talking about something like an FPGA or ASIC.

~~~
DarkShikari
The real problem with ASIC solutions is that they generally have terrible
compression; you usually don't get good results until you get up to the range
of tens of thousands of dollars (for real broadcast encoders). Combined with
the utter lack of upload bandwidth that most users have, this could become a
serious problem.

The best option I could see in the near future is ARM's recently-announced
2Ghz quad-core CPU; it's low-power enough and hopefully cheap enough to stick
in a small box but fast enough to run a good encoder at a good resolution. But
even that would probably put the cost well above $100, at which point you
might as well start selling capture cards and using peoples' CPUs instead.

------
jwecker
Keep in mind as you brainstorm that there are additional API's coming out,
including a pay-per-view API, subscriptions, etc. Also integrated ad networks
if desired. Monetization shouldn't be a big problem moving forward. I'm sure
there are some brilliant ideas waiting to be discovered involving PPV...

