
Ask HN: How to rip videos from sites like CNN.com? - skcin7
So, my brother is a film maker and I am a computer scientist and we had an idea the other day to make a documentary film about the 2012 US Presidential Election and the role that big business plays in it. Who knows if we will actually complete this. Right now it is just an idea.<p>What is the best way to rip videos from "streaming only" sites like CNN.com? An example of some footage I'd like to get is:<p>http://www.cnn.com/2011/POLITICS/09/07/republican.debate/index.html?hpt=hp_t1<p>I examined the page with a HTTP Debugger and also viewed source code but could not seem to find the original source URL of the video. I was wondering if it is possible to find the URL of the video somehow and if it is, how?<p>Thank in advance for any insight you may provide! :-)<p>Also, it has just crossed my mind that this type of question might not be acceptable here, since you're not really "supposed" to download videos from streaming only sites. If this type of question is not acceptable please politely let me know.<p>EDIT: I intend to proceed only after obtaining the copyright owners permission.
======
Skalman
Using Firefox Web Console it's possible to obtain all the fetched URLs. The
one containing the video was:
[http://ht.cdn.turner.com/cnn/big//us/2011/09/07/bts-msnbc-
pe...](http://ht.cdn.turner.com/cnn/big//us/2011/09/07/bts-msnbc-perry-romney-
socsec.msnbc_640x360_dl.flv)

I don't have a generic way of extracting this information from any site, but
looking at those network sources that are used is always a good place to
start. Using extensions such as Flashblock will let you control when you're
going to start downloading, and there will be fewer URLs to look through.

~~~
skcin7
Oh hell yes. This method works perfectly :-)

------
ChrisNorstrom
I'm not a lawyer but...

Colbert and John Stewart do it all the time and I doubt they ask for
permission for every single little clip. It would count as 'fair use'.

<http://www.orbitdownloader.com/>

Use Orbit Downloader, it rips everything, I've been using it for a year or two
now, it's amazing. Open up Grab++ within Orbit, reload the page and the video
will show up in Grab++, select it, hit download and it's yours.

~~~
skcin7
Hmmm very interesting. First of all thanks for posting that, it works
perfectly as well, using the Grab++ tool.

The method that Skalman posted using the Firefox Web Console works perfectly
as well, although in the CNN video that I posted above, that method seems to
download a .flv file, whereas your method seems to be downloading a .mov file.
Interesting....

EDIT: Both the .flv and the .mov seem to be the same file. They are both
exactly 11,840,167 bytes, so that leads me to believe that they are the same
file. It is interesting however that your method and Skalman's method download
the same exact file just with a different file extension. I wonder why this
is.

------
rhizome
For a documentary film you're going to need copyright clearances, so you
should start at the licensing office of CNN.

~~~
skcin7
I will have to look into that. Thanks for the response. Does anybody know how
to rip videos from various websites, though? I feel like it is definitely
possible. I use a program called JDownloader which can rip videos from
Youtube, MegaUpload, and various other "streaming only" sites. Somehow the
program is able to extract the URL of the video file(s) and download them
directly. This program however does not support any of the news sites
(including CNN).

At this point I am mostly curious about the theory. I am curious how a hacker
would go about finding the URL and download the files themselves. I looked at
Youtube source code and examined it with a HTTP debugger and can't seem to
figure out the URL of where the video is stored but I know it is possible
because the JDownloader program does it. I am sure that I, along with many
here, would be able to benefit from knowing how to download from various
"streaming only" video websites.

~~~
rhizome
I think HN is not a good place for this question, then. Good luck.

------
acak
I think you ought to make it clear that you intend to proceed only after
obtaining the copyright owners permission. Might stop people from flagging it.

~~~
skcin7
Good point. I agree with your suggestion and have made an edit to my post.

