Hacker News new | past | comments | ask | show | jobs | submit login
Ytdl-webserver: Webserver for downloading YouTube videos (github.com/algram)
129 points by setra on Sept 22, 2018 | hide | past | favorite | 66 comments



I use Android and have Termux[1] installed. If you create a shell script named '~/bin/termux-url-opener' and share URLs to Termux, it'll invoke that script. Here's the contents of mine:

``` #!/bin/bash

pip install --upgrade youtube-dl cd storage/movies youtube-dl "${1}" ```

Works a treat!

[1] Termux (Terminal emulator with packages) - https://f-droid.org/app/com.termux, https://play.google.com/store/apps/details?id=com.termux


On iOS I’ve installed Pythonista with Stash, pip installed YouTube-dL, set up a script to take a link and download it to mp4 and share it an app (usually VLC).

This can be triggered from YouTube app by pressing Share, then Pythonista script then the custom script, wait for download then select VLC.

It’s not great for big videos but was extremely satisfying to set up.


This is great! Does it also support play background music that you download as a .m4a ?


VLC does yes. That said the real issue is that the download part can’t run in background for more than 4 minutes, so you can’t download big videos.


Anything you can share code wise, curious to learn more.


Here's an example:

  #!python3
  """Open url in VLC — Pythonista for iOS app extension.
  
  1. take media url from appex(share)/clipboard/command-line
  2. open it in VLC iphone app
  
  e.g., to listen to Youtube video in the background.
  
  It accepts a media url from any site supported by youtube_dl 
  https://rg3.github.io/youtube-dl/supportedsites.html
  """
  import sys
  from urllib.parse import quote
  
  import youtube_dl
  
  import appex
  import clipboard
  import console
  
  import appex_webbrowser as webbrowser
  
  def open(webpage_url):
      """Play media on *webpage_url* in VLC"""
      with youtube_dl.YoutubeDL(dict(forceurl=True)) as ydl:
          r = ydl.extract_info(webpage_url, download=False)
          media_url = r['formats'][-1]['url']
      # play the url in VLC
      # vlc:// + url leads to a popup
      # https://wiki.videolan.org/Documentation:IOS/#x-callback-url
      webbrowser.open('vlc-x-callback://x-callback-url/stream?url=' + quote(media_url) )
      
  
  def main():
      if not appex.is_running_extension():
          if len(sys.argv) == 2:
              url = sys.argv[1]
          else:
              url = clipboard.get()
      else:
          url = appex.get_url() or appex.get_text()
      if url:
          open(url)
          console.hud_alert('Done.')
      else:
          console.hud_alert('No input URL found.')
  
  if __name__ == '__main__':
      console.show_activity()
      try:
          main()
      finally:
          console.hide_activity()
where appex_webbrowser.py is:

  """webbrowser.open() replacement for app extensions in Pythonista for iOS.
  """
  from objc_util import nsurl,UIApplication
  
  
  def open(url):
      app = UIApplication.sharedApplication()
      app.openURL_(nsurl(url))


Wow that’s way better than mine. I still find coding with two thumbs on an iPhone se does not fully make use of Pythonista. But there’s no way I’d buy an iPad that doesn’t have BT mouse support.


you could pass a video url retrieved by youtube-dl to VLC instead of downloading the video.


HN doesn't support fenced code blocks. You should indent code with 2 (or more) spaces to make it shown verbatim.

https://news.ycombinator.com/formatdoc


That's my workflow too, works well for many websites.

To watch I use NewPipe instead of the official YouTube client.


I find NewPipe's built-in downloading functionality more convenient - although it only works for YouTube videos of course.


Made it work. Thanks!


It's not as slickly packaged, but I made a webserver that turns YouTube channels into proper RSS subscribe-able audio podcasts:

https://github.com/frou/yt2pod

If you find yourself often having your phone in your pocket with the screen blazing, just to be able to listen to YT content, you might like it.

(it's on the todo list to Dockerize it)


Did you know iOS podcasts supports video podcasts? Haven’t looked at your code but maybe wouldn’t be too much work to make video work too. I’ll talk a look. Thanks!

I’ve thought that it would be a great way to avoid the algorithm addiction and only see my subs.


That's in fact already supported, but I didn't document it because I felt it muddied the pitch.

https://github.com/frou/yt2pod/blob/31b55410c38dbf655b641eee...

Add this per-podcast in the config file:

"vidya": true

I hear you about not wanting to spend too much time in the circus.


In my experience, YouTube-dl needs to be updated quite frequently, as YouTube changes their design.

How is this handled here? Is periodically kill / restart the container enough?


I don't know if it applies to the web server, but for the command-line version of youtube-dl, it has a -U option that updates itself. One could create a batch file or shell script that runs that each time, before downloading a video, or do it manually once in a while. Like someone else said, I think the creator updates the app fairly often.

Edit: I had written a post about youtube-dl describing its use a bit, and also a quick look at its code:

youtube-dl, a YouTube downloader in Python:

https://jugad2.blogspot.com/2013/03/youtube-dl-yourube-downl...

Also check out: pafy - Python API for YouTube:

https://jugad2.blogspot.com/2014/06/pafy-python-api-for-yout...

Edit 2: And this might be of use:

How to download PyCon US 2013 videos for offline viewing using youtube-dl:

https://jugad2.blogspot.com/2013/04/how-to-download-pycon-us...


It's not been my experience, but it is changed often enough that if you don't use it often, then yes it might give that impression.

https://github.com/rg3/youtube-dl/commits/master/youtube_dl/...


It looks like they're installing youtube-dl via apt, from the Ubuntu 16.04 repos (which is what the docker image uses as a base), do even rebuilding wouldn't cut it. I wonder if this will even work out of the box, because YouTube changed something that required youtube-dl to fix a month or two ago.


But it isn't actually using that in the app, it's using the https://github.com/przemyslawpluta/node-youtube-dl npm package.


That's just a wrapper calling the youtube-dl binary indirectly: https://github.com/przemyslawpluta/node-youtube-dl/blob/mast...



Well. That's a mess, I should have read more.


Which in turn relies on the original youtube-dl.


I added a travis cron job to update the image daily: https://github.com/Algram/ytdl-webserver/pull/6 and another guy came around and made the image even more friendly for automatic updates https://github.com/Algram/ytdl-webserver/pull/7

So if you ever run into issues you would just pull the latest container.


Same as how most containers update, rebuild a new version push and deploy.


The more tools built around youtube-dl to make it usable by your grandma, the more likely it gets shut down, like Popcorn Time.


I was just thinking the other day of writing a small utility to download a YouTube video and metadata, stuff the metadata in a database, and be able to have a local, searchable copy of videos and channels I'd like to have for a while -- I don't trust they'll be around forever.

I was thinking it could be better if the utility would be more useful if it integrated with mythtv, but I might start as a standalone.


Having a local, searchable version of some channels would be great, especially if it can be configured to download new videos automatically. Some channels are notorious for having videos deleted (e.g. commentary channels that have to fight with contentid or educational channels that have videos involving gunpowder or uranium). Manually backing them up and organizing the videos is very cumbersome.


Actually Apple provides a way to either download videos or audio directly on the device through Siri Shortcuts. The integration is great: Just open the share sheet in Safari or the YouTube app and run this shortcut: https://www.icloud.com/shortcuts/48d3103eafc04c7098094794e06... Add a convert media step if you want audio. I think I got it from this reddit thread: https://www.reddit.com/r/workflow/comments/2p0vga/workflow_d...


Just yesterday I built an Android app to cast audio-only (ogg format) from YouTube to Chromecast. Had to do it because:

1. Baby needs her white noise to sleep

2. No other sites have those "8 hours womb sounds" as good as YouTube

3. The YouTube app refuses to cast to Google Home mini

Came across ydls[1] which uses youtube-dl and ffmpeg to download then transcode media on-the-fly. It's not very effective but works great! We no longer have to turn on the TV in our baby room. Check out the source code[2] if you are also a new parent.

1: https://github.com/wader/ydls

2: https://github.com/daohoangson/android-ydls


Save the video's audio as a local file if you want to keep using it. Any YouTube video is bound to disappear without warning at any time.


Is there a way to download Netflix videos too? My reason is I just can't watch anything at less than 1.5X speed nowadays and Netflix has no such option on mobile (there is a chrome extension though). Any ideas how to do either (dl or increase playback speed on mobile)


...you can't watch anything at less than 1.5x?

Is this something you've trained yourself to do...? Like deliberately?


Yes I have. I often watch video tutorials at 4X speed (chrome used to cut off audio after that so I couldn't go over that limit).

Nowadays i even watch movies at 1.5x speed and it looks very normal to me. Once you get used to it you can't have it any other way (especially documentaries and slow movies)


It it's not uncommon to habitually listen to podcasts at that speed, and after a while you get used to it and everything else just sounds slow. Not exactly intentional, rather a side effect of something that made sense by itself.


This is fine for podcasts, lectures and books where the purpose is to take in information. Timing in movies and TV shows is important; silences and slow pans are placed there for a reason. I can't imagine why one would watch a movie or TV show at 1.5x when the purpose isn't to take in information, but to enjoy artwork.


The downside of nootropics?


To my knowledge there isn't a way to bypass its DRM atm (to directly download a file).


It's certainly possible, look for the various groups releasing untouched Netflix video on your favourite filesharing platform.

They're not about to make their method public though, and last I read up on it I don't think they can do 4k? But SD/720p/1080p is all there.


I believe 4k uses hdcp to your monitor, but the other formats are as simple as screen capturing.


SD/720p/1080p netflix can be de-drm'd losslessly, there's no capture/re-encode step needed (...if you know the technique, I don't but assume it involves the download to watch offline feature).


+1. The lack of 2x playback speed makes documentaries on Netflix unwatchable. Third-party Chrome extensions of unknown origin doesn't seem like a good idea.

Unfortunately, easier to stick to YouTube and VLC for documentary watching.


Get Greasemonkey and you can limit the video speed controller to certain websites or turn it off when you're not using it. (And also disable auto update of the script).


You mean apart from the obvious?


Obvious what?


p2p+native video player.


Sorry I still don't get it. Do you mean torrents when you say p2p?


Meh; nodejs that uses youtube-dl frontend? why can't I just use youtube-dl?


For instance if you run this on your desktop you could bookmark "http://local-desktop-ip:3000" on your mobile device and be able to download videos while on mobile.


I made the same thing for command line: https://github.com/kissgyorgy/interactive-youtube-dl


built a simple MacOS Safari extension that sends the current url to my server, which then runs a simple script to download using youtube-dl. I then use my iPad to view the videos using the nPlayer which can read NFS shares. very useful for downloading and viewing videos from variety of "alternate" video sites


Does this download to the client or server? The latter would be useful on a NAS / Plex box.


Is downloading youtube videos legal?


Come again? Are you implying that we do not have a Fair Use in storing a local recording of a video broadcast for personal use? https://en.wikipedia.org/wiki/Sony_Corp._of_America_v._Unive....


It's not that clear-cut outside of the US.

In the UK, it's only legal if it's for 'time-shifting' with the expectation the recording is only watched once and then deleted:

https://www.gov.uk/guidance/exceptions-to-copyright#time-shi...


Thank you for sharing. I knew the UK was a bit backward, but not to that extent.


It's not legal in the UK period, as you can only timeshift a broadcast, and anything shown on-demand like YouTube isn't a broadcast.


The UK is screwed.


How could it not be? You "download" the stream every time you watch it. It's to everyone's benefit to cache it if you want to watch it more than once. That way you don't use that much bandwidth. You might be served less ads, but you shouldn't be watching them anyway.


The law can make a distinction between bits you downloaded to cache the video as you streamed it, and bits you downloaded to keep a copy offline. Of course, the result is similar- you have a copy of it on your hard drive somewhere- but the actions you took are different, your intentions are different, and the law can look at those things if it wants (which it does, all the time).


Probably only on creative commons-licensed videos (there are some). Otherwise, it might be an unlawful replication.

And yes, there will be arguments about how you're just saving to a file what otherwise comes in via the player. I honestly don't know the definitive answer. :)


Despite all the comments pretending this is all super ok scrapers are obviously not aboveboard. Sites may try to place limits on you or your account but it is unlikely you will go to court over a TOS violation unless you're really abusive.


How would you watch them without downloading?


Streamed bits might have different Colour than bits downloaded through youtube-dl, even if they're the same bits. It depends on your legal jurisdiction.

http://ansuz.sooke.bc.ca/entry/23


-deleted link- may be a DMCA violation




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: