Hi this is Tim. I made this website. It's currently running on a $5 droplet on DigitalOcean so conversions will take a while. I'm testing out the waters here on HN to see if there's anyone interested in it. Please let me know if you have any questions regarding it.
Why are you transcoding all the time? You could probably save your little droplet a bit of heartache by using ffprobe to check if the video is already h264 (or h265)+aac and just do -c copy in that case.
Did you look into using something like ffmpeg.js to do the heavy lifting client-side? Note I have no idea if ffmpeg.js supports all the features you need (ie HLS).
Fantastic question. I'm not even doing that yet. How embarrassing!
Right now I'm just spawning sub processes like crazy and hope they don't overwhelm my machine. In the future, I plan to use a queue and also to monetize on API queries.
I haven't looked into queues in Javascript yet. If you have recommendations, that be appreciated :)
This is really just an MVP to see how the internet crowd reacts to it. I'm glad it does in a positive way :)
When I run the demo, the video plays as expected. However, I wanted to check the network tab to see what it's doing under the hood.
Based on the description, I expected to see a bunch of HTTP GET requests to ipfs.io, fetching the different segments of the movie. However, instead, I see a varying number of websocket connections to ipfs.team, soundclart.com and libp2p.io.
Could someone explain what is going on here? Is this just another IPFS-to-web proxy (based on websockets instead of http) or is this already part of the HLS streaming mechanism?
The ipfs.js library requests the files through WebSocket, and those various domains you see are different IPFS gateways. (for redundancy/latency I'd assume)
IPFS is a distributed p2p filesystem. The idea, I believe, is that you can put your big media files on it, to share bandwidth and (iiuc) reduce censorship risks; but it's better to do it with a stream-friendly codec, so this site will turn your file into that first.
The acronym is for Inter-Planetary File System, with the main idea being getting a file from the closest person who already has it, so you don't have to request it from a central server that is potentially very far away.
The main benefit on earth, as far as I understand, is reducing bandwidth required of a single person to share media by distributing hosting, similarly to torrents.
Is this one of the first examples of an IPFS-based API? This is very cool.
I like that you can take something that is effectively on your disk, ping some endpoint, and it will get "pulled" by the service as a peer and process it.
Next step could be to make the "ping" step as an IPFS pubsub thing.
I'd argue eternum is not quite at the same level since it just takes an IPFS object as an input (but no IPFS output). Op made a thing that takes an IPFS object as an input and produces a different IPFS object as an output.
The thing that doesn't make sense to me is that you say any file but then seem to be doing video-specific stuff to it. So did you mean to say any _video_ file?
HLS is HTTP Live Streaming, an open, pure-http approach to live streaming video with support for adaptive bitrate delivery. first popularized on iOS circa 2008 as an alternative to Flash and other proprietary streaming systems.
Does this require resources on your server? (Apologies because I don't understand what this is.) What's to stop me from uploading 10TB of video files and using all your resources?
Why HLS rather than DASH? Is there some way that HLS is a better fit for IPFS? Or is HLS easier to use, or more well supported? (Doesn't YouTube use DASH?)
I built this mainly because I was feeling the pain while implementing a website for a customer that is allowing their users to upload and stream files in the browser.
As the file upload should accept any kind of input, but the streaming only allows for certain files to be played back in the browser I researched different codecs that are available on the web for streaming and came across: https://github.com/ipfs/js-ipfs/tree/master/examples/browser...
So your customer is using IPFS on his site? How do you manage that? Do you run the IPFS node for him and put the uploaded videos on your node?
Is your SaaS fetching from IPFS, then reuploading because your customer's visitors upload using js-ipfs, then you fetch from your node, totally unrelated to the website's backend?
My customer is using ipfs-api-js on their website and it uploads to an ipfs node (in our case Infura). In the future we might run a node if the customer requires it. For now Infura is very convenient.
My Saas is fetching from IPFS, then reuploading so that the upload and conversion of the file both could potentially be triggered on the client side. Many Dapp architectures I've seen so far have no backend component.
Github: https://github.com/TimDaub/ipfs-hls-converter
Edit: I haven't chosen I license for this yet, sorry.