
Show HN: Entire concerts algorithmically "reconstructed" from YouTube videos - brettwelch
http://switchcam.com/
======
angusgr
Wow. Many thoughts arose:

\- Is this more, less, or equivalently legal (copyright-wise) to just posting
the youtube videos on youtube in the first place? It seems like the act of
compositing them in this way would be significant, but maybe not?

\- The normalisation of audio could use some work. SwitchCam seems to do
something (the youtube volume on each clip was different for me), but it
didn't quite work.

\- I'd be really interested to know how automated this is, and how much human
curation is required to get it right.

\- Presumably, the next step is stitching the multiple videos together to make
3d models, allowing you to pan to places between the various camera operators.
:)

~~~
brettwelch
Thanks!

\- I'm no lawyer, but the advice we've received is that we're good on the
legal side.

\- You're on the money re: audio. It's miles better than it was 1 month ago,
but we're still not 100% yet.

\- There is a human curation step (~5% of the process), but that is mostly for
removing really crappy videos. Searching/Synching/Sequencing is automatic.

\- I dunno about _next_ step(!), but i agree the idea is super interesting :)

~~~
seats
I wonder how possible it would be both technically and legally to use the
audio from the various angles as inputs into an audio processing step where
you distill just the true music audio and clean it up. Ideally you'd have one
'cleaned' version of the audio playing and switching views just switches
video.

This may be more legally grey than just piping through audio since you are
actually producing a derivative work (even if it is mechanically produced),
but the net effect would be awesome. You'd end up with better audio than any
one person could record, and the more angles you get the higher quality you
can make the audio.

Definitely a great idea and very well done for the first cut. You just need
some Wilco on there.

~~~
javanix
It would probably be technically "possible" but I doubt that you'd end up with
anything that sounds listenable.

The only bootleg audio that usually sounds decent are soundboard recordings,
and when you find audience tapes spliced in with them the difference is
immediately noticable.

~~~
kokey
I think theoretically you can make it quite good, but it's not going to be a
simple task and you probably can't rely on purely the fans' recording. I think
the best way would be to modulate them together based on a weighting depending
on position or distance away from where the music is coming from, also
discarding distorted recordings etc. That said, you probably only want this
lightly mixed in along with a good recording from the desk, all adjusted for
the viewing angle.

------
OoTheNigerian
Really cool stuff!

A little suggestion.

The first video I picked had only one camera angle for the first two tracks of
the concert. So it was an effort for me to understand how it works. I would
suggest you have a prominent link to a 'model video' that has multiple camera
angles so new visitors can easily see it in action.

Very good job lads!

Of course, I presume the bigger play is for switchcam to be the default app
that people use to record at concerts and other events. Smart!

------
baddox
Cool idea, but I was sad that it wasn't finally an implementation of an idea
I've had for a long time. My idea is to actually use video and audio
information from distinct sources to create a single video/audio stream that
is of better quality and/or completeness than any of the constituent parts.
Essentially, my idea would do to video what Photosynth does for photos.

<http://photosynth.net/>

~~~
jeswin
That is just so much harder; though I haven't looked at the website yet, seems
to be overwhelmed right now. Besides, I would want one consistent audio signal
instead of one that varies in noise, volume, whatever. Video from different
sources is alright though, since we are used to switching scenes and cameras
all the time.

~~~
baddox
The idea I'm talking about should provide a single consistent audio signal. I
know nothing about audio processing, but it seems like it should be possible
to take multiple bad audio signals and combine them into one signal that's
better than any constituent audio source. Perhaps one audio source captured
low frequencies well, while another captured higher frequencies better.

~~~
JonnieCache
_> take multiple bad audio signals and combine them into one signal that's
better _

The problem eventually comes down to the fact that "better" is subjective.
We're in the murky realm of art here. Should your algorithm keep that fret
noise or the squeaking of a vocalist's intake of breath? Are they "noise," or
are they part of the performance?

 _> I know nothing about audio processing_

Not wishing to be rude, but this much is very evident. Recording engineers
position their microphones with millimetre precision in order to combat phase
issues, and that is in an ideal studio scenario. Doing what you suggest is
basically impossible.

Maybe I'm overstating it, you could probably do _something_ and it'd be a nice
bit of research, but you wouldn't get useful results in the way that you're
imagining.

~~~
joshu
Presumably one would get rid of per-device degradation and compression
artifacts.

~~~
stuaxo
This is the main thing (also tricky)

------
evan_
Reminds me of "Awesome, I Fuckin' Shot that", a Beastie Boys concert filmed by
50 audience volunteers and edited together:

<http://en.wikipedia.org/wiki/Awesome;_I_Fuckin_Shot_That>!

~~~
saurik
<http://en.wikipedia.org/wiki/Awesome;_I_Fuckin%27_Shot_That>! <\- working URL

------
mambodog
Very cool, just a slight niggle about the 'lightroom' colour scheme (dark area
surrounding the video) which I see being poorly implemented time and time
again: it doesn't really work when you have elements of bright white on the
page, because they just end up looking even brighter next to the dark parts.
The glaring white strips are distracting while watching the video, moreso than
even an all white background.

TL;DR: lose the strips of white from the video viewing page.

~~~
brettwelch
Thanks for the feedback, I agree actually. We've been talking about changing
the color scheme to either be all dark or all white but haven't pulled the
trigger.

------
wittjeff
I had been thinking about something like this, but in the context of
recreating riot crime scenes. After the last riot in Toronto the police
received over 1M stills and thousands of hours of relatively low quality cell
phone video. The main task for them is to connect different shots of
individual offenders over time to build a) a coherent story demonstrating
premeditation, b) an unbeatable description and connect it to some identifying
info that might not necessarily be present at the exact time of the offense.

------
ansonparker
Site seems to be struggling under load.

I am excited about the concept - was thinking about this very idea while
watching Coachella streaming live this year and then seeing all the cams
people were uploading.

~~~
nprincigalli
Google cache:

[http://webcache.googleusercontent.com/search?q=cache:http://...](http://webcache.googleusercontent.com/search?q=cache:http://switchcam.com/&hl=en&safe=off&sa=G&strip=1)

~~~
EwanG
Even using that, all I get are Internal Service Errors...

------
andrewfelix
Sweet execution. Finally a use for all those dickheads who hold up their
phones in the middle of a _live_ performance.

~~~
exDM69
Yeah, now that you can't see anything but camera phones while you're actually
at the concert, you can watch it from youtube when you get home.

Thankfully this isn't too much of a problem at heavy metal gigs. No-one wants
to take their smartphone to the moshpit :D

~~~
hyperbovine
The solution must be to institute moshing at all live shows, then. Lilith
Fair, here I come.

------
cleverjake
Honestly one of the coolest things i've seen in a while. If they can get
around the legality of it, im sure they'll be snatched up quickly.

------
jamesgagan
Pretty cool. Looks like they are a 500 startups project, formerly Veokami.
Guess they used some of their funding to buy a better name!
[http://www.startupsmart.com.au/growth/veokami-
among-500-star...](http://www.startupsmart.com.au/growth/veokami-
among-500-startups-latest-intake/201110174213.html)

~~~
brettwelch
Yep, that's us. I miss Veokami sometimes, but my cofounder and I were two of
about 4 people who could actually pronounce the name :p

Incidentally, Switchcam.com was only a little more than $1k, which was
definitely worth it.

------
chartley
Sorry guys, probably should have sured up servers for HN load - my bad =\ On
Caltrain, brb...

------
jjcm
This is awesome. My only suggestion is to allow the user to switch angles
without changing the audio track. Since these are concerts, the audio track is
going to be one of the most important things. Often times one video will have
significantly better audio than the other ones. It'd be nice to be able to
choose that one to stay constant. Alternately, it'd be awesome if there were a
way for the bands themselves to upload the recorded concert audio.

I realize that often times getting the recorded audio for a concert can be a
finicky matter, but maybe you could partner with one of the many music
festivals that are cropping up to make it happen.

------
hospadam
This. Is. So. Cool. Wow. I tried several concerts, several angles, and
everything works pretty smoothly. Any critiques I might have are quite minor.

------
ianbishop
This is really cool.

I went to add a concert with time/date but landed up getting lot of videos
that didn't meet the criteria. The concert was part of Osheaga (a festival in
Montreal) but the listing shows up as Parc Jean-Drapeau, where the festival is
held. I think including the ability to indicate what event it was part of
might narrow down your search a bit.

Since most of the 300 videos found were unrelated, I decided to go through
manually and select only those which were relevant. By default, all 300 videos
are checked with no easy way of unchecking them. Your average user isn't going
to be able to just use a jQuery one-liner like I did, so this is something
else you should consider!

Overall, great job and I look forward to seeing how you guys progress with
this idea!

~~~
brettwelch
Thanks Ian, for the feedback and also for adding an event. I just saw those
come through on the back end. We've put the processing on hold for the moment
until the load becomes manageable - hope that's cool. We'll email you when
they're up.

------
pavedwalden
I love it. Any chance you would expand to more general "news" applications?
The audio wouldn't be as good for syncing against, but I've always wanted an
interface like this for watching protest videos.

~~~
PStamatiou
Take a look at this: [http://blog.switchcam.com/uc-davis-pepper-spray-
incident-fro...](http://blog.switchcam.com/uc-davis-pepper-spray-incident-
from-all-angle)

------
aclements18
Very impressive. Very cool. I assume this will eventually extend beyond
concerts to all kinds of other live events (sports, plays, etc). This is going
to be big.

What's also interesting is how much the effect of SOPA will have on such an
awesome service like this. So disappointing to think about how the copyright
holders would rather shut you down then come up with a great way to enhance
the service and share revenue with you.

Best of Luck. We'll all be rooting for you guys.

------
blcArmadillo
Pretty cool. One thing... after switching camera angles from the first camera
I can switch between all other camera except I can never get back to camera 1.

------
jhealy
This perfect circle gig is a good sample:

[http://switchcam.com/event/a-perfect-circle-
lollapalooza-201...](http://switchcam.com/event/a-perfect-circle-
lollapalooza-2011/)

Some of these concerts may have bootleg audio available, often recorded by
audiophiles with decent equipment. It'd be amazing if supported syncing that
up so you'd get great audio with multi-camera video

------
phil
Seems to work. That is incredibly cool/terrifying.

------
wingerlang
Nice. However - the timeline should be default. I was pretty meh'd until I
found it, works great.

Now it would be nice with an underlying concert-track or / per song mp3. I
don't like the crappy cellphonesound. I guess the syncing will be hard though,
with the videos lag not very constant (most often, seems to work good now but
I have a nice broadband connection)

------
phil
So if you can do this with concerts, presumably you can do it with any
collection of video shot in the same place, if there's enough audio in common.

That makes me think that all kinds of crowd video (protests, speeches, etc)
should be stitchable, which seems like it would open larger (and potentially
morally ambiguous) markets.

------
mitjak
Pretty cool indeed. A question though: how much of a need is there for a
service like this? Beyond the more mainstream artists and festivals, most of
the artists whose shows I've seen have typically had more respectful audience
members who didnt put recording with their cellphones ahead of enjoying the
show.

------
jwblackwell
This is really awesome. I hope you don't have any issues with copyright
though. Best of luck with the project!

------
mrjasonroy
Absolutely amazing, great job. I just got chills from watching the National
and feeling like I was back there.

------
balbaugh
First thing I thought of was the Nine Inch Nails fan project "This One Is On
Us". It works. Very cool and I wish you all the best.

<http://www.thisoneisonus.org>

<http://www.ninwiki.com/This_One_Is_On_Us>

------
fat0wl
Does anyone believe that this is done algorithmically? All the examples I
checked seemed to be really poorly sync-ed, & as a DSP engineer I don't
believe this task to be incredibly difficult...

would also be cooler to have a seating chart to click around to make sense of
perspectives

~~~
chartley
It's all algorithmic but the flash player only seeks to the nearest key frame
(every 2s) so that's why it isn't flawless. This can be overcome ... but I'd
prefer to work on the seating chart thing - thanks!

~~~
fat0wl
Ah, didn't realize the flash / keyframe limitation. Best of luck! very cool
idea.

------
mitchellhislop
This is one of the coolest things I have seen. While there are some little
bits of feedback, I can admit that I spent the last 30 minutes watchings
concerts, amazed at how great this felt.

MPAA: More of this, less of SOPA. If you give us great ways to get content, we
will pay for it.

~~~
brettwelch
I'd love to hear your feedback, Mitchell... And thanks for the kind words.

------
aslewofmice
Absolutely one of the better ideas I've seen lately! An original idea with a
lot of wiggle room for future growth.

I think it could be pretty cool if users had the ability to reference a higher
quality audio source (when available - something on Archive.org or uploaded
mp3s).

------
inspiredworlds
Great concept and like how you can change views! Been following them since
they were veokami.

------
autrefoius
Maybe it's just me, but the site rick-rolled me on each concert I tried to
check out.

~~~
brettwelch
Sorry about that. It only happens when the JS borks out completely (which is
rare), when JS is turned off, or when you're using a really old browser (which
i assume you're not).

If you're inclined (I completely understand if not) can you tell us the error
that the console in Chrome or Firebug gives you?

~~~
autrefoius
The error I'm receiving on Firefox Aurora 10.0a2 (2011-11-27) is:

VKLog is not defined
[http://switchcam.com/assets/js/utils/facebook.js?ver=2011112...](http://switchcam.com/assets/js/utils/facebook.js?ver=20111129-223208)
Line 81

VKLog.dlog("checkSessionStatus()");

It does seem to work with Adblock disabled, so I'd guess a subscription is
blocking something essential.

~~~
chartley
Interesting, thanks for poking around! I don't think we use anything dodgy
(except maybe Facebook libraries) but I'll see if I can set up that
configuration and debug it. Really appreciate you taking the time to write
this up. Cheers, Chris

~~~
BrandonM
I'm having the same problem. I think it's a bad idea to use Facebook libraries
for critical functionality. A lot of people (myself included) block Facebook
resources from loading on external sites to decrease their ability to track
user activity.

------
PaulJoslin
Cool concept, fairly good execution.

I think I would advise not to just focus on concerts, but potentially other
mass events, e.g. sporting events.

I guess you're using the audio stream to do some form of sync / time stamp
though, so that may limit potential uses.

------
njloof
Is music a requirement? I'm trying to add Henry Rollins' spoken word at
Coachella from April 18 2009, but your YouTube search always seems to fail. I
can see the videos on YouTube just fine if I search for myself.

------
jkap
I had the idea to do this (albeit, manually) a while ago, syncing the audio
from Justice's "A Cross The Universe" with Youtube video of the concert it
came from. Really cool to see it done, though!

------
wittjeff
Watching Ocean's 11 last night it occurred to me that other contexts in which
intelligent multi-camera stitching might be profitable are casino security and
retail loss prevention.

------
catshirt
this is awesome. seriously. maybe you could package the technology in a way
that lets other platforms use this without youtube. i am reminded of the
original Color app.

------
minikomi
Maybe a next step could be a video app which had some kind of time code
embedded in the resulting video - for much easier syncing.

------
devinmrn
Great work, hope you can handle the traffic that is coming your way. I would
be surprised if you didn't get bought by Google.

------
berberous
Mind blowing. I don't know that I'd ever really use it, but this is super cool
and really well done!

------
nico
Looks a lot like Hadza <http://hadza.com/> (ex Wesync)

------
unreal37
I want. Love the idea. Site is down so I can't tell if it works. But plus 1
for the idea alone.

------
citricsquid
My out loud comment when I tried it was "Holy shit this is genius if it
works".

------
ChrisArchitect
Great stuff. This is like Photosynth for videos! Remember Photosynth?

------
lisper
Ummm... am I missing something? Every video I click on is a rickroll.

------
pirko
Cool idea but when I tried "Arcade Fire - Austin City Limits" 2011
[http://switchcam.com/event/arcade-fire-austin-city-
limits-20...](http://switchcam.com/event/arcade-fire-austin-city-limits-2011/)
it was far from correct.

------
giulivo
to me, it looks amazing

some techie questions, how does it work? how do you get to know which videos
are actually part of a specific event? how do you match the video with the
actual song?

------
smackfu
Too bad some artists apparently take down concert videos.

------
nodata
Excellent work!

It says a lot that the music industry didn't invent this.

~~~
grabble
That's because they're too busy suing people to innovate.

------
tomkinstinch
Very cool. What about other applications? News reports?

------
meatsock
great idea. do you have any solution to all of the shaky and unsteady video?

------
loboman
Amazing, thanks!!

------
gmatty
wow. super cool

------
aDemoUzer
very cool. worked swell.

------
georgieporgie
This is awesome for two reasons: 1) it's just plain cool and 2) it finally
gives a reason to tolerate annoying people who hold up their cell phone
cameras during concerts.

~~~
plytheman
It's funny because watching a concert I thought how cool it was that when
someone had a camera or phone in my line of sight I could simply switch to
another unobstructed view. Then I realized the chances were good that my new
view was coming from the very camera I switched views to avoid.

------
saltwatershane
This is freakin awesome! Keep it up! How big is your company?

