
Very cool, but very creepy, open source project - monochromatic
http://www.notcot.org/post/43233/
======
keane
Made with:

1\. OpenCV 2.0 - C++, C, Python interfaces; BSD license -
<http://opencv.willowgarage.com/wiki/>

2\. FaceTracker - C/C++ API; "research purposes only", to download email
Jason.saragih@csiro.au -
<http://web.mac.com/jsaragih/FaceTracker/FaceTracker.html>

3\. Method Art Image Clone - realtime cloning library (from gts, glew, glib);
MIT license - [http://methodart.blogspot.com/2011/07/fast-image-cloning-
lib...](http://methodart.blogspot.com/2011/07/fast-image-cloning-library-ive-
written.html)

4\. openFrameworks - C++ toolkit; MIT license -
<https://github.com/openframeworks/openFrameworks>

5\. FaceOSC (ofxFaceTracker) - active appearance model addon for
openFrameworks; "open source" -
<https://github.com/kylemcdonald/ofxFaceTracker>

~~~
kragen
So it might be open-source, but depends on the non-open-source (and not even
publicly available) project FaceTracker?

~~~
keane
Kyle McDonald discusses this:

"I imagine FaceOSC being used to prototype ideas surrounding face-based
interaction. I created it because Jason Saragih, the researcher behind
FaceTracker, uses an open source non-commercial license for his code. He asks
that anyone who wants to use the code email him directly, as a way to keep
track of the usage. This is great, but I know that one of the fastest ways to
get cool stuff happening is to make new tools and research accessible to a
wide audience. So I asked him if it would be ok to make a standalone app for
people to prototype their ideas — even if they don't have access to the code.
Everyone already 'speaks' OSC so I thought this would be the easiest way to
get the technology out there. Eventually, if people need to integrate it into
a single application, they can contact Jason directly and use my
ofxFaceTracker addon to get started:
<https://github.com/kylemcdonald/ofxFaceTracker>

And if they need to go the commercial route, there’s FaceAPI
<http://www.seeingmachines.com/product/faceapi/> "

Source: [http://electronica.fm/news/110714/music-your-face-artist-
kyl...](http://electronica.fm/news/110714/music-your-face-artist-kyle-
mcdonald-talks-face-tracking-music-making-kinect)

~~~
kragen
"Open source" and "non-commercial license" are mutually exclusive. See the
Open Source Definition, which is the official definition of the term "open
source" by the people who invented it, <http://www.opensource.org/docs/osd>,
point 6, "No Discrimination Against Fields of Endeavor," whose rationale is
explained on <http://www.opensource.org/osd.html>: "The major intention of
this clause is to prohibit license traps that prevent open source from being
used commercially. We want commercial users to join our community, not feel
excluded from it."

~~~
cookiecaper
I really hate the hijacking of the term "open source" by the OSI. I think open
source is a generic term that should mean what it says -- the source is open,
i.e., accessible and modifiable. I don't think that the distribution
limitations should go into it.

While I certainly appreciate all the open-source projects that allow
commercial usage, I _really_ think that a trend of projects using non-
commercial or licensee-only licenses would do a lot of good, because as it
stands I'd rather have the source code for every program I use even if I'm not
allowed to share that code with someone who hasn't paid the original author.

I recognize that copyleft and all that is important and good, but I think we'd
see much more source availability if people were less rabid about their
demands for unlimited distribution. Unlimited distribution is rad but it
greatly reduces the effective profitability of the product, and there's some
software that just doesn't get made if it doesn't have some cash behind it.

~~~
kragen
> I really hate the hijacking of the term "open source" by the OSI.

The OSI was founded a couple of months after the invention of the term "open-
source software", by the same people who invented the term. They aren't the
ones who are "hijacking" it. You are. It's dishonest. Please stop doing it,
and please stop attempting to rewrite history in order to justify your
dishonest attempt to hijack the term. The people who read HN aren't so
ignorant that you can fool them that way.

~~~
chc
What's the correct term, in your opinion?

~~~
ceejayoz
I've heard the term "shared source" used.

------
apu
For face replacement in video, the state-of-the-art is this upcoming SIGGRAPH
Asia 2011 work:

[http://www.eecs.harvard.edu/~dale/docs/faceReplace_sa2011.mp...](http://www.eecs.harvard.edu/~dale/docs/faceReplace_sa2011.mp4)

[http://www.eecs.harvard.edu/~dale/docs/faceReplace_sa2011.pd...](http://www.eecs.harvard.edu/~dale/docs/faceReplace_sa2011.pdf)

Having worked on this problem before, I know how tough it is to escape the
uncanny valley when doing replacement, and these guys have really done
impressively well at it (albeit with a fair amount of manual preprocessing and
in controlled situations).

------
ericgearhart
I think the "creepy" factor of the images is probably due to the "uncanny
valley"... Pixar fought this effect when they were first rendering humans

"The uncanny valley is a hypothesis in the field of robotics and 3D computer
animation, which holds that when human replicas look and act almost, but not
perfectly, like actual human beings, it causes a response of revulsion among
human observers. The "valley" in question is a dip in a proposed graph of the
positivity of human reaction as a function of a robot's human likeness."

<http://en.wikipedia.org/wiki/Uncanny_valley>

~~~
pavel_lishin
Once they get over that barrier, though, it'll become much creepier for an
entirely different reason.

~~~
dolphenstein
The net will be awash with G.I.R.L. 2.0 !

------
pavel_lishin
How long before this becomes good enough to fool people on Skype? And how long
before someone ends up writing software to detect this?

I think this is the first time in my life that I've felt like I was living in
a scifi novel.

~~~
Aloisius
Real-time virtual puppeting has been done in movies/television and research
for a while now and yes, it can easily fool people.

A professor friend of mine Jeremy Bailenson at Stanford actually uses the
Kinect to track facial movements and uses 3D models of others to create
puppets in real-time. Even more interesting, he can morph your face with the
person you're video conferencing with to create a feeling of commonality in
them.

He actually wrote a book on it called _Infinite Reality_ [1] which talks about
all kinds of ways people will probably get manipulated in the future. He talks
about things like mirroring movements (which he can do automatically in a
video conference), looking into the eyes of every participant in a group video
conference and other really interesting psychological hacks.

[1] [http://www.amazon.com/Infinite-Reality-Avatars-Eternal-
Revol...](http://www.amazon.com/Infinite-Reality-Avatars-Eternal-
Revolution/dp/0061809500/)

~~~
rglullis
It is amazing how DFW nails that in Infinite Jest. In the book, people have
the technology to use video-phones, but they stopped using it after realizing
that video-phones eliminate the convenience of being able to communicate
without actually partaking in all other kinds of communication. A silly
example: with a video phone, you _must_ be paying attention at the person in
the camera, can't talk to someone while clipping your nails.

Also in the book, people started using more technology to cover for these
problems, from software that would make you look "presentable" to a complete
avatar that would simulate all the motions that one _should_ do while on
camera.

In the end, people just realize that no one is actually using the video part
of the phone anymore, so they just stop using it and go back to audio-only.

So, as much as I think we will have the technology to fool someone as you
mention, I don't think it will happen simply because no one will adopt that
technology in the first place.

Kottke also comments on it: [http://kottke.org/10/06/david-foster-wallace-on-
iphone-4s-fa...](http://kottke.org/10/06/david-foster-wallace-on-
iphone-4s-facetime)

~~~
mahyarm
As a user of video phones/laptops, what ends up happening is that you clip
your nails anyway and keep on talking. Just like if a close friend was doing
that while you were in the same room. It quickly becomes not a big deal.

~~~
rglullis
I don't mean to pick on you, but what is so exciting about having a cam on to
see (or not see?) your friends doing random things?

I see the value in having the occasional video conference, especially when
talking with family or friends that are not geographically close. But I still
see it as an attention cost. I would be slightly annoyed if someone that I'm
video-chatting with decided to treat it as "not a big deal".

~~~
mahyarm
It's exactly a distance thing. Close friends and family that want to see you.
It's like going over to a friends house and hanging out. You don't have to pay
attention all the time, but it's nice to see facial expressions and such. You
don't clip your nails for the entire conversation either, and you can stop and
go within the conversation. Eventually video calls can take as much attention
as an audio only call, and you can look at the person when you want to.

~~~
rglullis
Funny. Just yesterday I had to use my girlfriend's laptop and (not being a mac
user) I didn't know that Skype on Mac OS X has a "start video call" by
default.

I had a skype call with a business associate, and it was quite bad when I
realized that I was talking to him without wearing a shirt, and that the
camera started by itself. Now I can only hope that I was fast enough to cancel
the video before it actually started streaming.

Perhaps you are right in the sense that people will get used to the idea that
"always-on" video is normal. However, I will not. I am as far as possible from
being technophobic, it's just that I _like_ the barrier of not being on
display.

------
apitaru
Kyle just posted a new demo video - he's playing around with the idea of the
"Scrambler Suite" from A Scanner Darkly - <http://vimeo.com/29391633>

~~~
bh42222
Very cool. Much better than the effect used in the movie.

------
Geee
Really awesome. That second video shows much better results. Now, still some
fine-tuning and someone could create application where people can try out
different make-ups or eyeglasses.

~~~
rohit89
Hair styling is another big area where this would be really useful.

------
chaostheory
Here's where his broken link meant to go:
[http://methodart.blogspot.com/2011/07/fast-image-cloning-
lib...](http://methodart.blogspot.com/2011/07/fast-image-cloning-library-ive-
written.html)

------
mcantor
Wow, how uncannily timely. Just yesterday, I read the part of a speculative
fiction near-future novel, "Halting States" by Charles Stross, which includes
this as an interesting detail with an eye towards technical imperfection; one
of the protagonists is bemused by someone's neck glitching up into their face
during a video call while they use this sort of software.

------
andrewpi
Reminds me somewhat of the scramble suit from A Scanner Darkly!

------
jewel
I imagine this will do to movies what autotune did to music. In other words,
you no longer will have to find someone who is both good at acting and
attractive.

~~~
ippisl
It could also let movie producers get rid of the high salaries paid for
popular movie stars.

~~~
5hoom
That's an interesting point, and one that I could easily imagine happening.

I suppose the only problem for the movie producer is who holds copyright on a
persons face?

Imagine George Lucas & co. wants to make 'Indianna Jones 5 - the prequel' with
this technology using a likeness of a young Harrison Ford.

The studio owns the likeness of the character 'Indianna Jones', but surely
Harrison Ford owns the likeness to his own face (IANAL, but curious if there's
precedent for this sort of thing).

~~~
ippisl
I think the more common use case would be getting face copyright from someone
beautifull , and building it to stardom.

I think if a major studio would want to build stars that way , it won't take
them long.max a few years. so there won't be a long transition period anyways.

------
tnc
Here's the paper: <http://www.cs.huji.ac.il/~danix/mvclone/>

------
cypherpunks01
Cool! The title should be edited to describe the link though.

And as for being an open source project, link/source please?

~~~
wanorris
Here's at least some of the author's code. Not sure if it's the whole thing.

<https://github.com/kylemcdonald/ofxFaceTracker>

Edit: that code doesn't actually cover the substitution. Here's more info:
<http://vimeo.com/29279198>

------
AdamTReineke
High-res source 3D face scans + a Kinect to track target head position and
rotation better would hide the modifications quite well. Awesome project.

------
protagonist_h
This could be used in video call centers in the future. Image you make a video
call to your bank, and a blonde girl appears on the screen. In reality,
however, you are talking to a dude in India. However, this would also require
"voice substitution."

------
swah
And once more, while we were discussing about a new language to replace
javascript, some folk wrote a kickass software program in C++. :)

------
rhizome
Why aren't the videos embedded simply from Vimeo? Why do I have to wait on
"uploads.notempire.com"? Not to mention that it's not behaving very
"empire"-like.

------
croddin
If it is open source, where is the code? I am only seeing links to libraries
it uses.

~~~
mdda
The thing being claimed as Open Source is OpenFrameworks
(<https://github.com/openframeworks/openFrameworks/>), which is MIT licensed.

However the source for FaceTracker looks like it's "please email us" - so the
licensing for that isn't clear. The end result could be (legitimately)
proprietary, as long as they supply the MIT license notice.

------
cfontes
Really cool stuff. would be nice to have this feature used together with
augmented reality games.

We could then use your preferred char outfit and face while playing a wii like
game, so the game would present a video of you as the main char with any
outfit, like playing streetfighter being Mario :D

Really ingenious idea.

------
Aqwis
How do professional movie productions do this? For example the Winklevoss
brothers in The Social Network both had the face of one of the actors.

~~~
keane
Armie Hammer's face was digitally transplanted on top of Josh Pence's body in
shots where both brothers appeared in the same shot during post. Otherwise,
they would change angles and re-film a scene with Hammer as both.

For a video that shows how they did this, see
[http://videos.nymag.com/video/Vulture-Exclusive-The-
Winklevi...](http://videos.nymag.com/video/Vulture-Exclusive-The-Winklevii)

------
zerostar07
This will be perfect for plastic surgeon and hairdressing applications. Also
to try out a new smirk before actually growing it.

------
swah
This would be much better than face pixelation on Google street view.

------
atomicdog
Online dating just got a whole lot riskier...

------
chintan
Brilliant!!

------
dadads
I approve of this!

