
Live Avatars with face-api.js - dsaffy
https://blog.pragli.com/live-avatars-with-faceapi-js/
======
sudosteph
I don't understand how this is desirable at all. It seems distracting and
kinda creepy. But I'm also creeped out by those talking faces in the iPhone
commercials, so maybe I'm just way out of the loop on this matter due to being
on the autism spectrum. I just don't want to know the faces my coworkers are
making when I'm not talking to them, and I don't want to think about how they
may be perceiving my face when I'm not talking to them.

~~~
krsdcbl
I actually thought this might be a cool solution exactly because of people
feeling that way - if you think streaming, for example, or video calls, where
you'd rather not like to be streaming your actual face, technology like this
might make it easier to convey a more natural dialogue whilst retaining more
privacy

~~~
sudosteph
That's an interesting point, because my criticism really is only based on
using this for "always on" status detection. I do think it could be an
improvement to streaming conversations, especially if you wanted to save
bandwidth or have privacy like you mentioned. Maybe even for something like
for online school classes?

I read a while back about how for kids enrolled in online classes, teachers
had a hard time telling who was being attentive during lectures without
streaming video from every single student. This could be a good lower-
bandwidth way to ensure kids are actually present, while also acting as a nice
anti-bullying measure since they can't be judged on the quality of their real
clothes or housing if it's not a real video.

------
ibudiallo
Side note: this reminds me of a passage in The Machine Stops, by E M Forster
(1909).

> “In the air-ship —” He broke off, and she fancied that he looked sad. She
> could not be sure, _for the Machine did not transmit nuances of expression_.
> It only gave a general idea of people — an idea that was good enough for all
> practical purposes, Vashti thought. The imponderable bloom, declared by a
> discredited philosophy to be the actual essence of intercourse, was rightly
> ignored by the Machine, just as the imponderable bloom of the grape was
> ignored by the manufacturers of artificial fruit. Something “good enough”
> had long since been accepted by our race.

------
mrcoles
Love this! Are you exploring more variations in the avatar expressions or does
keeping it more limited work better?

Also, are there any unexpected positive or negative outcomes so far? For
example, do people feel pressured to smile or anything else like that?

~~~
dsaffy
Thanks!

I'm still exploring for sure - but there are benefits to some of the limits.
I'd love a more realtime mode for when people are actually in a conversation
in our product, but haven't gotten that working perfectly yet.

We initially felt pressure to smile before I made the default a pseudo-smile
(check out the three faces in the blog post). Now the "neutral" face looks
somewhat happy so no more complaints.

------
ShakataGaNai
I rather love this idea. It is a rather elegant solution to the problem of
"how do we provide office like presence". I work for a not-heavily distributed
company but we have some permanent remote and some WFH days. We never turn on
the video for our zoom meetings. However most of these people we see
regularly, so not a huge deal.

On the flip side my wife works for a heavily distributed company and their
solution is "you must always have video on for zoom". It's not a bad idea, but
sometimes you can't or don't want to have video on - especially when you're
call is at 7am (or earlier) to support people in timezones farther ahead.

It would be awesome to have a little dashboard of my teammates with decent
integrations (like shown) to know when they are actually around and have some
feeling of teamdom... without having my video or audio on all the time. Having
video on for long duration is certainly a lot creepier than working in an
office.

------
rlargman
Very cool! I love the idea of reactive avatars. In general, I find it annoying
to have to upload a profile photo, so being able to select an avatar, and then
have it be responsive, is great.

~~~
dsaffy
Thanks!!!

------
Gracana
This is pretty neat. I think it would be great as a chat heads up display
feature for video games.. your avatar could be a creature or a character from
the game or whatever you wanted.

~~~
dsaffy
That's a cool idea!

------
chrisa
I think this is super neat!

And I could really see a use case for when bandwidth is a problem for true
video calls... next to full video, streaming the avatar info would be next to
nothing - so it could turn "video" calls into basically just the audio as far
as the internet connection is concerned.

Awesome!

~~~
dsaffy
Thanks! I'm working on making it more real-time... would love to satisfy that
use case even better.

------
tbirdz
I don't understand the reason for the facial expressions. Isn't everyone just
going to be neutral 99% of the time when they are just working by themselves
on the computer?

~~~
dsaffy
They are a lot of the time in practice. For us, the movement with a little
facial recognition change is just a piece of the puzzle. We also added
(optional) integrations like Spotify and Slack and are working on more to give
other signals.

Overall goal is to help with loneliness of remote work and make people feel
more comfortable just starting audio / video conversations with one another.

------
jampekka
The expression sample seems a bit dystopic. The "normal" face has a huge
smile, whereas the "sad" looks more neutral. May be a cultural thing, but it's
not very common to see people with the "normal" looking expression outside
genuinely happy situations or where one fakes such situation for sustinence or
monetary gain.

~~~
dsaffy
Yeah... I mostly just went off feel. Since most of the time people working are
"neutral", and the "neutral" looks pretty serious / not super happy, it just
made a bad vibe for the team for everyone to be looking mopey. So I switched
it to be a bit happier.

The half-empty view is that it's "fake happiness" but the half-full view is
that it better conveys your intent to your team, that you are (hopefully)
happy to be hanging out and getting work done with them. A wall of smiles also
makes people feel more welcome and included than a wall of meh.

Still room to play with this though.

~~~
mirimir
For me, "smile showing teeth" almost always shows up as either fake or
aggressive.

~~~
dsaffy
Maybe I should just make the mapping customizable, so that if one wants the
"neutral" emotion to translate to "serious", they can do that.

------
vwee
From your images it looks like there's a Spotify integration (logo in avatar's
top right corner). How's that work? Does clicking it send you to the user's
music stream, or can you join the stream, send you to their profile? Anyway, I
like the idea - brings a personal touch to the user's virtual office presence.

~~~
dsaffy
Thanks! Right now, it's just a hover that shows what they're playing. But
would be cool to sync you up with them on click.

------
carocaroz
This is awesome! Love the idea and definitely solves for a need.

~~~
dsaffy
Thanks!!!

------
NikkiA
And yet another step towards an internet that is actively hostile to those
that don't want to be permanently represented by their real life stupid ugly
face.

Bravo.

~~~
rlargman
But that's the point, it isn't your real life face, only the avatar you chose,
so you don't have to show your face if you don't want to.

------
bearton
Have you thought of using this for online classrooms/e-learning? Could be a
neat use case.

------
the8472
facerig + live2d webgl should be able to do this already, in higher quality to
boot.

------
bwb
Very cool, really digging it!

~~~
dsaffy
Thanks! Appreciate you giving it a look.

