Hacker News new | past | comments | ask | show | jobs | submit login
Live Avatars with face-api.js (pragli.com)
128 points by dsaffy 50 days ago | hide | past | web | favorite | 41 comments

I don't understand how this is desirable at all. It seems distracting and kinda creepy. But I'm also creeped out by those talking faces in the iPhone commercials, so maybe I'm just way out of the loop on this matter due to being on the autism spectrum. I just don't want to know the faces my coworkers are making when I'm not talking to them, and I don't want to think about how they may be perceiving my face when I'm not talking to them.

I actually thought this might be a cool solution exactly because of people feeling that way - if you think streaming, for example, or video calls, where you'd rather not like to be streaming your actual face, technology like this might make it easier to convey a more natural dialogue whilst retaining more privacy

That's an interesting point, because my criticism really is only based on using this for "always on" status detection. I do think it could be an improvement to streaming conversations, especially if you wanted to save bandwidth or have privacy like you mentioned. Maybe even for something like for online school classes?

I read a while back about how for kids enrolled in online classes, teachers had a hard time telling who was being attentive during lectures without streaming video from every single student. This could be a good lower-bandwidth way to ensure kids are actually present, while also acting as a nice anti-bullying measure since they can't be judged on the quality of their real clothes or housing if it's not a real video.

Some steamers currently use tools like FaceRig, which generates a avatar face that mirrors the expressions captured by your webcam.

I've actually had a couple people say that they used it that way - only on calls, and then switch back to keyboard/mouse detection after call is over.

The more interesting use to me would be to generate the look of the avatar based on the person’s face. Distinct avatars provide useful visual cues in software like Slack, Pivotal Tracker etc. but many people just don’t like taking photos. Maybe the author of this post can provide insight: Does this implementation generate the avatar’s visual configuration or is this only concerned with dynamic facial expressions?

The facial recognition just detects position and emotion, not other visual information (eg hair style). You plug that in yourself in an avatar editor. Could be interesting to play with!

Snapchat proves you wrong. ;)

People are playful and I can imagine that a remote settings is akin to some sort of playful roleplaying setting. So you build your avatar around your remote workstyle - can be funny in a setting where every one knows his peers.

This is similar to workplace settings: casual/avatar if you are not facing customers, formal/no-avatar if you face customers.

Could be cool to see, what happens in the future. Even iPhone supports those live emoji face masks.

Maybe it's a generational thing. Nobody I know even uses snapchat any more. I had it for a while, but it felt like a fad. Slack and Discord (which are basically just nicer IRC) are the primary methods of communication for both professional and casual conversations in my social sphere.

I've worked remote for years, and never wanted anything more than a basic "do not disturb" status. The creator says they're doing this to address loneliness, but some of us don't have a problem with addressing loneliness via text (or we get enough socializing in outside of work to be content). For people like me, this type of tool just creates extra mental overhead and distractions. I get that some people are into this because it's neat tech, but I'm really not convinced that it's a real value-add rather than a fad.

Thanks for the comment. I'm not sure it's for everyone. I also made an option for the avatar without the facial recognition, so it just grays out if your keyboard/mouse doesn't fire for 15s.

The tech is actually quite cool. Like I could see it being used to make a game for kids that they "play" by trying to match expressions or something. I just personally would not feel comfortable working at a job where usage of this was expected, even if I could manually control the image.


Why to what exactly?

Right. This reminds me of the series of video chat developments in Infinite Jest (description here: https://www.theverge.com/2019/5/5/18518355/infinite-jest-vid...).

Masks have started to arrive! Only a couple more decades until we give up on video entirely.

Side note: this reminds me of a passage in The Machine Stops, by E M Forster (1909).

> “In the air-ship —” He broke off, and she fancied that he looked sad. She could not be sure, for the Machine did not transmit nuances of expression. It only gave a general idea of people — an idea that was good enough for all practical purposes, Vashti thought. The imponderable bloom, declared by a discredited philosophy to be the actual essence of intercourse, was rightly ignored by the Machine, just as the imponderable bloom of the grape was ignored by the manufacturers of artificial fruit. Something “good enough” had long since been accepted by our race.

Love this! Are you exploring more variations in the avatar expressions or does keeping it more limited work better?

Also, are there any unexpected positive or negative outcomes so far? For example, do people feel pressured to smile or anything else like that?


I'm still exploring for sure - but there are benefits to some of the limits. I'd love a more realtime mode for when people are actually in a conversation in our product, but haven't gotten that working perfectly yet.

We initially felt pressure to smile before I made the default a pseudo-smile (check out the three faces in the blog post). Now the "neutral" face looks somewhat happy so no more complaints.

I rather love this idea. It is a rather elegant solution to the problem of "how do we provide office like presence". I work for a not-heavily distributed company but we have some permanent remote and some WFH days. We never turn on the video for our zoom meetings. However most of these people we see regularly, so not a huge deal.

On the flip side my wife works for a heavily distributed company and their solution is "you must always have video on for zoom". It's not a bad idea, but sometimes you can't or don't want to have video on - especially when you're call is at 7am (or earlier) to support people in timezones farther ahead.

It would be awesome to have a little dashboard of my teammates with decent integrations (like shown) to know when they are actually around and have some feeling of teamdom... without having my video or audio on all the time. Having video on for long duration is certainly a lot creepier than working in an office.

Very cool! I love the idea of reactive avatars. In general, I find it annoying to have to upload a profile photo, so being able to select an avatar, and then have it be responsive, is great.


This is pretty neat. I think it would be great as a chat heads up display feature for video games.. your avatar could be a creature or a character from the game or whatever you wanted.

That's a cool idea!

I think this is super neat!

And I could really see a use case for when bandwidth is a problem for true video calls... next to full video, streaming the avatar info would be next to nothing - so it could turn "video" calls into basically just the audio as far as the internet connection is concerned.


Thanks! I'm working on making it more real-time... would love to satisfy that use case even better.

I don't understand the reason for the facial expressions. Isn't everyone just going to be neutral 99% of the time when they are just working by themselves on the computer?

They are a lot of the time in practice. For us, the movement with a little facial recognition change is just a piece of the puzzle. We also added (optional) integrations like Spotify and Slack and are working on more to give other signals.

Overall goal is to help with loneliness of remote work and make people feel more comfortable just starting audio / video conversations with one another.

The expression sample seems a bit dystopic. The "normal" face has a huge smile, whereas the "sad" looks more neutral. May be a cultural thing, but it's not very common to see people with the "normal" looking expression outside genuinely happy situations or where one fakes such situation for sustinence or monetary gain.

Yeah... I mostly just went off feel. Since most of the time people working are "neutral", and the "neutral" looks pretty serious / not super happy, it just made a bad vibe for the team for everyone to be looking mopey. So I switched it to be a bit happier.

The half-empty view is that it's "fake happiness" but the half-full view is that it better conveys your intent to your team, that you are (hopefully) happy to be hanging out and getting work done with them. A wall of smiles also makes people feel more welcome and included than a wall of meh.

Still room to play with this though.

For me, "smile showing teeth" almost always shows up as either fake or aggressive.

Maybe I should just make the mapping customizable, so that if one wants the "neutral" emotion to translate to "serious", they can do that.


From your images it looks like there's a Spotify integration (logo in avatar's top right corner). How's that work? Does clicking it send you to the user's music stream, or can you join the stream, send you to their profile? Anyway, I like the idea - brings a personal touch to the user's virtual office presence.

Thanks! Right now, it's just a hover that shows what they're playing. But would be cool to sync you up with them on click.

This is awesome! Love the idea and definitely solves for a need.


And yet another step towards an internet that is actively hostile to those that don't want to be permanently represented by their real life stupid ugly face.


But that's the point, it isn't your real life face, only the avatar you chose, so you don't have to show your face if you don't want to.

Have you thought of using this for online classrooms/e-learning? Could be a neat use case.

facerig + live2d webgl should be able to do this already, in higher quality to boot.

Very cool, really digging it!

Thanks! Appreciate you giving it a look.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact