If you are not on a budget, you can use a mirrorless camera with a big aperture (say f1.6), wide angle lens as a "webcam" by connecting the HDMI out of the camera to the computer using an Elgato Camlink. That will also get you a nicely blurred background.
I do this, and a lot of people do a double take whenever I enter a zoom call. Best commment so far was "What is happening? Are you in a movie?".
Yep this is a neat trick too. Also I must say (as a hobbyist photographer myself) that even if you're on a budget, if you want to go the optical route, you can easily grab a cheap used DSLR like a Canon 60D which often goes for <$200 used, and slap on an old M42 Russian lens with a conversion adapter and get fantastic bokeh for video calls.
The mildly annoying part is most DSLRs don't have an AC adapter input to allow them to be constantly on, but you can buy a "fake" Canon battery that sits in the battery compartment and wires in AC adapter input.
> The mildly annoying part is most DSLRs don't have an AC adapter input to allow them to be constantly on,
> but you can buy a "fake" Canon battery that sits in the battery compartment and wires in AC adapter input.
Yeah, I left out that detail. One should also make sure that the camera has a clean HDMI output. I can't use my old EOS 500D for this since the HDMI output is grody with focus aids and stuff.
You don't even need to use HDMI or have a capture card; you can connect the camera with USB and use gphoto2/v4l2loopback to present it to the OS as a video input. With older cameras that have USB2, you will be limited to a lower bandwidth and resolution (200Mbps -> 1024x680), but it still looks fantastic, especially with high-quality lenses and shallow depth of field that enables video codecs to encode the out-of-focus-areas efficiently and concentrate on detail in the focal plane.
I experimented with the old 550D of my dad. I got a successful clean picture using either USB (canon supplies a "USB Webcam Software", though the 550D is not officially supported it works. I also managed to get clean HDMI out using "magic lantern" (a firmware modification for some canon DSLRs which enables a lot of hidden features, allows clean HDMI, and on a beta branch allows "simulated focus bubtton press" to keep the camera from shutting down due to inactivity)
Also overheating. I don’t know about the Canon but my Sony had to have a number of settings changed to keep the temp down, and even then on a hot day it’ll shut itself off.
I don't know about Canon, but the Sony one is lower resolution than the HDMI capture. I was able to buy a cheap Chinese USB-HDMI capture card for like $8 and it works actually quite well, and is driverless.
You probably don't even need to shell out for an elgato camlink.
Most halfway modern canon cameras are supported by their webcam tool (iirc they expanded support to a lot of models during the pandemic) and I'm pretty sure that Sony has an equivalent as well.
If you're on Linux there's a way to make this work with gphoto2, but it involves a little bit of fiddling and is less plug and play.
A modern camera like a Sony A6400 (which I use) has facial focus tracking and deals with this quite well. I use a 16mm prime lens for my video calls -- 50mm would be way too close for my facial distance (about 2 feet from the lens).
I have a Sony rx100 that I do this with, which is just a point-in-shoot. It has excellent picture quality and bokeh. It looks much better than this.. the blur is the same everywhere in the background, which isn’t what it looks like from my Sony.
Are there any readily available stereo cameras besides RealSense? Turning this into a product for video conferencing would be a huge hit, been thinking of it for a while.
- Structure Core: https://structure.io/structure-core
Closest to RealSense in functionality that I've tried so far, but pricier. Software is in a private developer portal instead of a Git repo where everyone can submit issues and pull requests and comment, so it's not as easy to get help as RealSense when you run into issues.
- Mynt Eye: https://www.mynteye.com/ From my understanding the S models eat into your GPU and the D models have on-board processing, but the D models seem sold out.
I'd be very interested to know about others, especially if there are more that can do on-board processing, since I typically need my entire GPU for other tasks.
You're going to have photographers arguing that this is merely depth of field, and for it to be called bokeh you need light sources/reflections to bloom in the shape of the aperture. Which would be a super cool feature.
Yeah that would be cool! I guess that's actually not that hard to implement since I just need to do some sort of 2D convolution with the aperture shape, instead of just cv2.blur.
The tricky thing is that in order to get different amount of blur at different depths, I interpolate between precomputed 3 levels of blur, since that vectorizes well at -O3, I'm not sure how to compute a per-pixel varying blur efficiently without going to the GPU.
That's what I thought this was doing, based on the description. The reason that Meet backgrounds look fake is because they do a Gaussian Blur instead of simulating a lens. A physical bokeh will blow out highlights, whereas a Gaussian Blur will smooth out bright spots.
If you are not on a budget, you can use a mirrorless camera with a big aperture (say f1.6), wide angle lens as a "webcam" by connecting the HDMI out of the camera to the computer using an Elgato Camlink. That will also get you a nicely blurred background.
I do this, and a lot of people do a double take whenever I enter a zoom call. Best commment so far was "What is happening? Are you in a movie?".