
Gcam, the computational photography project that powers the Google Pixel camera - PleaseHelpMe
https://blog.x.company/meet-gcam-the-x-graduate-that-gave-us-a-whole-new-point-of-view-3ee86657d6c9#.epxa2b2rq
======
valine
> So the team started to ask — what if we looked at this problem in an
> entirely new way? What if, instead of trying to solve it with better
> hardware, we could do it with smart software choices instead?

I feel like the lofty language takes something away from the article. It's not
like using software to improve image quality is anything new. The image
stacking technique they describe isn't particularly novel. I've done stuff
like that for years. It even works on 3d renderings from a path tracer. Its
cool their camera is fast and provides a nice ux when stacking images, but
that's only possible because of the cameras hardware.

~~~
lnanek2
Agreed. I think my last 5 smartphones have all had an HDR mode. Claiming it is
something new is a little strange.

Even HTC's relatively new technique of recording a couple seconds of
continuous video as long as the camera app is open so it can provide an
"instant" shutter button is years old at this point and that came out long
after HDR in smartphone apps.

~~~
muro
It's not the same as "normal" HDR:

[https://research.googleblog.com/2014/10/hdr-low-light-and-
hi...](https://research.googleblog.com/2014/10/hdr-low-light-and-high-dynamic-
range.html)

~~~
lightbyte
That article describes exactly how "normal HDR" is achieved with a DSLR.

~~~
fulafel
I don't think "normal HDR" traditionally includes this:

"HDR+ addresses [low-light blur/noise] by taking a burst of shots with short
exposure times, aligning them algorithmically, and replacing each pixel with
the average color at that position across all the shots. Averaging multiple
shots reduces noise, and using short exposures reduces blur."

~~~
ynniv
Reducing shutter speed to reduce blur isn't new
([https://www.dpreview.com/articles/1528656007/notimagestabili...](https://www.dpreview.com/articles/1528656007/notimagestabilization)),
accumulating sensor exposure is how DSLRs work, and aligning frames is how
image stabilization works. It might not be what we call "HDR", but there's
nothing new here. The combination in a phone camera sounds nice.

------
boulos
Sigh. I'm disappointed that this piece seemingly gives no credit to Andrew
Adams, Jiawen (Kevin) Chen, and all the folks that made gcam happen. There
isn't even a mention of Halide...

Edit: just to be clear, I like Marc just fine, but I don't see the point in
excluding the team.

Disclosure: I work at Google and know the folks involved.

~~~
jpm_sd
It's a desperate attempt to make X look good. There haven't been too many
successful "graduations".

~~~
Ajedi32
What rate of success would you consider acceptable? I thought the whole point
of Google X was basically "let's throw stuff at the wall to see what sticks",
so it seems pretty much expected that not a lot of products would make it out
of the lab before being killed.

------
kumarm
Surprised by Negative Tone of Comments.

I have been using PixelXL since launch and every one of my friends who looked
at photos are amazed by the camera (Many of iPhone Users). PixelXL made me
default photographer at parties (And fact that I can share Photos using Google
Photos with everyone at Party).

Pixel Camera is amazing compared to any existing SmartPhone camera.

Side Note: The only issue I have with Pixel is that its easy to break glass.

~~~
snovv_crash
I've had the same reaction, but with an HTC 10. I'm curious now what a side-
by-side between the two would look. The 10 uses a really novel lens with OIS
and HDR, whereas the Pixel has no OIS but uses fancy EIS.

I'm actually surprised how little love the 10 gets, considering the noise made
around the hardware quality of the Pixel, and the fact that HTC made both. I
guess people just like the Google brand or something.

~~~
Grazester
Seems like the HTC 10 really isn't all that great.

"The real surprise from these tests isn’t the winners, though — it’s how bad
the losers are. In some situations, the LG, HTC and Sony phones put out
pictures that loom downright unusable, something I’d expect more from a 2013
Moto G than "

[http://bgr.com/2016/12/14/iphone-7-camera-vs-google-pixel-
ga...](http://bgr.com/2016/12/14/iphone-7-camera-vs-google-pixel-
galaxy-s7-htc-10-dslr/amp/)

~~~
snovv_crash
Well that's just a reference to [http://www.techradar.com/news/how-2016s-best-
camera-phones-c...](http://www.techradar.com/news/how-2016s-best-camera-
phones-compare-to-a-dslr)

It looks like they had smudges on the camera lens. I've taken much more
challenging shots than that without the horrible blur they have. But smudge
any smartphone lens with fingerprints and it will look like that. Hell, smudge
a DSLR lens and it will look like that too.

------
ipsum2
This blog post is remarkably light on details. For the actual research paper
behind the software in the Pixel camera app:
[http://www.hdrplusdata.org/hdrplus.pdf](http://www.hdrplusdata.org/hdrplus.pdf)

------
felixfurtak
Surely many of these improvements are simply due to the Sony Exmor sensor? HDR
processing is only really possible when you have fast (and full resolution)
burst mode available on the sensor.

~~~
fulafel
Given the relatively long timeframe of the project, we don't know if that was
an idea/requirement from the gcam side.

------
mihaifm
> One direction that we’re pushing is machine learning

Is machine learning the answer to everything these days? Is it over-hyped...or
should I start worrying that I don't know anything about it (as a developer I
mean).

~~~
coziestSoup
The limits of ML (as we know it) is still not fully known, so it's worth
throwing a wide range of problems at it to see what sticks. I think that's why
you see talk of ML everywhere, because people are still feeling out the space,
seeing how far ML can go. Eventually, the dust will settle and the next big
thing will come along.

~~~
jacobush
So yes, should start worrying?

------
legohead
Had an interesting/cool moment with my Pixel at the theatre today. There was a
line to get your picture taken in front of the show's giant poster. The
attendant would take your phone and snap a pic.

When the attendant used my phone, she couldn't tell it took a picture --
because when you press the button it's _instant_. She was so used to other
phones taking a second or two before a picture was taken.

~~~
GuiA
Just tried on my iPhone 7, it's definitely instant. Perhaps that's more of a
UI feedback flaw in the Pixel then?

------
srtjstjsj
Article does not deliver on the headline (of course). It says that X wrote
software that became HDR+ , Google's version of HDR.

~~~
PleaseHelpMe
Please > Most recently, Gcam’s HDR+ technology launched as the default mode
for the critically acclaimed Google Pixel phone. DxOMark, the industry
standard for camera ratings, declared that the Pixel camera was “the best
smartphone camera ever made” in 2016. Reflecting on the evolution of the
project, Marc says, “It took five years to get it really right…and we’re
grateful that X gave our team the long-term horizons and independence to make
that happen.”

------
jakobegger
There's a lot of pressure on smartphone makers to make really good image
processing software because they are extremely restricted hardware wise --
there's just no space for a proper lens and a big chip. And it's amazing how
far they've gotten in the past few years. Phones are now better cameras than
most point-and-shoots.

I wonder if there's any chance that some of that amazing software makes it's
ways to pro cameras? It seems that camera companies can't keep up.

For example, the autofocus and whitebalance on my phone is almost always spot-
on and perfect. My 'real' camera (a Fujifilm X100T) often gets the focus
wrong, and the automatic white-balance is not as reliable. It seems like
camera makers just can't keep up on the software side.

------
ChuckMcM
[Heh, the cynic in me is amused when these things come out on the heel of a
lot of bad news about Google.]

I think computational photography is perhaps the biggest change to picture
taking since Ektachrome. Seriously it takes pictures with existing hardware
that are better pictures and does it by applying some interesting science to
the mechanisms in the pipeline. I've been very impressed with the results and
how rapidly the camera comes up with the image after taking it. I wish my
Canon SLR had this as part of its software load.

~~~
cromwellian
What's the 'lot of bad news'? I couldn't find anything on HN search recently,
except discontinuation of Google talk. Perhaps you confused Google with Uber?

~~~
ChuckMcM
Several articles covering advertisers pulling ads from Youtube[1,2,3] because
Google runs them next to objectionable content, a 4% drop in Google's share
price [4,5]

[1] [http://www.theverge.com/2017/3/22/15029214/att-verizon-
googl...](http://www.theverge.com/2017/3/22/15029214/att-verizon-google-
youtube-pull-ads-boycott-hate-speech)

[2] [https://www.theguardian.com/media/2017/mar/17/google-
pledges...](https://www.theguardian.com/media/2017/mar/17/google-pledges-more-
control-for-brands-over-ad-placement)

[3] [https://arstechnica.com/business/2017/03/big-us-companies-
pu...](https://arstechnica.com/business/2017/03/big-us-companies-pull-youtube-
ads-after-extremist-content-sparks-uncertainty/)

[4] [http://www.cnbc.com/2017/03/20/google-stock-downgraded-to-
ho...](http://www.cnbc.com/2017/03/20/google-stock-downgraded-to-hold-on-ad-
fears.html)

[5] [https://www.recode.net/2017/1/26/14402080/alphabet-google-
ea...](https://www.recode.net/2017/1/26/14402080/alphabet-google-
earnings-q4-2016)

------
Mindless2112
Marc Levoy (who the article is about) gave a lecture series [1] at Google
about digital photography.

[1]
[https://www.youtube.com/playlist?list=PL7ddpXYvFXspUN0N-gObF...](https://www.youtube.com/playlist?list=PL7ddpXYvFXspUN0N-gObF1GXoCA-
DA-7i)

------
iamleppert
I like how they have taken credit for implementing HDR along with a
smartphone-sized burst camera.

This is purely a technical implementation and is not novel in almost any way.

~~~
chairmanwow
I agree that this article was thoroughly disappointing. I went in expecting a
lot and received VERY little information.

------
avip
Lots of PR-ish tone here. Image fusion is pretty standard in video cameras,
it's part of the imaging pipeline. The IP on that dates back 15-20 years.

------
dharma1
It's a step in the right direction for mobile photography but not really
revolutionary. Exposure fusion and pixel alignment has been around for a long
time.

Would like to see more options for tonemapping (really like ACR's
highlight/shadow recovery), higher dynamic range sensors and most of all 10bit
or raw video on mobile phones.

Halide is great btw

------
pasbesoin
Ok, since maybe Google people will be looking at this:

I have a Nexus 5x. The Google camera app recently, finally, received a
revision that fixed some significant bugs, e.g. crashing on "zoom out".

It also introduced ISO offset/adjustment. However, the widget to make that
adjustment appears only briefly, at the top of the screen, immediately after
touching the screen -- to set a metering point or to zoom. This makes that
control both difficult to trigger, at the top of the screen, and difficult to
use with its small scale and position that leaves one's finger obscuring the
readout. On my 5x with a Pleson glass screen protector, the resulting value
also tends to jump a bit as my finger is removed (1). Finally, the setting
only remains for a matter of seconds, until the app clears any custom (i.e.
via touch point) metering setting (2).

I like to adjust the ISO offset setting until I get the exposure I'm after on
the screen. I wish the control was easier and always available to trigger,
easier to use, and that the value set would persist at least until the shot is
taken and preferably, or optionally, until it is manually changed back.

You finally gave me ISO offset control -- thank you! But the control is
difficult to the point where I mostly don't even try to use it.

The app update also introduced a widget along the right side that displays an
ungradated meter of (digital) zoom level. Beyond seeing the relative zoom
setting, this control doesn't really do much that I find useful. (Actually,
sometimes I find myself waiting for it to clear so that I can see my
composition better.)

That widget position on the right side would be the perfect place to
optionally display a touchable ISO offset scale. More real estate, making fine
adjustments easier. Finger not in the way of seeing both the values and the
shot, while adjusting it.

If not by default, then maybe something that could be selected in the options?

Friends really like my photos from the Nexus 5x. I'd love to have more-better
controls when composing them. Thanks.

\--

1\. My zoom level can similarly jump a bit when I lift my fingers from the
screen. I assume this is because of the screen protector, as opposed to the
underlying phone/app, and have come to live with it as the price for the
screen protection. The jump in ISO offset, maybe because of the small size of
its widget as well as its placement, is harder to control.

2\. This momentary clearing was already happening before the app update. I
like to take my time composing my shot, and this clearing already made doing
so somewhat frustrating at times.

------
Ajedi32
So it's just the same HDR+ they've been using in Nexus devices for a long time
now? They didn't do anything special with the Pixel's camera, specifically?

------
tambourine_man
Sorry to hijack the thread but I found your handle a bit disturbing.

What do you need help with?

~~~
ars
He created his account 231 days ago. Whatever he needed help with presumably
is no longer an issue.

~~~
chairmanwow
It never hurts to ask. Maybe he's been waiting for this day for 231 days?

------
ktta
The surprising part about this is that X decided to go with Medium as their
publishing platform.

~~~
reitanqild
Let me continue: I specifically dislike how medium breaks urls.

~~~
will_pseudonym
I'm surprised Google wouldn't also use their own URL-breaking property, AMP.
:)

~~~
reitanqild
Or their other url-breaking platform - blogger - who found out it was a good
idea to redirect to a different top level domain based on visitor location :-/

------
rmcfeeley
WHO WROTE THIS?

\--

CATHOLIC

~~~
dang
We've banned this account for breaking the HN guidelines. If you decide you
don't want to be banned, you're welcome to email hn@ycombinator.com and
promise to follow the rules in the future.

