Hacker Newsnew | comments | ask | jobs | submitlogin
Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second (techcrunch.com)
448 points by Osiris 263 days ago | comments


revelation 263 days ago | link

DCT is already lossy [1], so the statements around 8 megapixels are completely pointless, and worst of all, its 1990 lossy technology. Wavelet transformations completely destroy any DCT.

That said, if their emphasis is on producing pictures with minimal time delta at highest resolution, algorithms used for still pictures are out of place. Video compression algorithms still use DCT and wavelets, but they do so only after they have reduced redundancies between series of pictures, a process that tends to work significantly better than anything you can get out of these lossy transformations when you want to preserve quality.

Of course, eliminating redundancy in a series of pictures might have tipped them off to the fact that the image sensor isn't actually producing fresh pictures at the rate they want.

1: as used in JPEG. The transformation itself is perfectly invertible, assuming infinite precision arithmetic.

-----

jpap 263 days ago | link

You are right on the loss: it's purposefully introduced as a quantization step after performing the DCT, and before losslessly compressing the resulting coefficients with Huffman and encoding to the final JPEG bitstream.

Despite all of that, JPEG has now become computationally tractable. I remember the days where it took tens of seconds to encode a JPEG on a commodity machine. Now, with the help of SIMD, we can encode a high quality image in msec on a mobile device.

Fortunately you can choose the quantization matrix that determines the amount of loss. Even if you were to choose a unitary matrix, no human, not even superman with his laser eyes, can "detect" the quantization noise.

For SnappyCam, I chose to invest in JPEG a little more because it's a ubiquitous standard for still image compression.... and with the right hardware and algorithms, quite tractable.

I'll consider adding a JPEG "quality setting" so you can choose the amount of loss introduced... sounds like a great idea to me.

The idea behind SnappyCam was also to code each picture independently, and not rely on motion prediction or video codecs. If you try and pull a single frame from a HD video you might be disappointed: they compress the YUV dynamic range (studio swing) and it looks washed out, even if you land on an i-frame.

Lastly, as far as I can tell, the image sensor is yielding complete scans with each frame. I'd hazard a guess to say that any motion prediction or frame deltas might actually slow the whole chain down.

-----

smith7018 263 days ago | link

Man, I hope I'll know as much as you do one day... Congrats on the app! I just bought it and I'm really liking it! One suggestion; would outputting to a gif file be too difficult? I could really see people using this to create gifs of their lives that they could share to Tumblr or Facebook.

-----

jpap 263 days ago | link

Thanks! :-) To be honest, none of my formal training prepared me for what went into SnappyCam---only the "how" to go about learning to do so. I'm sure you could pick up a few tricks in the same way by working on a cool pet project or two.

With so many recent requests for AGIF, I'm absolutely going to add it to the app. (It's been on my list for a while, but lower priority than getting up a solid core product and the social sharing that is in development at the moment.)

-----

zxcdw 263 days ago | link

Out of curiosity, what kind of educational background do you have if you don't mind the question? As a highschool dropout it interests me. :)

-----

jpap 263 days ago | link

I'm Aussie, so I did my schooling in Melbourne, Australia.

I don't recommend the same path for everyone, but my ugrad at RMIT University, dual bachelors of EE and CS. I then went on to do a Ph.D at the University of Melbourne in EE. My dissertation was on mathematical optimization of wireless and wireline DSL. (Prof Jamie Evans is an awesome guy if you're on the lookout for an advisor!)

I've been in SFO for just over 5.5 years, and started SnappyLabs after winning the "greencard lottery".

SnappyCam, you might say, is the embodiment of both a very practical ugrad and somewhat applied but very theoretical pgrad.

-----

prawn 263 days ago | link

Congrats fellow Australian. Two of us here in an office in Adelaide just bought the app.

Guy behind me said "Hey, this is pretty cool. Have you seen... oh, you're looking at it already."

-----

dorian-graph 262 days ago | link

I think all us Australians are coming out right now to congratulate him, haha. /Brisbane

-----

megablast 263 days ago | link

Hi, what is Adelaide like to work in? I assume you guys are in IT? From what I hear, it is mostly government work.

-----

prawn 262 days ago | link

I run a two-person web development business and have worked out of Adelaide for myself for 15 years. Always seems to be enough work around and a decent amount of variety.

The other SnappyCam purchaser I mentioned is an iOS/Android developer who is a sub-tenant in my office. We actually have a cheap, spare desk in this room at the moment if you wanted to visit and work for a while. Email me if you want to ask any questions.

-----

nl 263 days ago | link

Adelaide is good. Plenty of gov & defence work if that is what you are after, but I've worked here nearly 15 years and never done either (ok, my current place is quasi-government, but still..)

-----

jpap 263 days ago | link

Onya mate! That's bloody awesome. :D

-----

lukego 263 days ago | link

Impressive stuff! And just in time to capture some action shots of my puppy :-) -Appreciative fellow Australian, in Switzerland :)

-----

jpap 263 days ago | link

Oi oi oi! :D

-----

wluu 263 days ago | link

high five

Another fellow Australian and Melburnian.

Well done on the app John!

-----

danpat 263 days ago | link

Hah, I told you getting your PhD was a waste of time ;-)

-----

jpap 263 days ago | link

LOL! Thanks @danpat. :-) How are you going these days? I heard you might be moving to the Big Apple?

-----

mansr 262 days ago | link

Motion estimation/prediction is used video coding because it minimises the compressed size. However, it is incredibly expensive to perform the motion search. A typical video encoder spends well over half its CPU time in this stage. After motion estimation, the residual image is encoded in the usual way, so speed-wise the motion search is a complete loss.

-----

svantana 263 days ago | link

SIMD, you say? Are you relying mainly on NEON optimizations or are you also doing encoding stuff on the GPU? Very impressive performance I must say!

-----

jpap 263 days ago | link

Thanks! :-)

I first tried using the GPU, using old school GPGPU textures and OpenGL ES 2.0 shaders, but unfortunately the performance wasn't there for a variety of reasons given in [1].

SnappyCam has since been making extensive use of ARM NEON for the JPEG codec and a bunch of other image signal processing operations, like digital zoom. It's a great instruction set!

[1] http://www.snappylabs.com/blog/snappycam/2013/07/31/iphone-k...

-----

igravious 262 days ago | link

Just curious, I know nothing about low level ARM stuff. I was wondering is this iPhone/Apple specific tech or is the work you've done portable to other mobile platforms? Congrats on what you've done. I couldn't quite work out whether you've optimized the hell out of the standard DCT algorithms or whether you've come up with new algorithms. If it's the latter would you be able to publish them or would that give away too much secret sauce? ;-)

-----

nine_k 262 days ago | link

No, NEON is an ARM-specific tech, and is widely available and used e.g. on Android smartphones.

It's like MMX / SSE of x86 world, a set of extra instructions to process many small integers in parallel in one instruction. Since image data are usually independent 3-byte pixels (or 3 planes of 1-byte subpixels, one per color channel), NEON is great for many image-processing tasks.

See e.g. https://en.wikipedia.org/wiki/ARM_architecture#Advanced_SIMD...

-----

portmantoad 262 days ago | link

One thing I've gotten very into recently is multishot techniques, where I take multiple shots in burst mode, align them, and then average them to reduce sensor noise. Similar to what http://www.photoacute.com/ does but they do more advanced superresolution stuff that your invisible noise might preclude, but a simple average or median in areas where there isn't too much motion really improves the quality quite dramatically in some cases, particularily in low lighting situations. If your frame rate is that high it probably wouldn't be hard to get a really good alignment between frames, so I thought I'd bring this up as a thought for a future feature (as in - you select the frame you like, then the program grabs a few frames immediately before and after and uses them to increase the image quality of the final output).

-----

lcrs 262 days ago | link

Re video's "studio swing" dynamic range, the YUV components do have a different encoding range to those in JPEG, but if you expand them back out to 0-255 the image is in fact the same - you lose a lil' fraction of your bit-depth but no dynamic range.

I think you definitely made the right choice though - it's interesting that the obvious delta-coding and motion compensation tricks to reduce bandwidth are rarely used for video acquisition apart from the most limited devices like phones, stills cameras and the GoPro. Everything that can afford to uses per-frame coding like ProRes, REDCODE, AVC-Intra, DNxHD, Cineform... being able to seek quickly is important!

In fact Canon's 1DC 4k camera uses (dun dun duhhhhh...) motion JPEG :)

-----

revelation 263 days ago | link

It's just bizarre that you would be doing the complete JPEG process at the instant you get the image from the sensor. As you note, there are a plethora of steps that JPEG performs, from color space conversion, to DCT transformation (essentially a gigantic matrix multiplication), Huffman coding, quantization, arithmetic coding and encoding as JPEG bitstream.

The only reason would be that you are pressed for memory or bandwidth, but certainly you have the resources to store one full frame and produce deltas, or just apply part of the JPEG chain, enough to remedy memory pressure. You can always encode it to an actual JPEG after the process.

And yes, pulling single frames from a completely encoded video isn't helpful, because they can get away with more compression. But there are very sophisticated algorithms for eliminating the redundancy between frames, which would have been my first avenue in attempting to do something like this.

-----

gruseom 263 days ago | link

which would have been my first avenue in attempting to do something like this.

Have you attempted to do something like this? Because he not only has attempted, he's done it. Therefore I think you should stop talking down to him ("completely pointless", "it's just bizarre", "my first avenue"). It comes across as wanting to prove how smart you are instead of seeking to learn from someone who has done incredible work and—lucky for us—is bursting at the seams with enthusiasm to share it.

Oh and congratulations jpap on what's looking like the most successful and technically solidest HN launch in quite some time! I hope your hard work pays off.

-----

eclipxe 263 days ago | link

Thank you for saying what I'm sure a lot of us were thinking.

-----

zimpenfish 262 days ago | link

wild applause

-----

tptacek 262 days ago | link

This whole thread should be framed and hung in the HN lobby.

-----

jpap 263 days ago | link

Oh, I actually do both---see the other thread on the topic.

I buffer as much as possible, while also encoding on any other cores that are available.

There's a reason why a "simple camera app" can total some 80 kLOC. :)

-----

philhippus 263 days ago | link

Why is this comment getting downvoted? I see that it adds insight to the subject and makes some good points, which OP even acknowledges. The written complaint is that revelation is not being deferential enough, which is bullshit on an in-depth technical discussion.

-----

kelnos 262 days ago | link

There's a difference between "not being deferential" (which I don't think the complaint is) and talking down to someone who's clearly built something very cool. Put another way, there's a difference between dismissing someone's approach and asking questions to try to understand it better.

-----

gruseom 262 days ago | link

"Deferential" has nothing to do with it. In a technical discussion, the focus should be purely on the content. Inserting one's self into it (such as by being supercilious) detracts from that.

-----

jasonwatkinspdx 262 days ago | link

> Wavelet transformations completely destroy any DCT.

It's not quite that simple. They have different strengths and weaknesses, so you can't say one is categorically better than the other. DCT has uniform frequency resolution, which sounds desirable but isn't once you start quantizing due to ringing, etc. See slide 25 or so of [1]. Wavelets overcome this problem by adapting the resolution in opposite ways at the extremes. This works fantastic for medium signal rates, but can have severe low passing at low rates. For low rate coding DCT with the bells and whistles (deblocking, etc) will typically win.

[1]: http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro...

-----

seldo 263 days ago | link

This is neat tech and works pretty much as advertised, but man, this UI is pretty rough. The blue background and curvy borders are strangely superfluous; tapping the left-bottom corner controls pops up an intermediate selector but the right-bottom controls work in-place; taking a shot produces a big "infinity" symbol that fades in and out of view -- I don't know what it means.

Good work on tech, please hire a UX specialist :-)

-----

jpap 263 days ago | link

Fair comments, and much appreciated.

I did all of the graphics design myself, in the app and on the web. :-)

The infinite sign you see does require an explanation. I'll take your advice and think about how it can be done more simply.

It's basically telling you, the user, that the capture buffer has filled, and you're now dropping (some) shots.

-----

josh2600 263 days ago | link

Have you thought about displaying a semi-transparent bar to show the buffer? Or maybe a one or 2 px white mark creeping up the side of the screen (turning to red as it get towards the top)?

Just some thoughts. If you have a buffer, and I'm gonna get fubarr'd if I hit the limit, you should probably show me the buffer (not just a warning that it's too late).

-----

jpap 263 days ago | link

Versions 1.x.x of SnappyCam had a linear buffer [1] but I felt it was distracting.

I generally can see the "end" of the circular buffer around the shutter button, so it doesn't seem to be an issue for me. Perhaps I tend to touch it on the lower-right instead of dead-center.

I made an effort to support lefties in the UI (see Advanced Settings), but the buffer doesn't spin the other way just yet. (To be honest, I've had to deprioritise that in favour of other features.)

Are you left handed?

[1] Yes, that's me jumping near the GG bridge. I'm quite good at it now, as you can imagine: http://a3.mzstatic.com/us/r1000/085/Purple/v4/c5/06/d5/c506d...

-----

josh2600 262 days ago | link

No, not left handed, but I didn't get the infinite visual cue until you explained it here or the border.

The red bar on the bottom woul probably be fine of you could make it like 70% transparent until it gets toward the end, the vacillate it between 0% and 50% so it looks like its flashing. Some visual indicator that I should be paying more attention to it.

-----

seldo 263 days ago | link

I hadn't realized the line around the shutter button was supposed to change at all. That's definitely not where I'd put it.

-----

danpat 263 days ago | link

Nice crotch shot jpap :-)

-----

voltagex_ 263 days ago | link

A red thing might look too much like a "Recording" dot, though.

-----

nwh 263 days ago | link

Looks like they've an instagram-type site set up too.

http://snappyc.am/2LdRMF28U0

http://snappyc.am/4HHxyCad7D

http://snappyc.am/3G3i6QCJUk

-----

aroman 263 days ago | link

Wow, I'm usually pretty skeptical about these new niche social networks, but I've gotta say that is really really cool.

Great work on the "living photo" idea -- that kind of adds this magical sort of feeling to it, which, I've gotta say, I felt.

I hope your hard work pays off. You've built something special here :)

-----

jpap 263 days ago | link

Thanks mate---comments like yours make my day. :-)

I'm not actually planning on creating a new niche social network. I think, given the vastness of some of the ones that've emerged of late, it would be more fruitful to leverage the ones that exist today.

Can't wait for you to see it!

-----

pdog 263 days ago | link

Given that niche social/mobile networks for short videos are so hot right now (ie. Vine, Instagram Video), it might be more fruitful for you to plan on creating a new one.

-----

ianstormtaylor 263 days ago | link

+1 I think this could actually take off as a niche social/mobile network. (Despite how ridiculous that sentence sounds.)

-----

jpap 263 days ago | link

The Cinemagram guys started off down that path, an Instagram clone for animated GIF cinemagraphs, then quickly pivoted toward short videos to compete with Vine and Instagram on similar territory.

I'm still not convinced. :-)

-----

prawn 263 days ago | link

Get convinced. These could be a lot of fun. You just need some help with the design/interface, but the tech is great.

-----

robryan 263 days ago | link

Yeah, I think there is a niche for this. One of the main criticisms of vine is that there is a certain magic to pictures compared with video which comes off as more real. We don't want to save the reality of our lives, just the filtered moments.

I really think this could fill a space in between the perfect single shot and the realness of video.

-----

jpap 263 days ago | link

Glad to hear it. I feel the same on the format: it's closer to a photo on the left of the spectrum between photo and video, where vine/instagram/cinemagram are on the far right.

You've put it very nicely--there's definitely some magic about a "silent moving picture".

It requires and sparks the imagination in the viewer, evoking emotion perhaps as easily as a carefully crafted cinematic short.

-----

prawn 262 days ago | link

It's also a photo with more context. If the presentation allowed the owner to pick the starting point, and users to slide forward and backwards, commenters could suggest other funny/interesting frames.

The work required to set up a basic social network around this would surely be insignificant when compared with what you've done to date! Yell out if you need help on the design side of things.

-----

tomrod 262 days ago | link

It honestly looks like the photos from the Harry Potter movies. Love it! Great work!

-----

skrebbel 263 days ago | link

Just for your info, the top link actually crashed my Windows Phone 8's Internet Explorer. Clearly that's an IE bug and nothing else, but I thought you should know :-)

Keep up the good work!

-----

jpap 263 days ago | link

That's unfortunate. I suspect there's a JS memory leak in the viewer, as it will cause a crash on iOS Safari after prolonged use. It's on the list.

Thanks for letting me know!

-----

jpap 263 days ago | link

Nice find! It's going to form part of the Facebook Timeline integration that's coming soon... as well as embeds, though Josh already linked to the "press samples" page in the article. :-)

-----

marcamillion 263 days ago | link

Do these work for you? When I load the site, I see the photo scrub on the right hand side of the screen - and the slider moves up and down...but I don't see the big image loaded in the center of the screen where I expect the image to be.

Unless it takes a while to load - in which case I was just being uber impatient.

-----

jpap 263 days ago | link

I saw this a couple of days ago. Are you using Chrome?

It might be yet another Chrome canvas bug. :-(

Try Safari and let me know if you can? :-)

-----

dlsym 263 days ago | link

Does not work in FF 22.0, neither in Chrome: 28.0.1500.71 or Opera. (OS: Linux Mint)

-----

jpap 263 days ago | link

Ouch. Looks like some work for me ahead.

The problem with Chrome 28.0.1500.x (.95 here) is troubling me. It seems a more recent problem that I'm convinced is another browser bug.

Thanks for the detailed version report, that's really going to help. :-)

-----

groby_b 262 days ago | link

Hm. Works on Chrome 30.0.1582.0 (Canary) - after repeated page refreshes. Was stuck on 98% several times. Disabling the cache doesn't allow me to repro it, emptying the cache doesn't either.

Works with 28.0.1500.95, too.

However, looking at the console, I see occasional instances of Resource interpreted as <blah> but transferred as MIME type <foo>. Not a big deal, but maybe a pointer.

-----

icebraining 263 days ago | link

Works fine in Firefox, FWIW.

-----

BuddhaSource 263 days ago | link

OMG, it works on IE 10! Congrats!

But not working on Chrome or FF :|

-----

jpap 263 days ago | link

What version(s) are you on? I spent a lot of time checking in more recent versions of FF after developing the site.

I do pretty much everything on OS X these days, but I did fire up a couple of VMs with Win XP, Win 7, and Win 8 to test.

I did my entire dev on Chrome, and apart from what appears to be a new canvas issue at present, I'm not having any major issues. (I'm looking into the canvas problem.)

-----

wingspan 263 days ago | link

It also doesn't work for me in Chrome on Win8. It does work in IE10, though.

-----

jpap 263 days ago | link

It shudders me to think that IE won over Chrome. :(

I saw an issue also reported on here earlier today a few days ago. Sounds to me like a (new) canvas bug in Chrome.

There are unfortunately several workarounds for browser bugs in the HTML5 viewer. The AS3 Flash port was surprisingly very solid. I can't wait to share more info on that when it's time for another major release.

-----

asmosoinio 263 days ago | link

Chrome on Mac here. I was also worried it might not work, as after 100% it still took a long time (didn't measure, but maybe 10 seconds?) to actually show anything. Almost closed the tab, lucky I did non.

Cool app + site, congrats! Would love to use this to analyze my disc golf throws, and share with my fellow disc golfers.

-----

Myrth 263 days ago | link

Chrome 28 on Win8 here - works great, thank you for making it! Even though I personally can't use it since I'm on Android.

-----

ronjouch 262 days ago | link

Impressive job! But I confirm the webapp does not work for me either, it stays blocked at 0%. I'm using FF (Nightly) 25.0 on Ubuntu 13.04 x86_64.

-----

9oliYQjP 263 days ago | link

jpap, I don't quite fully understand the implementation (though I'd love to one day be proficient enough to). But maybe you can explain how the format compares to motion JPEG. Or maybe it's very similar? About 15 years ago I dabbled in live video recording on old Pentium II hardware with an old BT878 video input card. Motion JPEG was the only feasible option to obtain relatively high quality (for the time) results albeit at the cost of disk space.

-----

jpap 263 days ago | link

There are a lot of similarities actually.

In SnappyCam, each photo is compressed to a separate JPEG file. There's no inter-frame compression, no motion vectors, etc. The same as mJPEG.

The main differences are:

* Each photo is stored in an individual file. This makes seeking through the living photo blindingly fast. (I guess you could do this with motion JPEG by utilizing an index.)

* Each photo also has full metadata. Try rotating the camera as you shoot. It will follow you. :-) Same goes for the geo-tagging: included are a bunch of timings that aren't normally included, so you can know the "precise" usec when you took the photo.

* Each photo has it's own thumbnail. That allows me to cheat a little bit in the photo viewer: you will see a flash from blurry to clear as you scroll around.

(There are more cheats in the viewer for decoding and downsampling at the same time before you zoom, to make the photo load faster as well. One of the handful of reasons why I rolled my own decoder as well.)

-----

ramanujan 263 days ago | link

This is amazing work. Could you explain why you decided to go with many individual stills rather than filling in the gaps in a video codec? It's a really counterintuitive approach.

-----

jpap 263 days ago | link

Good question!

Several reasons:

* Video codecs are much more complex.

* Random access seek is a lot slower, unless you're using all I-frames. (That's now a codec option on iOS, but not when I started.)

* "Studio swing" reduces the dynamic range of the YCbCr components so the quality suffers.

* Each frame lacks their own thumbnail, unless you maintain an adjunct "thumbnail video"

* Each frame might(?) not be able to have attached separate metadata, like geotagging, sensor settings at time of capture, etc.

* Deleting one frame causes a "hole" and headache.

* Standards compliant JPEG means export is super easy.

* Anything above full HD video is difficult to deal with in 3rd party software.

-----

andrewf 263 days ago | link

I think it's fantastic that you've managed to turn a long, hard optimisation slog into a real product win. Add me to the list of Australians willing to buy you a beer - but not back home, I live in SF at the moment :)

I'm curious about the low-quality preview you get when scrolling through all the shots. Are you storing low-quality data separately or do you also have a fast, low-qual JPEG decoder? (Is the Huffman encoding between blocks independent?)

-----

jpap 263 days ago | link

Hey Andrew, would love to catch up over a beer. :-) Drop me a note via email: jpap {at} snappylabs.com

You've got a good eye: as part of the JPEG image compress, I also generate a low-resolution thumbnail that's embedded into each file as Exif metadata (along with geotagging, and other camera settings that define the shot, like exposure).

They are used as a "first-in" placeholder for an image.

The full image is then downsampled and decompressed simultaneously [1] exploiting the fact that the (Retina) screen resolution is often much lower than the full JPEG resolution.

As soon as you start zooming, the image is decompressed yet again at the full resolution and replaced in-place as quickly as possible so hopefully you won't see it. :-)

[1] As outlined in http://jpegclub.org/djpeg/ the technique relies on the fact that the top NxN DCT block of MxM coefficients, N < M, can be inverted to form a NxN pixel lower-resolution image of the original MxM block. When N is {1, 2, 4} a fast inverse DCT algorithm can be used with great success.

In fact, N == 1 is a trivial inversion and it might be tempting to use it as the low-resolution image instead of a thumbnail, but you still have to unpack all of the DCT coefficients to get to it, which can be expensive (Huffman).

-----

Oculus 263 days ago | link

I have a feeling that soon SnappyLabs is going to have Apple knocking on their door with a very nice offer.

Kudos to them, sounds like they deserve it.

-----

jpap 263 days ago | link

Thanks!

I just hope Apple's engineers don't get pissed off by the press. SnappyCam is built on their hardware, which can do remarkable things.

Though we as app developers don't get access to a lot of their smarts, e.g. hardware JPEG codecs, I'm sure there's even more innovation in their work that often goes unacknowledged.

-----

ahknight 262 days ago | link

You'd think so, but they take 2-10s to recover from a photo and you ... don't. Clearly there's some room for optimization there. :)

-----

gosu 263 days ago | link

This looks fantastic. Watching people's reactions in that example image was really interesting, and it occupied me for a good few minutes. "Why can't you do the same thing with video?" Because rewinding video is really painful, especially online video.

Criticism:

I use my thinkpad's pointer stick to move the mouse cursor. It's impossible to keep the cursor inside the "control strip" while moving it up and down and also looking away from the strip (and at the image). Too much accidental x motion is introduced.

It would be better for me if you were to enable the scroll wheel (which I can simulate on my pointer) as an alternative time control, or perhaps let me click on the control strip and then hold down mouse1 for as long as I want my y motion to control the position in time.

-----

jpap 263 days ago | link

@gosu, despite what Josh wrote, you can traverse your pointer across any part of the living photo online. :-)

Love that you picked up on the expressions! It wasn't until I got the photos out of the app was I fascinated by the same thing. I really can't wait to enable this functionality for everyone soon. :-)

More elaborate mouse movements are possible, only in HTML5 full screen mode; required to "capture" the mouse (think a game).

The problem with that, too, is that instruction or a tutorial is required. (I'd try to make things as intuitive as possible, despite the failure in the other thread RE UX and the infinite shutter.)

-----

gosu 263 days ago | link

facepalm

What was happening is that I was trying to keep the mouse in the control strip, and it would go off the right side of the image.

Thanks a lot, Josh.

Edit: By the way, the fullscreen functionality isn't launching. But I do have a weird browser (conkeror on xulrunner 22.0).

-----

jpap 263 days ago | link

haha, no worries.

In the app, you need to start with your finger near the thumbnail strip. (But can move it away for fine-grained scrubbing if you wish.)

It's no surprise that the learned behavior is transferring to the web viewer.

-----

peter_l_downs 263 days ago | link

Any chance of this coming to Android soonish? This is seriously cool!

-----

jpap 263 days ago | link

The fast JPEG codec was written for the ARM NEON SIMD coprocessor found in the iPhone. Most Android devices also sport the same architecture, so it is indeed possible.

The code for the codec is written in mixed C and assembly, so it can be "easily" ported to Android by making use the JNI.

While the R&D for the fast JPEG codec took about a year to perfect, the iOS app took just about the same time to get polished (including the NodeJS backend work, the HTML5 website and embeddable widgets in AngularJS).

Writing the rest of the app would take a few months of full time work, and it's not yet clear if that might pay off at this stage.

We'll see... and glad to hear there's interest! :D

-----

fluidcruft 263 days ago | link

Don't overlook the fact that the source for the stock Android camera is available under a commercial-use-friendly open source license and has a quite nice native android UI. You don't have to reinvent all the wheels unless you're stubborn.

https://android.googlesource.com/platform/packages/apps/Came...

I would buy that in a heartbeat.

Unrelated, how quickly can you alter exposure settings? Can you get 30 pictures per second with three interleaved exposure brackets? (i.e. burst of 10 HDR photos / second) That would be very, very, very, very cool.

-----

jpap 263 days ago | link

That's really interesting. I wasn't aware of that. I'll have a look at it once social sharing is out the door.

I did consider getting into other aspects of iPhoneography, like HDR, etc. The trouble with HDR in particular is that there's no API access to direct the sensor into each of the bracketing modes.

In the case of HDR, it might be more fruitful to attempt some kind of image signal processing, similar to "Clarity" on Camera+.

I looked into that for a while, and I figured that Camera+ might be using some version of the Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm. In any case, what they've done is really neat from a DSP perspective. :D

-----

est 263 days ago | link

Hi,

There's also a cool technology allows you to save near the same jpeg with much, much smaller file size.

https://news.ycombinator.com/item?id=2940505

-----

vidarh 263 days ago | link

If I was you, if you're unsure it might pay off, I'd go to the phone makers and offer them licenses for just the encoder.

-----

est 263 days ago | link

There's similar app on Android for years

https://play.google.com/store/apps/details?id=com.spritefish...

The claimed speed is 30 fps, the moar RAM the better I think.

-----

jpap 263 days ago | link

I thought the app was pretty cool, just super slow to save out the JPEGs.

That's one of the reasons I spent a lot of time to make sure SnappyCam could compress these images, thumbnails and Exif metadata included, at a ridiculous speed.

-----

lvs 263 days ago | link

I might point out that, depending on the device, you could have many more cores to work with in Android-land.

-----

venomsnake 263 days ago | link

We should create a "marketing core" term the same way we have marketing HDD size. I have seen big.little advertised as 8 cores.

I think the main advantage of the androids will be the fact that high end devices have generally more RAM than iOS counterparts. So even if the codec cannot be pushed as far as on the iOS the bigger possible buffers can help.

-----

jpap 263 days ago | link

Yes, more RAM definitely helps.

On SnappyCam, I had to arbitrarily limit the size of the buffer to a fraction of the system memory because there's no way to know "how much" RAM can be allocated to avoid the dreaded memory warnings until you hit one; and then it's a three strike's out policy: you get two and the third kills the app.

The first two are "soft" warnings, but I suspect have a lower threshold than the "hard" one that sends SIGKILL.

In setting the limit arbitrarily, I unfortunately have no choice but to select it rather conservatively: it might otherwise be (a lot?) higher.

-----

K2h 262 days ago | link

is there any value in having buffer size selectable for advanced users so they can play with it and see where the sweet spot is on their hardware?

-----

est 263 days ago | link

Yes, SnappyCam has a faster algorithm advantage.

Android on the other hand, has more open and better hardware, e.g. larger full-res camera, much larger RAM, faster external storage or even OTG, more CPU/GPU cores. In theory if SnappyCam is full ported to Android you can make it faster than 8M pixels @ 20fps

-----

jpap 263 days ago | link

The possibilities are exciting for sure! :-)

-----

dvt 263 days ago | link

I might try to (naively) implement something similar for Android just to see how fast it goes without too much low-level fiddling. It really is pretty darn cool.

-----

jpap 263 days ago | link

Awesome! Drop me a line, would love to see how you go. :-)

-----

gandalfu 263 days ago | link

Purely wishful thinking: how about this and the latest lumia camera?

-----

jpap 263 days ago | link

I don't know much about their device (maybe Nokia might send me one?!? hehe)...

... but they must have an awesome JPEG encoder. I'd assume 41Mpx stills would need to be compressed in no more than 1 second for a reasonable UX. That there is a 41+Mpx/sec encoder.

I've also noticed they are using higher quality chroma sampling (4:2:2) so their encoder is actually doing a lot more work than say SnappyCam.

But I bet they're not doing it in software, either.

-----

gandalfu 263 days ago | link

They will send you one if you send them 670 USD back...

-----

More



Lists | RSS | Bookmarklet | Guidelines | FAQ | DMCA | News News | Feature Requests | Bugs | Y Combinator | Apply | Library

Search: