Hacker News new | comments | ask | show | jobs | submit login
Real-time GIF images (github.com)
211 points by erikvdven on Dec 20, 2015 | hide | past | web | favorite | 67 comments

Motion JPEG is arguably better suited to the task as the browser knows not to cache the previous frames by rule. It's actually designed for this exact purpose.

I made a little example project a few years back. The format is dead simple, keep the connection open, flush multiple JPEGS separated by a header. It's used pretty often by cheap security cameras and is supported by most browsers. It just seems to be not widely known about.


I had a lot of fun making a little real-time visitor counter using Motion JPEG a few years ago! I was quite proud of that :)

The demo is still live:


I've found MJPEG to be more bandwidth intensive, especially as the resolution increases however, since its flushing the full frame every time (whereas with GIF you could send just the differences between frames).

Would depend on what you're sending. If you take make good advantage of gif interframe transparency and are just sending small updates the result could be very small. That still leaves the problem unless I'm mistaken of the browser hanging on to the old gif frames.

Are you saying that retention is bad from a memory use perspective, or that there is a risk of looping mid-broadcast?

Memory use. If it thinks it might finish and need to loop it would hold on to all the frames it's received.

I believe gmail caches all the images so that leaves out a huge amount of the world maiboxes.

I have tested an image generated with this script in both Gmail and outlook clients and it works just fine. Else I wouldn't post it over here of course :)

You're right. Google changed the way they did images a while back to prevent email image marketers who used to track when the user opened an email using a 1pixel transparent image downloads.

"1x1 transparent pixel" is called a "tracking pixel".

Gmail still allows marketers to track when a user opens an email (even better than before, since it no longer asks users whether they want to load images). However, they proxy the request so that the tracking pixel host no longer gets the user's IP address or third party cookies.

Where can I read more about this? Doesn't this help spammers? I thought I was safe if I disable auto-loading of images!

Then don't open spam? I don't think a spam email has made it to my inbox in almost 5 years. Google's proxy protects your private data and only informs the sender that the email has been opened. I would hardly call that "unsafe"

You're safe either way, but if you have autoloading disabled they still can't track that you've read until you load the images.

The change is that clicking to load images is no longer the default [0]

Do you get enough spam on your Gmail for this to be a worry point? Genuinely curious; I get roughly zero.


> You're safe either way

Eh? Before they used track-me.example.com/IDHERE and got blocked, now they use IDHERE.track-me.example.com and don't get blocked.

Move the unique ID to DNS and Google will let it through.

I assumed by safe you meant something else.

Yes if you load images they can track that you have opened the email.

I fail to see how that is 'unsafe' though. It's expected functionality.

It's expected that they can track you, when before they blocked the images to prevent that?

They changed about two years ago to run all images sent to a gmail address through their servers (and presumably store them). It specifically didn't effect marketing though. I was working in email marketing at the time and there was some temporary gnashing of teeth but this was not much more than a blip.


From my testing, it seems like Gmail will cache via its proxy, but that cache respects the cache headers, so you can instruct Gmail how often to refresh the data.

(That, and you can cache-bust on a per user basis using a unique identifier in the query string.)

Few month ago I tested this, and it's seems they do not cache. Instead they are just using proxy to enable anonymous download of the image. But you never it any cache from google directly.

I actually wrote a little tool to help test this: https://github.com/kale/image-cache-logger

You'll see that it in fact does get cached by Gmail, but only for a limited time.

They do cache at least sometimes, and they must for it to work.

If I send you an image URL http://example.com/fH27cAw, and they URL is requested, you must have opened my email (unless something is requesting them regardless).

It does work in gmail. Jetstar, an Australian air travel company, uses this technique (IIRC) and it works fine in gmail.

And regardless, I cannot support a project that I know will primarily be used to annoy the shit out of me in my own inbox.

Dont dismiss something just because the current usecase doesnt suit you.

Actually, I implemented this very thing in C# on top of weather radar imagery back in 2006 (to be able to overlay real-time weather on a 3rd party mapping widget), so I've given the entire concept full consideration already.

I'm mildly baffled by the coding style of the main encoder class[0]; where do these conventions come from?

(All-caps three character properties, namespaced(?) arguments, spaces e v e r y w h e r e, curly bracket array access.)

Also, I didn't know `class` and `array` were case-insensitive.

[0] https://github.com/ErikvdVen/php-gif/blob/master/GIFEncoder....

Comments say it's based on this: https://github.com/demouth/DmImage/blob/master/example/sampl...

Some of what you mentioned is already present there. Some other things are new. But they're both terrifying...

Extra details: the rewrite into gifencoder happened in 2007, so gifmerge must be older than that (maybe 4.x times). The original license forbids modifying gifmerge without author's approval - since this file doesn't mention it at all, I doubt dmimage is legal.

> But they're both terrifying...

Welcome to PHP! Truly the language which most directly represents the soul of the web - a stateless static documentation presentation system which we shoehorn application functionality into regardless of the pain!

The classic coding book 'Code Complete' recommends adding spaces after and before brackets. The idea being the increased white space improves code readability.

I used to agree, but more recently have decided it ultimately ends up making things more difficult to read by increasing the amount of scanning your eyes need to do.

This is better: https://github.com/videlalvaro/gifsockets - it's event based so doesn't tie up lots of memory while running and doesn't stop after 30 frames.

Oh, this is actually neat:

> what about progress bars for stuff that your server is doing in the background

This seems like a really good micro-service.

Yes, the idea is brilliant! But the purpose of my source code is to use "real-time" gif images in mail clients. Unfortunately, videlalvaro's solution is only possible for mail clients which support loading images from its original source. Google loads the images from their proxy, which downloads the images first. It DOES re-download each image at the moment you re-open the e-mail message, but it won't make it possible to manipulate the images real-time. I've tested it. It does work in clients like outlook, cause outlook shows the images from its original source, but gmail doesn't. So sending a gif image with a certain amount of frames is better, in my opinion.

Yep, i was expecting OP's code to be the PHP implementation of that, but a quick look at index.php shows its not. Streaming gif frames, now thats what i call a live gif.

Funny how we go full-circle. This is roughly the way I did live video in '95.

First thing I thought of when I read the title were 1990s era webcams.

When I read the title, the first thing I did was laugh and think about telling you to 'check this new thing out'.

A guy I used to work with wrote a tiny JavaScript library that would write into GIF content to create motion graphics. It actually worked, in 1997.

A bit later, web browsers cleaned up their implementations, and it no longer works.

Good times, though.

I don't think it is really funny. Hacks like this is how progress is delayed. Because why innovate when you can rely on "hacks that provably work in the real world"?

This hack is an innovation in its own right. And it should work with most browsers, contrary to completely new technology.

Well, it was innovation 20 years ago. Today there are more direct ways that achieve the same effect (such as canvas) without taking these roundabout ways.

Good luck finding an email client that supports canvas.

Good luck finding an email client that will load animated gifs from remote sources and will keep the connection open indefinitely. Personally I'd consider either one of those to be a bug worthy of a ticket.

The only issue with this kind of hack is that it uses a lot of bandwidth continuously, as long as the image exists. I wouldn't exactly consider GIF to be a good compression format for live streaming.

This might not be an issue for users on broadband internet connections, but on low bandwidth, metered cell phone networks, and congested Wifi networks, this could cause issues (such as slowdowns or unintentionally using up your metered bandwidth allocation) that wouldn't be nearly as bad if a piece of Javascript was used to live-update a page instead.

I wonder if it would be possible to make a live WebM version of this instead.

If I read the source code correctly, this is not actually streaming something continuously; instead, it is making a 30-frame gif that has a start-time that is dependent on when you loaded the gif.

Additionally, GIF has only 256 colors.

Only 256 colours per frame, but they don't have to be the same ones, and can be combined to use a wider palette - for example: http://notes.tweakblogs.net/blog/8712/high-color-gif-images....

This reminds me of an IOCCC entry (further details elided; read the hint file if you want it "spoiled") [1]

[1] IOCCC 2013 winner, most timely rendered: http://www.ioccc.org/2013/mills/

Netscape did a similar thing to this in 1994 with their Fishcam, converting frames from a video capture and served up as new frames of a GIF image. It was eventually shut down in 2007 and the original developer[0] setup a dedicated site for it: http://www.fishcam.com/

Nice to see the forgotten technology repurposed.

[0] http://www.montulli.org/lou

Great bit of source code. We've taken a similar approach to create free timers for email marketers.


Well a good way to troll. You send an image to someone in the email and then change the image once the person forwards this email to others.

not just trolling but could also be used for social engineering...

You are right. You receive one image containing a inspirational message for your family and decide to send to your family members. Then it changes to a image asking for money to be sent to an account because you are in need. I could see this working.

Funny coincidence -- I just received an email with a countdown like that today and was very curious. In my case, the email was using: https://www.realtime.email/

Now you can do it yourself haha ;) I still have to cleanup the code a bit. Thinking of creating a class which allows you to create such image by just writing a multidimensional array, so less code to write. But I couldn't wait to share this already so... But updates are coming soon.

Intriguingly, I've been thinking about exactly the same thing recently.

why not go to the next level and do Real-time video generation: https://pv.idomoo.com/

Well, the purpose of the code is to use it in mail clients. And because mail clients don't allow you to use javaScript and such, using an image is the most obvious alternative.

from the source code: "Updated at 2007. 02. 13. '00.05.AM'"

Urmm, going by the filename ("GIFEncoder.class.php"), that's just the GIF Encoder. This post isn't about the ability to generate GIFs using PHP (as that class does), but about varying the content of the GIF on a per-request basis (in "real-time").

Looking at the git commit history, this was created ~2 days ago.

PHP. On my Hacker News? :)

I am confused? How is this being upvoted? This kind of project has been a staple in the programming community from gaming to forums for two decades now.

"Why are people upvoting something they didn't know about that I did know about?"

This site isn't for "today I learned"'s. There is a reasonable limit to the context of conversation and submissions that happen here.

And instead of being snarky, you could actually explain what is so novel about dynamic image generation on a server.

It has? Can you give an example?

This is generating a custom GIF file on the server. It's most likely been done by someone somewhere before, but not something I've been both gaming and forum-ing for a little while.

This is roughly how early webcams worked. So, it's indeed not novel.

Gaming forums did generated JPEGs based on time/location (so they could troll you by telling you your IP, as was typical of the teenage demographic)

I have not seen a generated GIF before.

The Roxen [1] web-server had a much more advanced example of this for a site they produced for a watch manufacturer.

The watch on the home-page displayed the current time with moving hands that were behind reflective glass. It was a server rendered gif. (c) 1996 or '97.

[1] http://roxen.com - now an Enterprise / Newsroom CMS.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact