I'm not sure what resolution these images are, but if it's very high-res, it may make sense to make intermediate-sized versions first. I expect that would make recreating thumbnails (i.e. in a different size - or multiple sizes) much quicker.
Also, converting the (hi-res) images to JPEG before further processing may make the data somewhat easier to deal with. 25-30MB per image * 2 million would be 50-60 terabytes of data? Plus, how often would someone really benefit from having access to a 25-30MB TIFF vs a much smaller JPEG version in the same resolution? That alone could be reason to start with JPEG compression first.
It's not about performance, it's about <del>parallelism</del> concurrency.
If you don't care for Node's ability to handle multiple requests <del>in parallel</del> concurrently, with a single process with only one thread, than you might not have a need for Node at all. Why not use Ruby or Python?
If you really want to force Node into being a dumb scripting engine, you could use the various sync functions that Node.js provides. Problem is, they block the entire thread. But if you want to, you can. And I believe with some clever hacks, you can use the blocking nature of (for example) the fs sync functions to make other calls (including http requests) synchronous as well. But unless you use Node to build something that's not a server (people build command line tools with it as well), I don't think you want to.
By using synchronous functions, everything Node is built for will go down the drain. Everything done will be done in sequence, and during the time the sequence of operations has not finished, the Node runtime will be unavailable to serve any requests.
I suggest you look into promises as a somewhat saner way to deal with asynchronicity than callbacks, although you can pretty far with just the Async library. The future might be in Generators, a new feature of EcmaScript 6. This is available in V8. You can use this in Node 0.11 (unstable) if you use the --harmony flag. See here for a nice writeup: http://blogs.atlassian.com/2013/11/harmony-generators-and-pr...
You're confusing concurrency and parallelism. Node is a single-threaded event loop. It can only do one thing at a time. Node can perform many operations concurrently, but they're not running in parallel. If any one operation blocks, the whole event loop stalls.
Parallel execution requires multiple threads running on multiple CPU cores. Node requires multiple VMs to do this.
Does anyone else have trouble understanding (or "intuitively reading") the graph in the f.lux beta preferences? I discovered that's a kind of "ego-centric" graph. I mean ego-centric just like there once where earth-centric (and later) helio-centric models of the universe.
Because the graph is totally ego-centric, the graph starts when you wake up. I just can't wrap my head around that. In my mind, I wake up at a specific clock time, and the universe is configured in a certain way at this particular moment. In particular, the sun has a certain position in the sky. (interestingly, I use an earth-centric model in this regard).
What's (relatively) constant for me is how the sun moves through the sky (this depends on where you live on earth, plus time of year). Obviously, it's beyond my powers to change the time of year. I could change where I live on earth, but I'm not doing that very often. What's directly controlled by me is when I wake and go to bed... Why can't I change these positions on an otherwise static "map"?
I don't want to express the current year as relative to my life either. I.e. three periods: "the time I hadn't been born yet", "the time that I live", "the time beyond when I died". It's rather insane. Yes, we use Jesus date of birth as a reference point now, you could say that it's bad and we should count from a different epoch or so, but at least things are not expressed relative to my life.
The point I was trying to make is that I see this graph as doing something comparable to saying you're (by definition) born in "the year zero, at 00:00:00".
This is a nice example of a "commitment device", "a means with which to lock yourself into a course of action that you might not otherwise choose but that produces a desired result". https://en.wikipedia.org/wiki/Commitment_device
"Attendees [...] learned a technique called "pre-hindsight" that uses emotional cues to create more foolproof plans. It works like this: Imagine that six months have passed, and you haven't achieved the body of your dreams. How surprised are you? The less surprised you are, the less likely it is you will succeed at your goal. Then think in detail about each reason you wouldn't be surprised if June comes and the number on the scale hadn't budged. Each reason—whether "I don't have time" or "I don't like running in the mornings"—is a possible cause of failure. Using the surprise level to anticipate these is crucial to creating a plan to address each weak point."
I like this technique a lot. Thinking about how surprised you'd be if you do not achieve your objective.
I think the technique has similarities with the exercise that Steven Covey writes about in "The Seven Habits" where you must think about what you want other people to say about you at your funeral. It engages you in a special kind of way, perhaps helping you to uncover knowledge that you did not know you held.
"This is annoying at the level of Arrays, but gets more difficult with more complex types, and function interactions. The recent brouhaha around the Promise/A+ highlights one such example: It is difficult to return a Promise of a Promise as a value from onFulfilled because then duck-wraps the return value as described in the Promise Resolution Procedure"
Could anyone share a pointer to this brouhaha around the Promise/A+ [spec, I assume]?
It seems the blog had a kind of "invisible" filter on the comments, which made it appear there were zero comments, or one (your own) if you had posted one. Now a whole pile of comments have become visible (manually approved I think). Funny to see all the reactions. Most commenters were probably not aware of one another.
I'm not sure it would be worth it. While I think it is neat to finally see .NET popping up more I really don't see what this benchmark proves even if the C# and js both sorted the same type of data (string/floating point).
Also, converting the (hi-res) images to JPEG before further processing may make the data somewhat easier to deal with. 25-30MB per image * 2 million would be 50-60 terabytes of data? Plus, how often would someone really benefit from having access to a 25-30MB TIFF vs a much smaller JPEG version in the same resolution? That alone could be reason to start with JPEG compression first.