Just the other day I had to make a choice between using threads and event loops for an IO bound workload (in Python). The constraints were that I already had a piece of code that implemented its own event loop and I didn't feel like re-writing it; and I needed this code to be very memory conservative. Well, I tried threads. I figured why not? They are so ridiculously easy to work with in Python. Turns out there is no problem whatsoever for my workload to run on a 1000 independent threads and the memory usage goes up by only about 20MB. The best thing about it is that I don't have to re-write my underlying library and since everything I do here is IO bound, the GIL does not bother me.
My big problem with event loops is that while your framework may be all asynchronous, your code will not be. Try to make a call to MySQL and you will block the entire process. On the other hand, in this case your thread will just be put in a sleep mode and another thread will take over. Yes, there is overhead, but depending what you are doing, threads can be a great tool.
Of course a web server developer can think multicore is no big deal. The entire architecture of the web makes it easy to split load between entire machines, splitting it automatically between two cores isn't that hard. And that's fine and dandy, as long as you're working in a domain where the real computational workload is being done somewhere else and you're mostly bashing <1MB of strings together. Pretty disingenuous to project out from there, though; ask the database implementors if anything has changed.
Also the conclusion is really weird; concurrency will be hidden away from programmers even more than it is now, but it will take the form of message passing and event-loop programming? That's hardly hidden, that's fundamentally rewriting how you write programs to work with concurrency. It seems to contradict his entire thesis. The details may be hidden but they already are; semaphores, for instance, are already an abstraction.
> the multicore machines are already here and you can already see that nothing has changed
Really? I see lots of change, I look at problems and solutions differently and care if a piece of technology can use multicores. Just because you keep your head in the sand doesn't mean multicore isn't changing the world and leaving you behind.
Agreed. When I write code on a single core machine, then watch it peg only a single core when I run it on a multicore machine, I start thinking about changing languages. That's a pretty big change! [Before you say I should leverage features in my current language to perform better on multicore machines, keep in mind that's also a big change! I'd rather switch to a language that does it for me, or at least makes it trivial.]
The author defines parallelism as splitting a computation into many independent tasks which interact very little, and concurrency as modifying things from different threads/processes/tasklets/whatever without incurring in hairy bugs. (As if it is ok to have hairy bugs in parallelism, or the lack of interaction rules them out).
Is this the standard/accepted/correct definition ? Or he just made it up ?
My understanding is that parallelism is concerned with actual execution of the code in parallel on the machine. Whereas concurrency is concerned with the semantics of the language, i.e. whether it is possible in the language to define multiple "tasks" that need not be run in serial order.
Whether those tasks actually run in parallel or not, is a concern of parallelism, not of concurrency.
Concurrency does not necessary mean parallelism; you can have have threads interacting in a time-shared manner on the same processor.
Parallelism implies concurrency; a parallel program has threads of execution (which may not actually be shared-memory software threads) that are executing simultaneously.
The author's definition of parallelism is wrong - at least, it's different from the one myself and everyone else I know in the high performance computing and systems research community use. Many interesting problems are not embarrassingly parallel, and figuring out how, exactly, such problems can be solved efficiently in parallel is hard. (And efficiency is a solution requirement, since it's not worth the parallelization effort if you don't achieve it.)
I think the point the author is trying to arrive at is parallelism does not necessarily mean shared-state across threads. Which is true, and I share his sentiment that we should probably not reach for shared-state threads to solve our parallel problems.
Then the wikipedia article doesnt quite match with whats taught in CS courses. In any case the blog post does not agree with the wikipedia position either. I dont think it helps to overload standard definitions with different concepts, one should use new/different words.
"Concurrency should not be confused with parallelism
Concurrency is a language concept and parallelism is a
hardware concept. Two parts are parallel if they execute
simultaneously on multiple processors. Concurrency and
parallelism are orthogonal: it is possible to run
concurrent programs on a single processor (using
preemptive scheduling and time slices) and to run
sequential programs on multiple processors (by
parallelizing the calculations)."
The posts definition does essentially agree with your definition, because it is concerned only with software.
You write parallel software (use parallel algorithms), when you want to exploit hardware parallelism (usually multiple CPUs) to do one thing. While you write concurrent software to do multiple things at once, that require same resources (CPUs, Memory, etc.). There is theory of parallel algorithms that is concerned with extending of big-O notation to multiple processors and performance effects of inter-processor communication, while concurrent algorithm theory deals with synchronization problems.
In essence one can illustrate the difference by looking at synchronization primitives used: while concurrent code mostly uses locks, parallel code uses barriers. And on many parallel supercomputers of eighties, synchronization of parallel code was implicit, because all processors had common clock and thus you could do all synchronization by simply knowing that what given processor is doing when.
This is relevant to this "threads are useless" debate by the fact, that threads are meant for concurrency and not parallelism, because they implicitly share resources (memory) that is mostly not required to be shared. Large part of complexity of modern multiprocessor hardware (and operating systems) comes form effort to detect which part of memory is shared and which is used only by one processor (or thread). Because sharing whole memory array incurs non-trivial contention and precludes you from doing any meaningful caching.
I wouldnt say they are my definition :-), it is the standard one. Re-defining same terminology as something else or making them open to interpretation, is detrimental. More so when the point of view could be contentious.
Ideally definitions shouldnt only be essentially the same (for some values of essence), but obviously/unraguably the same. Or else we discuss definitions :-)
The point I was trying to make is simple. By conventional terminology: Concurrency deals with semantics,
parallelism deals with execution. I dont see how
parallelism as splitting a computation into many independent tasks which interact very little,
and concurrency as modifying things from different threads/processes/tasklets/whatever without
incurring in hairy bugs.
translates into the standard definition. What the author describes as parallelism could easily be concurrency if the split happens at the semantic level.
BTW concurrency needn't have any explicit synchronization primitives at all and needn't have anything to do with synchronized clocks. Semantics does not care about execution or clocks it cares only about meaning. Since parallelism deals with execution, there clocks could become relevant.
Consider list comprehension in Python. Its a concurrency primitive. But whether the elements of the list are computed simultaneously, depends on how the Python code is executed and hence an issue of parallelism.
"Concurrent code" and "parallel code" terminology can also be confusing. If a piece of code describes/defines things you want to compute, then it can only be concurrent and not parallel. But if the code also describes how it is going to be executed in the hardware then it could be called parallel. So I am not confident of the view
threads are meant for concurrency and not parallelism
I have no difficulty in accepting that threads are not always the right abstraction. Shared mutable state is a significant problem.
But forwarding that argument by conflating conventional terminology as was done in the article hardly helps.
I'm not quite sure I see the point. Programmers don't need to care about multi-core, because things like your web server and database will take care of it for you? Then what about the people developing those software products?
Then what about the people developing those software products?
The article is very clear about this:
"I claim that essentially nothing will change for the average programmer with the advent of multicore machines."
He does not deny that people writing web and database servers do need to care about multicore CPUs. And I guess it is mostly true, so many daily tasks are I/O bound that the only concurrency most programmers will see are separate GUI/worker threads.
Of the tasks where parallelism does help a lot, a substantial subset is embarrassingly parallel. E.g. we parse a lot of large corpora (Wikipedia-size and larger). Parallelization is simply obtained by chunking a corpus, and giving every CPU in the cluster a chunk. Then there are number crunching applications with data dependencies where just a few OpenMP pragmas can easily parallelize programs. Of course, there are remaining hard tasks which require more hand waving.
I don't think equaling "database using web applications" with "average programming" is being clear about the matter. Simply because that segment might not be immediately influenced doesn't mean the new technology won't be picked up.
And even as a web developer, I often write more than just something I throw at the web server. Many web applications require cleanup scripts, batch processing of messages or static content, or perform some kind of statistical analysis on their data models. Given enough data, these are all areas where multiple cores can be relevant.
But arguably this smaller set of programmers have high impact. So it would be a little presumptuous to dismiss the need to understand parallelism. Notions, as those held by the author of the blog post, can have a few undesirable effects. One, it discourages new hackers from messing around with parallelism, which may deny him/her of an opportunity to have a high impact. Two, though it is true that it is not necessary to know the inner workings of a car to drive, sometimes it helps.
But may be hackers weren't the target audience in the first place.
This is already the case with systems programming in general. The set of people who can program competently in VM-based languages is much larger than the set of people who can program competently in C.
I agree with you. But wouldnt it be good if high level languages supported concurrency. This would make it easier to program certain tasks. A popular opinion I encounter often and also reflected to an extent in the article is that such a route has no merit.
There is no reason to believe that HLLs have to be particularly inefficient (for example the scheme compiler stalin can hold its own and often run faster than an equivalent number crunching code written in c). I am not arguing that Python should be that high level language, but that the current state of "divide" (c for programming servers python for apps) is a little artificial and is a result of a lack of options. Not grappling with parallelism only makes the scene a little more entrenched.
I find X10, Fortress and Chapel quite exciting as they are trying to straddle this gap.
I'm all for models of parallelisms in high level languages. I was pointing out that writing parallel programs without high level models - such as using Pthreads or MPI directly - will probably remain at the systems programming level.
I think we'll need to add models of parallelism to high level languages, and most programmers will have to deal with it. But, that's not a given. This author is on the other side: maybe the majority of programs will be able to remain sequential, and we can allow the exploitation of multiple cores to happen mostly at the OS level through the scheduler.
For machines today, I would have to agree with the author. But what happens when a single core is no longer sufficient to handle your "simple" processing requirements in a timely manner?
Time requirements are constantly getting smaller, and computing requirements are getting larger. Serially generating data, even just for web pages, won't work forever (and I'm willing to hazard a guess that for the major players, it doesn't work now).
Multiprocessing, while a great advancement for Python, is not what I would consider lightweight. It's not a problem for the dual- and quad-core computing era, where you have a few number of spawned processes. However, if you go to a 32 or 64 core era, and the ability of multiprocessing to scale will be dramatically affected by the memory and processing overhead required.
We're fine now with the tools we have. However, if these tools do not evolve, we won't be fine in the future.
Could'nt agree more. If the point of the article was: for people who want to do the same old thing then they neednt bother about the new(^) changes, then thats almost a tautology. Someone will find a way to do something useful with the new capabilities of commodity CPUs. If I do not want to delve in those bits, it ensures that the person wont be me.
^ Disclaimer: parallelism/concurrency is far from new though.
My big problem with event loops is that while your framework may be all asynchronous, your code will not be. Try to make a call to MySQL and you will block the entire process. On the other hand, in this case your thread will just be put in a sleep mode and another thread will take over. Yes, there is overhead, but depending what you are doing, threads can be a great tool.