
Python Concurrency: An Example of a Queue - driscollis
http://www.blog.pythonlibrary.org/2012/08/01/python-concurrency-an-example-of-a-queue/
======
dudus
Would have been more helpful to link directly to the original post, that has a
lot more information and which he just copied to his site on the most part:

[http://www.ibm.com/developerworks/aix/library/au-
threadingpy...](http://www.ibm.com/developerworks/aix/library/au-
threadingpython/)

But I think that doing so wouldn't generate visits to his blog.

------
wslh
If you find it interesting I recommend my own articles on the subject:

\- Esoteric Queue Disciplines: [http://blog.databigbang.com/esoteric-queue-
scheduling-discip...](http://blog.databigbang.com/esoteric-queue-scheduling-
disciplines/)

\- Adding Acknowledgement Semantics to a Persistent Queue:
[http://blog.databigbang.com/adding-acknowledgement-
semantics...](http://blog.databigbang.com/adding-acknowledgement-semantics-to-
a-persistent-queue/)

\- Persisting Native Python Queues: <http://blog.databigbang.com/persisting-
native-python-queues/>

\- Using Queues in Web Crawling and Analysis Infrastructure:
[http://blog.databigbang.com/using-queues-in-web-crawling-
and...](http://blog.databigbang.com/using-queues-in-web-crawling-and-analysis-
infrastructure/)

\- Using Queues in a crawler to discover facebook, twitter, blogs on web
pages: [http://blog.databigbang.com/automated-discovery-of-blog-
feed...](http://blog.databigbang.com/automated-discovery-of-blog-feeds-and-
twitter-facebook-linkedin-accounts-connected-to-business-website/)

------
mekoka
Rather off-topic, but why is the module's name "Queue" and not "queue"? That
kind of things just rubs me the wrong way.

~~~
hoov
Historical reasons. It's been renamed in Python 3.

------
hoka
As someone learning python (on mostly windows, but I test on ubuntu now and
then to see if it works well) my advice is to just learn the multiprocessing
version as shown by hythloday.

I had a lot of problems with threads not exiting/joining properly, and even my
multiprocessing code was goofy on windows. I used the "poison pill" method at
[http://www.doughellmann.com/PyMOTW/multiprocessing/communica...](http://www.doughellmann.com/PyMOTW/multiprocessing/communication.html#multiprocessing-
queues) and everything works on ubuntu.

I'm aware of the __main__ thing for windows, but I don't think that's my
problem :-/

------
gsibble
This is indeed awesome and I've used it in several applications.

I forget though: does this use a single global interpreter or do the
additional threads use their own interpreters?

~~~
hythloday
Threads inherit the process's interpreter (which is how they share the GIL).
Contrast to multiprocessing, which spawns a new interpreter and communicates
with it via a python IPC, such as multiprocessing.Queue, which shares the
interface with queue.Queue or the slightly clumsy multiprocessing.Value class:

    
    
      from multiprocessing import Process, Value
    
      def f(n):
        n.value = 3.1415927
    
      num = Value('d', 0.0)
      p = Process(target=f, args=(num))
      p.start()
      p.join()
    
      print num.value # 3.1415927

~~~
Luyt
Ah, args should be a tuple:

    
    
      p = Process(target=f, args=(num,))

------
simgidacav
I heard that python concurrency model is based on userspace threads, so when I
saw the "setDaemon" call I investigated and learned[1] the difference between
daemon and non-daemon python thread (I hoped to see a different difference -
oh, a pun).

That said, why making daemonize a thread when we'll be joining it later?

[1]: [http://stackoverflow.com/questions/190010/daemon-threads-
exp...](http://stackoverflow.com/questions/190010/daemon-threads-explanation)

------
smegel
Do we need a HN front-page post now for every module in the Python standard
library?

~~~
salman89
If it makes it to the front page the it means enough people found it
insightful or helpful. I for one did, and when/if I want to do some concurrent
threading in Python, I know where to start.

