Ask YC:Python and Dual core Processors

[aitoehigie]

I used to use a single processor system and i noticed that i could not call up 2 instances of the Python interpreter. But on my dual core processor system, I can call up a maximum of 3 instances of the Python interpreter at any single time, after that, all i get is an error message. I would like an explanation on this, in the light of the Global Interpreter Lock (GIL) that Python has. Thanks.

[lilspikey]

That's strange. I can quite happily run multiple Python interpreters on my ageing (single core) 12"PB. Perhaps it was more of a memory usage issue?

[aitoehigie → lilspikey]

The old system was a 256MB ram with a Celeron M processor.

[aitoehigie]

I will really like to know how the Global Interpreter Lock really works in relation to single and dual core processors. Thanks

[swoodtke → aitoehigie]

The Global Interpreter Lock (GIL) is exactly what is seems to be: a single python interpreter only allows one thread to run a one time regardless of the number of CPUs or cores available.

This lock is per instance of the python interpreter. Multiple instances will have a corresponding number of locks.

There are a couple of ways to have python use multiple cores or CPUs.

The first is to run multiple python instances and have them communicate between themselves using IPC mechanisms (e.g. sockets, pipes, shared memory, etc.). The non-standard "processing" module implements this very cleanly using a similar interface to the standard "threading" module.

The second is to extend Python using C and explicitly release the GIL inside the C extension. This allows both the C extension code and the Python interpreter to run simultaneously. However, the GIL will need to be reacquired before the C extension can resume any interaction with the Python interpreter.

[Erwin]

What error message?

[aitoehigie → Erwin]

The error is "IDLE's subprocess didn't make connection. Either IDLE can't start a subprocess or personal firewall software is blocking the connection" I always ensure that the Python interpreter is part of my firewalls exceptions so it is never blocked.

[lilspikey → aitoehigie]

Ah, so that sounds more like a problem with IDLE than with Python itself.

You shouldn't have any issues running Python interpretors normally.

[Erwin → aitoehigie]

It's an IDLE IDE problem as lilspikey mentioned rather than Python itself. From what I can tell, IDLE listens on a fixed port on localhost in order to communicate with the actual Python interpreter, so starting multiple IDLEs may not work.

If you run multiple instances of just "python" from the shell (assuming UNIX), there is no problem.

GIL is not involved here, it only comes into play when you have multiple threads in a single process.

I don't use IDLE, but http://mail.python.org/pipermail/patches/2006-July/020329.ht... suggests there's some "-n" switch you can IDLE with that alleviates the problem somehow.

[schtog → aitoehigie]

I get this too sometimes but have never been able to figure out the problem.

It can happen after I have started and closed the interpeter and then start it again.