I'm doing a lot of python lately...but yesterday I compared one of my python lucene search programs with the java equivalent...and the Java one was considerably faster. Also, the fact that I could write the same program with Scala is enticing.
I feel like I am being drawn back to the JVM after pushing it away for so long. It really is an impressive system.
I wouldn't expect it ever. Rapid development not rapid execution. However, I didn't expect the difference in speed to be so large, considering it is calling into the Java lucene.
This might be overkill, but would it be possible to setup a mini-Java Lucene process to handle the indexing and queries, and talk to your Python process over a socket?
Like I said, overkill, but it could be worth it.
I say this being in the same boat: coming from the JVM to Python and looking back at some of the things I'm missing.
I feel like I am being drawn back to the JVM after pushing it away for so long. It really is an impressive system.