

Ask HN: Python vs Scala for a fast web-project - ajakhu

I was working on a web-project lately with a friend. We faced a situation where we realized that Python was being a bit slower than expected. Also, we were planning on using a database that was not native to Python (Python bindings used its REST interface).<p>So, we had to make a decision to either keep going in Python or port to Scala (Scala seemed apt for many reasons).<p>What would you have done in this case?
======
gelnior
If you are comfortable with Scala and think it fits well for your project go
for it. There are some frameworks like Lift or Play with which you can start
quickly a web app.

~~~
ajakhu
Yes, we were planning for Scala/Lift only.

------
playing_colours
Did you have any experience doing web development with Scala? I am now
choosing stack for my new project and considered Scala/Play as an option.
There are some problems both with Play2 framework atm(no Wax deployment
support, long rebuilding after small changes, work with actions is confusing)
and the language of Scala (complexity, version migration, SBT etc.) The
situation is promised to get better in future but most need their work done
now. If you are choosing bleeding edge, new shiny thing be ready to face
additional problems which can slow your development. If you prefer just to get
things done you'd review your Python code, find and fix bottlenecks. Maybe you
can combine techs choosing what is appropriate for some work - say Python for
web interface with critical parts in Java/Scala.

~~~
ajakhu
No, we didn't have any experience with Scala/Lift/Play. We were aware of the
learning curve of scala. But what we thought was in the the best interest of
project. I've read about the problems ingoing with Scala, but the benefits
outweigh the problems most of the times. And yes, learning a new technology
will definitely slow down the development (especially Scala).

------
manishsharan
Its not the language but your framework and architecture and your own
proficiency in the language ( whatever that might be) that matters most for a
fast web application.

Your code would spend most of its time fetching stuff from DB, external web
services etc and rendering HTML. Does your framework allow you to easily
incorporate database caches? Does it allow to to cache frequently rendered
HTML segments ? Can you offload certain tasks ( such as sending email etc.) to
a secondary process via a job queue ?

You could write a web application in C that could still suck big time unless
you get the application architecture right.

~~~
ajakhu
We had integrated redis in our project for server side caching. And I'd like
to clarify that Python was being slow for regular cases (atleast not as fast
as we wanted it to be).

As we were newbies to web development (although we have a lot of experience in
development), we had put in our best knowledge while designing the
architecture. We kept independent things independent. That's why we chose
flask, it gave us the flexibility we wanted. We were free to choose the
extensions we wanted to integrate.

------
gaius
First profile the Python app - is it being slow _in Python_ or is it waiting
on something else? If it is being slow in Python, can you rewrite the critical
section in C? And speaking of which, is there a C API to your DB that you can
use from Python? The Python mentality has always been that it is easy to
interface Python and C where necessary for performance.

~~~
pestaa
Or better yet, have a look at Cython or PyPy. Both give a performance
advantage over the standard intepreter.

~~~
ajakhu
The API problem will still exist even if we switch to PyPy or Cython.

------
petervandijck
If your goal is to launch, stop fooling around with languages and rewriting
your code. What exactly was slow?

~~~
ajakhu
1) A NoSQL database best suited our project. So, the non-nativeness was a
hurdle. Neo4j bindings for Python used JPype, which was rather unmaintained.
And I read at some places that it was a bad idea to mix python-java
(especially using the unmaintained JPype).

2) Using Flask gave us so much flexibility as a framework. SQLAlchemy was
probably the best thing that I've ever seen. But the work was made complex by
the fact that Python was not able to give us the performance that we wanted.
We tested the web-project running locally, and we noticed some exciting
results. Even though we had done proper memory management from our side, the
memory consumption was a little high than we had actually expected. I'm not an
expert web-technologies, but as a programmer for last n years, I've learnt a
lot about memory management. We projected that when the load would be actual,
python performance would suffer.

3) We wanted a real-time performance application so that the user experience
was the best. We implemented a part of the web-application and tested it. The
results were unsatisfactory. Then we planned on shifting to scala.

