While I'm pretty sure that Go is much faster than python in nearly all tasks, I'm still skeptical that a) they built a full replacement of the system in only a week and b) that they didn't improve the algorithm in the go code. I work at a python/django shop and I recently updated a process that could only handle 5000 transactions in 5 minutes to be able to handle over 100,000 in just under 3 minutes. This was all in python would go been a lot faster? Sure but how much is the new code path and how much is the advantage of golang?
Also what to point out that I fully realize that Go is a lot faster than python at preforming the same task but this kind of analysis is dangerous because it leads inexperienced engineers to believe that changing languages is a good performance optimization which should usually be the last step/idea considered. It's expensive and dangerous to switch languages mid stream and while it's sometimes the exact cure it needs to be weighed very heavily to make sure it's the right situation. This article, at least the headline misses that piece of the puzzle pretty fully IMO.
Actually, I think "Go is faster" has two primary built-in points:
1. It's faster in terms of real performance - given its proximity to C, optimization at the compiler level for multiple platforms, that its highly concurrent and run compiled (usually), this is no shocker.
But,
2. It's faster than transitioning from a dynamic scripting language than it would be to transition to, say, C or C++ or Java, etc.
Not that it's syntactically similar (obviously), but that language, testing and deployment design won't present any egregious bottlenecks and that there's so much built into the core library that complements an http-based service such as this.
In other words, the prospect of going from Python (or Rails or PHP or whatever) to "faster language X" has historically imposed an inherent cost in terms of development that I think a language like Go mitigates to some degree by design.
I think you are talking about 'go run', which one might easily mistake for an interpreter because of the speed of the compiler, but it's actually compiling your code and running it. Were you talking about something else?
I wasn't talking about go run, no - there have been a few examples of running go interpreted I've seen posted here. They may well just be experiments but I left the "usually" as my caveat.
Edit: I swear there was an interpreter demo'd, but I may well have been thinking of something like agora, which interprets Go-like code within Go.
Not sure I agree with you there. IMO, porting between languages with similar abstraction levels is not that hard. I ported some Python code, whose logic took weeks of fine-tuning, to C# in less than a day. Go still has a garbage collector and its type system is similar in philosophy to Python's duck typing. Unless you are porting extra dynamic code (eval(), monkey-patching) or deeply intertwined with an unusual library, it should be a fast process. On the other hand, I was leaving Python get real OS threads, not to increase performance.
This obviously also depends heavily on the complexity of the application and the available resources.
Even so, a production-ready transition in a few weeks time doesn't leave a ton of time for benchmarking and testing, but as someone who operates on a fly-by-the-seat ethos more often than I should I'm in no position to be tossing stones around.
There has been articles talking about rewritting being less of a terrible idea if your writing a SaaS. Not to mention the throw one out, you will anyway idea. I'd be interested though, in hearing more about what they rewrote into Go. Its most probable its not all of their codebase.
Yes, it sounds like its a websocket server. A lot of that stuff is relatively low complexity if you glue everything together with a message queue since they can be made language agnostic very easily.
> I'm still skeptical that a) they built a full replacement of the system in only a week
Keep in mind that Highscalability bases it's articles on other 3rd party research. They don't talk to the people they are writing about, even when those people (like me) offer to help them get their facts right.
While I do certainly make mistakes I take great pains to be as accurate as I can. And I do have a lot of original interviews on the site, though some problems have kept me from doing them lately. I give the sources directly in the article and they are directly from the people involved in the project. And if there are any inaccuracies in any article I would be happy to make corrections. I don't recall when you offered to correct mistakes, but if you did so and I missed it then I apologize.
It sounds like this is a very contained server process that does one thing: pull things from rabbitmq and pushes that to nginx which does the push stream. So this could be written in a week easily.
To be fair, there were some optimizations that were made as well. Everyone does that, right? :)
But overall, specifically our realtime service is a hybrid of CPU intensive tasks + lots of network IO. gevent was handling the network io without an issue, but at higher contention, the CPU was choking everything. Switching over to Go removed that contention for us, which was the primary issue that we were seeing.
This specific use case is multiplexing messages to lots of connections, which is almost the ideal use case of Go anyways. It wasn't a matter of re-implementing complex business logic. 1 week is reasonable given the specific problem.
Go is looking more attractive, but the language is not just the language.
When Go has available a significant fraction (subjectively) of the "batteries included" that Python has, then I'll start investing time in Go. Is it there yet (subjectively)?
It's definitely getting there. I'd actually say that Go's network "batteries" are just about as comprehensive and possibly easier to use than Python's (ie, socket, urllib, urllib2, httplib, smtplib, pop3lib, etc.).
It's also pretty cool that Go has a full crypto library in written in Go.
I was seriously looking at Go for a desktop app using the promising new go-qml, but there seems to be no elegant way to use Qt's system tray feature with go-qml. I'd say Go still has a way to go before it's got as many possibilities as Python for desktop GUI development.
But for command line utilities and network apps, I'll definitely reach for Go over Python now (although I'm increasingly playing around with Rust and hoping I can eventually just use Rust for just about everything).
Edit: Removed imaplib since Go has no equivalent in its stdlib.
I think Go is pretty competitive with Python's standard library. It doesn't have an FTP client (we use and are happy with github.com/jlaffaye/ftp) or an equivalent to imaplib (although github.com/mxk/go-imap/imap looks interesting), but for the rest of our workload Go has everything we need.
The Go standard library is pretty amazing as far as standard libraries go (I'd say it's more useful than python's, and definitely contains less cruft). Community provided libraries are expanding rapidly, but there is a strong focus on web services all around.
Thanks to all replies. It looks like for work I could use it for personal automation/productivity, but it would be harder to convince others in the department to use anything I might write that depends on newish/non-established third party libs. I work for BigCo. I'm using xlrd, for example, in a few python scripts, and even that's going to raise an eyebrow or two.
As for home, I'm playing around with imap and some related things at the moment, and pointers to 3rd party libs might be fun for me to try. We'll see.
Their asinine policy against package managers (yes, let's go BACKWARD. Everything was better in the good 'ol days!) is keeping me away from it. I don't have time to deal with more self-aggrandizing douche bags.
For a start, Go's SSL/TLS doesn't disable certificate checking by default ;-). Does that mean it's more "batteries included" than Python for TLS and every TLS-using protocol?
This part is telling to me. It means their codebase isn't so complex, and optimizing their application for their hardware would take as much time (or less) than adding hardware. For people with large, complex codebases, scaling horizontally increases capacity much faster.
In the past Go has had trouble with garbage collection lag due to a global mark and sweep implementation[0]. Was this not an issue with Disqus's new implementation?
It seems that this is what the found team was most comfortable with, so it makes sense that they proceeded to solve problems using tools they already knew well. At some point, they exhausted how far they could take their existing tools and started investing into new tools.
Given it sounds like this application is not "web facing" (i.e. not an API nor rendering HTML), the use of any "web framework" or Gorilla doesn't make much sense.
For me, it felt comfortable writing Go, considering I've been mostly writing Python for the past 8 years. So the transition was a lot more natural, compared to using Scala or Erlang or anything else.
Again, this is my subjective opinion, and this is why we chose to use Go for our new stuff instead of something different.
