A few weeks ago I was working on a small wildfire smoke and fire perimeter API, and I hit a few annoying snags due to tooling issues. I needed to process a lot of different formats of layered geographic datasets, and converting one thing to another, processing the data into various buckets, cleaning, aggregating, etc. all wound up being extremely cumbersome and verbose.
I write a lot of Go and I’m used to that. But when I hit snags in the less familiar territory of geographic data processing, it was a slog. Terrible documentation for the libraries was a major barrier, and otherwise it seemed as though essential features simply didn’t exist and I’d have to invent them.
I got the idea to explore Python for the project because people use it for data processing. I’ve used it in the past, though never for this. Whatever, I thought, at the very least I can validate that Go is a suitable tool.
Within a day I had rebuilt everything with Python. I built a flask app around the forecasting and fire perimeter tools, and had it deployed the same evening. It was mind blowing.
As an ecosystem I was absolutely blown away by Python. Do I like the language? Not really. I encountered so many cases where something could be so much faster and more efficient with Go. Deployment would be easier. I’d get more API from the same resources. Scaling would be ten times easier. Static typing tools kept blowing up my IDE because this library doesn’t support this, or the type tool is wrong about that. It was very janky at times.
Yet Python got it done. It’s live right now, and development is steady. Go was not steady and I didn’t see any solution to that in sight without reinventing countless wheels.
By the way, another nice thing about python is that you can force your program to drop into a debugger with an interactive shell at runtime- and inspect/print objects. In today's complex world of generic types, it can often be hard to see exactly which method is handling your data.
My favourite things about Python are the huge community and the rapid iteration. Don't got me wrong, I like Go, too. It was the primary language I worked in at my previous job. But sometimes you just need the big community and the huge pool of documentation, or you want something to work now instead of fiddling with it, and Python is great for that feedback loop.
I write a lot of Go and I’m used to that. But when I hit snags in the less familiar territory of geographic data processing, it was a slog. Terrible documentation for the libraries was a major barrier, and otherwise it seemed as though essential features simply didn’t exist and I’d have to invent them.
I got the idea to explore Python for the project because people use it for data processing. I’ve used it in the past, though never for this. Whatever, I thought, at the very least I can validate that Go is a suitable tool.
Within a day I had rebuilt everything with Python. I built a flask app around the forecasting and fire perimeter tools, and had it deployed the same evening. It was mind blowing.
As an ecosystem I was absolutely blown away by Python. Do I like the language? Not really. I encountered so many cases where something could be so much faster and more efficient with Go. Deployment would be easier. I’d get more API from the same resources. Scaling would be ten times easier. Static typing tools kept blowing up my IDE because this library doesn’t support this, or the type tool is wrong about that. It was very janky at times.
Yet Python got it done. It’s live right now, and development is steady. Go was not steady and I didn’t see any solution to that in sight without reinventing countless wheels.