OP here - FYI - I was a software engineer at Novell over 20 years ago, then went to the business side. Last January for some crazy reason I decided to build a website. I couldn't write HTML last January, and though this is a pretty tame site by HN standards, I'm pretty proud of it. I've had to learn HTML, CSS, Javascript, Jquery, Angular, D3, Require, Node, Apache, Bootstrap, noSQL DB (Cloudant/Couch), linux system management - the works. It's been fun.
Interestingly the python experience has been incredibly different from 20 years ago. Back then I was hunting memory leaks in C. In python I program at a much higher and more efficient level. The Web side, however,is much more like the old days. Instead of memory leaks I hunt obtuse bugs in angular that make me want to pull my hair out. The biggest positive change is stack overflow, which is really so amazing. And the whole open source community - creating all these great frameworks, and even responding to the occasional plea for help.
Cool site! I wonder how it will look after another decade of climate change. (And ... I don't know Seattle except by reputation, but SF has at least a couple of very distinct microclimates; the charts for the Mission and the Marina are surely very different.)
I'm in the same programming boat as you. (In my case, 18 years.) From your experience, what are the top three programming/learning tips you'd give the year-ago you?
1. Start small - I was way too ambitious on the first part of the project.
2. Start with python- my back end data work is all python and it is SO much easier and more learning-efficient than the Web stack.
3. I'm torn on this one. I PROBABLY should have learned more rigorously before jumping in- with angular especially- but I personally find it much more motivating to learn with a specific problem in mind. Though my goal this year is to go back and learn comprehensively what I spot learned in process.
Well for your case you might as well have done your backend in Node since you didn't know Python and you had to learn JavaScript for the frontend anyway.
Pretty cool to see you using CouchDB for this. I've always wondered how building analytics with CouchDB would be. Do you have any thoughts on that experience?
On a somewhat tangential note, I've always been curious about sentences like this: "Contact me at ross underscore rosen at revelision dot com"
Are sentences like the one above an effective way to prevent spam? Or are most spambots sophisticated enough to account for slightly obscured spellings of email addresses?
Bravo! This resonates with me on a couple of levels.
While I'm still active in software development, I haven't done web development in 10+ years. I'm currently working on a side-project which is also teaching me "the works" when it comes to web development and deployment. Of course, I will still be a n00b by the time I'm done, but I'll soon have written and deployed my own RoR, data-driven website.
My side-project also shares a similar motivation as yours: First, it's my belief that most data isn't taken advantage of because most people suck at data. I wanted to see if I could take an existing, well-picked-through data set and extract value, just by sucking less. This could also describe my side project, which is in the sports domain.
You reaching this point is extremely inspiring. Thank you for sharing!
Very cool site...with your background, it shouldn't be too hard to grok the concepts that you used to build the site...it's just that there are a lot of them, and joining them together (efficiently, and attractively) takes experience...if this is your first (or fiftieth) web project, it's one to be proud of.
(that said, you probably could've gotten away with just HTML, CSS, JS, jQuery, D3, and Bootstrap, as the site could probably run off of flat static files...but even sussing out that architecture is its own skill)
Which tools/languages/techniques did you enjoy learning and employing the most? It all sounds fun, but some of it sounds more fun than others (for example, I really love Python's legibility, and am impressed by D3's power but find it to have a pretty steep learning curve).
Yes on both of those- I really love python and find myself way more productive in it. D3 is undoubtedly powerful and if you are an expert I'm sure you can be efficient at it. But as a novice it takes a while. I do my analysis onipython/ matplotlib and then implementa public chart using d3. And I havea love hate relationship with angular. When you are using stock or already made directives, it's magic. But it can take FOREVER to solve the obscure bugs you create when you are doing something new.
The D3.js google group is chock-full of excellent advice, especially around reusability etc. NVD3 is a great way to put D3 in play with less hand-coding, too.
Check out Flask! I am no Python expert, but Flask is an excellent micro framework that makes (especially simple web) app building a simple(r) process. It's also fun.
I thought about doing flask, because I love Python and it would be way easier than monkeying around with Angular. But most of my code is D3, so it's a lot of javascript anyway. And by doing angular, I basically upload one page to the user, and then from there out, they never touch my server. (Data is served by Cloudant - a YC comp). So even though I've had > 10k page views, my server is hardly touched. And everyone gets really responsive interaction since it is all done in the browser, occasionally waiting for new data to serve. So, while this was painful to build, I can really see the allure of client-side single-page apps.
Very cool. I like the way that you show 'means vs time' on the left panel and then you can dig into the actual distribution on the right panel. FYI, I think that, e.g., http://weather-explorer.com/history/country/US/state/WA/city... should read "Daily High Distribution", not "Average Daily High" or something. The mean trend line is on the left panel, and the right shows the the entire distribution. I'd be curious to see also what the 2nd and 3rd moments look like vs time, to see if the weather has an equal 'spread' month over month or if it tightens up for certain periods of the year.
Also, you need to drop in a full post with commentary, analogous to what you did with your "learning python" post. More feedback about tools, resources, learning sites, etc.
While you might have forgotten some programming, and quite a bit has changed over the last couple of decades, what is quite interesting to me is the efficacy of the result. Although I don't know what your word looked like 20 years ago, this probably shows that over the last couple of decades your sense of how to do something useful and present it has developed.
I would have guessed that a "what I did to catch up after 20 years of non-programming" project would have had a lousy UI (presentation), which yours does not. And while the presentation tools are better, the decision of what to present and how are still up the programmer.
Yes... something is wrong with sunshine data. Go to comparison tool and look at Miami and Key West. They're only 100 miles away and Miami shows significantly less sunshine... and shows very low overall sunshine which is clearly wrong.
Though I should point out there are definitely weird things with the raw NSA data. For instance, their precipitation data at one and 6 hour increments do not match at all.
Had to check that: it is NOAA data (http://www.ncdc.noaa.gov/oa/climate/isd/), not NSA data. The NSA tries hard to be a one way street; it reads a lot, but does not publish much.
