
Dark Sky - Weather Prediction, Reinvented - huhtenberg
http://www.kickstarter.com/projects/jackadam/dark-sky-hyperlocal-weather-prediction-and-visuali?
======
_delirium
This is pretty neat!

I experimented with something like this in grad school to tell me when it was
"safe" to walk home, by scraping the past hour's NOAA radar images, coursely
gridding them as a poor man's way of reducing computational needs and noise,
and throwing them into some regression algorithms to get a prediction of rain
probability for [0,10] mins, [10,20] mins, etc. out to an hour.

It wasn't useless, but I gave up on it when it became clear that some
significant computer-vision work was needed to get good accuracy, mainly
object-tracking algorithms: correlating when a blob in one radar image is "the
same" as a blob in a previous radar image, even though the blobs change
shape/size semi-rapidly (and split and merge), in order to extract motion
vectors. At least with the regression techniques I tried, it doesn't seem that
they were able to implicitly extract the motion information from just the
sequence of images as input data, because prediction was _much_ better in a
small set of images I manually labeled with motion vectors. Hopefully they
have solutions for that!

In any case, I'd definitely use an app like this. I often predict short-term
weather by pulling up one of those radar animations and extrapolating the
motion of a storm in terms of "finger widths on monitor per 5 frames of
animation" or something, which feels like a not-very-21st-century way of doing
it.

~~~
tel
Can you even get away with object tracking? It always seemed to me that there
were a lot of latent phenomena you'd need to model in order to get the
dynamics of cloud motion. There's wind producing a line linear drift, but also
a lot of pressure and humidity interaction.

~~~
_delirium
If you want a real physical simulation, there's tons of stuff yeah, but on a
short term you can find considerable statistical regularities just by looking
at how things move, while being agnostic about why they move that way. Over a
30-minute period, storm tracks tend to be semi-regular (though varies by
region); if a storm's been moving ESE at 3 mph, and you have a bunch of
historical data, you can come up with a decent predictive distribution over
its likely heading/speed for the next 30 minutes. The uncertainty in the
distribution is basically where all the un-modeled physical phenomena fall. If
uncertainty is too high, then you can try to add in more complex models, or
additional data (e.g. wind readings at various points, in addition to radar
images).

~~~
tel
It's always seemed to me that weather clouds don't seem to move as much as
evolve directionally. That's where this intuition is coming from, though I
suppose I'm not honestly surprised to find that gross motion is predictive as
well.

------
brudgers
I wonder in all sincerity, how will it compete with looking out the window and
making your own educated guess? Or looking at the NOAA radar hourly
precipitation loop, as I do, and planning accordingly?

Because what appears to be missing is any sort of feedback mechanism by which
to improve the accuracy of it's predictions on the same scale as which it is
making its predictions.

In other words, if the prediction was rain in 8 minutes for 15 minutes, how
can the program determine if it was accurate? (And suggests the question, is
rain in 4 minutes for 18 minutes an acceptable level of inaccuracy?). Where
does the data that it was 18 minutes of rain come from?

~~~
cstuder
Concerning your first question: This is largely a topological problem: If
you're surrounded by mountains, the weather behaves distinctly different than
on the open plain. In the second case, high wind speeds lead to rapid changes
in conditions. For that scenario, such an app might give you some useful
information.

I, myself, happen to work in a meteorological company and have an easy
indicator: When our meteorologists all take off at the same time, rain is
imminent.

~~~
pixcavator
>>This is largely a topological problem: If you're surrounded by mountains...

Topological or topographical? It's kinda both in your example.

~~~
cstuder
Topographical was the word I was looking for.

I'm no expert, but I don't think topology features much in the weather models.
It's mostly numerical analysis and statistics.

------
kablamo
Do they really need $35,000 to do this? It seems like so many people produce
apps first and get paid later, I'm not sure what makes their situation
different.

Thinking about this more, its brilliant that they put their project on
kickstarter. It didn't click for me that people are using kickstarter this
way. They get some extra marketing and some extra $. Interesting also that
they aren't afraid of someone taking their idea because they have a head
start.

Good moves.

~~~
thegrossman
We've been working on the app for a while, but our day jobs always came first.
We eventually realized that without a source of funding, it wasn't going to
happen any time soon. And we're completely in love with the idea of
Kickstarter, so we figured we'd start there and only approach more
conventional sources of funding as plan B.

~~~
duck
How did you arrive at $35,000?

~~~
thegrossman
With a lot of hand-waving.

We figured out how much we needed to survive for 3 - 4 months, plus the cost
servers for a certain amount of time, then factored in Kickstarter / Amazon's
cut and the cost of the backer rewards.

We kept getting a number that was too high. So we eventually decided "Screw
it! $35,000 it is!". It's still probably too high... but we wanted to avoid,
at all costs, asking for too little money and then not being able to deliver
to our backers.

------
mark242
This is really interesting-- Formula 1 has systems like this whenever they go
to a track; whenever there's the chance of rain, you always see television
shots of the local weather radar, combined with "Rain expected at 3:30 local
time, intensity 1" overlaid on the map. Teams use this information to
formulate their strategies-- do we bring the car in to switch to wet weather
tires around 3:30, or do we wait to see what other teams are doing? It's
pretty invaluable input to how races are run on days that are "mixed
conditions".

This reminds me of that, only a much better display.

~~~
kgen
To be fair, even the expensive weather tracking systems they use get their
fair share of the calls wrong and they employ watchers at different parts of
the track (down the back straights, at the corners of the track, themselves at
the pit straight, etc) to get a better idea. And that's over a relatively
small area of space too.

------
shahidhussain
There's definitely a use case for this. It's awesome.

However, I wonder if it makes more sense to build the business around
providing the data on the backend rather than creating a beautiful app.

If the point is this to let users know whether they're about to get rained on,
all they need is a bit of text to get the message, rather than a beautiful
flowy animation. It looks awesome - I just wonder if it's just going to get in
the way after the 10th time checking it. A home screen widget would be ideal
(which is why there's lots of weather app widgets already.)

~~~
StringyBob
Combine it with Siri and google map searches: "Where can I walk to for lunch
and not get wet?" - I'd buy that!

------
gregable
For android, I've been using Rainy Days:
[https://market.android.com/details?id=com.neenbedankt.rainyd...](https://market.android.com/details?id=com.neenbedankt.rainydays)

It only offers the live radar map animation, based on your GPS location, but
that's often pretty good for eyeballing when the rain is going to hit you, or
when you are going to hit it if you are moving. Free, displays ads.

------
huhtenberg
I think this is a _really_ clever and unexpected take on weather prediction. A
real case of _Think differently_ , an innovation through lateral thinking.

Got a spare fiver? Let them know you agree :)

~~~
vijayr
This is a very very clever idea, and very useful too.

Why do they need so much money? for servers?

~~~
thegrossman
One of the Dark Sky guys here...

So yeah, we need money for servers, and to pay for rent / food / etc for the
several months it'll take us to make this thing.

When you start doing the math, a lot of the money we're asking for disappears.
10% of what we raise goes to Kickstarter and Amazon (for payment processing).
Another 30% - $35% goes to paying for the Kickstarter rewards, and then the
government takes it's cut. What's left is a minority of the money.

~~~
jhamburger
Sorry if I missed something from the page but how do we know this really
works?

~~~
thegrossman
Hah! You don't. ;-)

We'll be writing more about how it works for our fellow nerds soon.

------
silentscope
If this was combined with GPS and push notifications, it could resonate with
an awful lot of folks who would appreciate the heads-up. I can only imagine
what something like this could do if with was plugged into something like
Siri.

Talk about a personal assistant--it could remind you to bring an umbrella, or
tell you to take an alternate route because precipitation 5 minutes from now
might lead to delays on your regular commute, or change the way we learn about
and deal with flight delays.

Of course, this is predicated on its ability to accurately predict the
weather. I took a lot of meteorology in college and I don't think it's outside
the realm of possibility, especially when coupled with historic data of past
weather trends for specific locations.

Only one way to find out.

------
hopeless
There's no suggestion that the app will have a subscription cost and yet their
data and processing costs are going to continue monthly. It's like a high-tech
ponzi scheme. I just don't think it's sustainable so for that reason I
couldn't back the project :(

~~~
thegrossman
Our current thinking is that we're going to charge for API access to our data.
But I must admit, we don't have a well thought out business plan beyond "make
the best f-ing weather app and charge a decent amount for it".

~~~
hopeless
Sorry if I came across too negative. I've just got very militant about
investing my time/energy/money in things which don't make financial sense.

I was also building an app based on weather data do I have some idea of the
expenses involved and I quickly realised that my (admittedly niche) product
couldn't be supported by small once-off payments like the app store.

------
Urgo
Nothing really too remarkable here though I do like the smooth radar map.
Android already has a percent chance of rain graph built into it that though
it wont say "rain will start in 5 minutes and last 15" you can see this info
and predict it yourself. When I want more I love the graphs found here:
[http://forecast.weather.gov/MapClick.php?lat=35.82200&lo...](http://forecast.weather.gov/MapClick.php?lat=35.82200&lon=-78.65880&unit=0&lg=english&FcstType=graphical)

Anyway I guess I'm just spoiled with this type of info already built into
android so guess its needed on the iphone which explains why its an iphone
exclusive.

~~~
w1ntermute
> which explains why its an iphone exclusive

No, that's because your average developer is not using the same platform as
your average smartphone user.

------
stefanu
In Slovakia we have been using similar feature for couple of years now. It is
far away from being that pretty and interactive. It serves it's purpose for
knowing how the weather will be evolving during the day:
[http://www.shmu.sk/sk/?page=1&id=meteo_num_mgram](http://www.shmu.sk/sk/?page=1&id=meteo_num_mgram)
Charts from top to bottom: temperature (in Celsius), clouds (total, red-lower,
green-middle, blue-upper), precipitation, atmospheric pressure, wind speed and
last is wind direction.

Very useful for example when I want to go inline skating on the Danube dike in
Bratislava: how much time do I have until it starts raining? :-)

------
100k
This looks interesting. I'd like to hear more about how it works (they say
details are coming, so I look forward to that).

I get wanting to keep the app simple, but I have to say the screen shots are
crying out for the current temperature up in the header, next to the time.
After all, you need to know if it's going to rain AND how warm it is to
prepare to go outdoors.

------
joelhaasnoot
Here in the Netherlands we have <http://www.buienradar.nl/>, which does this
basically, and there's an Android app that gives you alerts when it's about
the rain <http://www.yoki.org/android-development/buienalarm/>

~~~
StringyBob
Similarly for the UK: <http://www.raintoday.co.uk/> although no app of which I
am aware.

~~~
mseebach
Rainwatch for Android is a thin layer around the precipitation radar images
from BBC.

<http://news.bbc.co.uk/weather/forecast/8?area=London>

------
ck2
Don't local news stations spend millions of dollars on fancy doppler radar
units just to get this kind of info (and fail)?

~~~
thegrossman
Nearly all local new stations use the same source of radar data as we (at Dark
Sky) do: The NOAA NEXRAD network: <http://radar.weather.gov/>

------
techiferous
Great design. Love the smooth radar animation.

The probability of precipitation (red bar beneath) seems lacking. I think just
sticking with the green bar (amount of precipitation) and varying the opacity
according to the probability of precipitation would be more intuitive.

------
sp332
And check out the story under their first update:
[http://www.kickstarter.com/projects/jackadam/dark-sky-
hyperl...](http://www.kickstarter.com/projects/jackadam/dark-sky-hyperlocal-
weather-prediction-and-visuali/posts)

------
xinsight
My solution to the problem of studying the animated radar images was to ditch
the VCR metaphor to control the animation and use the mouse or finger
position. Example (Toronto):

<http://radar.xinsight.ca/>

~~~
mambodog
In the context of audio or video this is usually called "scrubbing".

------
Brashman
How is Dark Sky able to predict weather on such short timescales? My
impression has always been that weather prediction is fairly imprecise and so
it only make sense in larger time scales (i.e. it will rain today, it will
rain tonight).

~~~
thegrossman
We're going to write a blog post that goes into the technical details soon.

Basically, we rely on the fact that while weather is chaotic and turbulent in
the long run (by "long run" I mean hours to days), it's fairly linear on the
timescale of minutes.

So we're able to extract velocity data from NOAA radar images, and use that to
project the storms into the future. Some storms are more coherent than others
so we're constantly monitoring how accurate our predictions are and adjusting
the projections accordingly.

Sometimes this technique breaks down, but it's surprisingly rare.

~~~
Brashman
Cool, sounds like you guys have done your homework. Will be neat to see the
blog post.

------
davidw
Awesome! I'd always wanted to do something like that with a site of mine,
<http://meteo-veneto.net/> but simply don't have the skills to do the
prediction part of it very well.

~~~
thegrossman
Learn the skills!

We're not meteorologists, and we're not even fancy-pants computer science
academic types. We just got sick of this app not existing...

So we hunkered down and learned about Computer Vision and experimented a whole
lot (which resulted in another fun project: <http://tinyfaceapp.com>). There
are so many awesome open-source data-processing libraries out there, which
when combined with cloud-based computing, gives individuals so much tremendous
power to pursue their weird passions.

~~~
davidw
I played around with OpenCV and the data a while ago, but in the end decided
that for the time being it's probably not worth it. 1) I only have data for
the Veneto, which is not that big 2) the data can be intermittent - if people
start relying on you, and your data dries up for a few hours, they're going to
be pissed off 3) storm prediction algorithms are much more complicated than
just tracking the image. I looked up a bit of the literature on it, and...
"it's complicated" if you want to really try for accuracy.

------
joezydeco
I'm interested, just to see existing products like AccuWeather's Predictive
Radar get more open to other clients.

<http://www.accuweather.com/radarplus-benefits.asp>

------
dirkdeman
Here in The Netherlands we have buienradar(<http://www.buienradar.nl>), or
drizzle radar. Works like a charm for short term rain predictions in our wet
little country!

------
viraptor
Not sure how useful is it as a separate app, but give me notifications about
possible rain 30 minutes before a planned event, or whenever I'm more than X
km from home, or just leaving and I'll be very very happy.

------
amjith
This is an awesome idea.

Scenario where it's useful: I walked to the grocery store and it's raining
outside. Should I wait it out by staying inside or is it going to continue for
more than 30mins?

~~~
thegrossman
The whole idea from Dark Sky came when I was stuck at a rest area off
Interstate-90 in western New York. It started to downpour and I had no idea
how long I'd have to wait if I wanted to stay dry.

Even looking at the radar map online didn't help. It was nearly impossible to
extrapolate mentally the stuttery radar animation.

------
thegrossman
As promised, here's more info on how Dark Sky works:

<http://blog.jackadam.net/2011/how-dark-sky-works/>

------
peterb
This is very similar to SkyMotion's RainCheck: <http://skymotion.mobi/>

This is a nice little mobile app that is location aware.

------
jrockway
This sounds like a fun weekend project. Where is a good source of raw radar
reflectivity data?

~~~
sukuriant
from elsewhere in the comment stream:
<http://news.ycombinator.com/item?id=3188562>

------
crtv
why do apple devs like to toss "reinvented" so lightly?

on the topic: pretty neat. depends on how accurate it is though. Hope European
version will be released to play around with.

------
fuzionmonkey
Haha, I noticed the location was Troy, NY.

RPI, huh?

------
HilbertSpace
Come ON guys: It looks like you are missing it! It looks like you could go
live, start getting okay revenue, 'bootstrapping', and doing 'organic' growth
NOW.

Notice something really special about your application and your users: You are
LOCAL, geographically LOCAL.

So, if you want to get users in, say, Boston, then it's essentially irrelevant
what you are doing in San Francisco, New York City, Miami, etc.

Sooooooo, to get started with only a small, tiny, cheap server farm, get
started in just ONE city.

When you have some good 'traction' and/or revenue from that one city, expand
your server farm, etc. to another city, say, an adjacent city.

Then grow across the country and world this way.

Besides, your ad revenue will be similarly local.

So, you don't have to bite off the whole country in one bite to be successful
and, instead, can do well one city at a time.

Also, this fact gives you a 'geographical barrier to entry' since when you are
in Boston you have no competition from a company in San Francisco or Miami.

------
ldar15
"Just do it already"

Er. "Fuck you asshole".

Not the response you were going for I expect, but nonetheless my automatic
response.

