
Show HN: All Clear Weather – creating new live weather datasets with ML - jacobsheehy
https://www.allclearweather.com/
======
19690401
Congratulations, this is wonderful! I'll be eagerly waiting for a
European/world version.

~~~
jacobsheehy
Thank you!!

------
matt_the_bass
Discussion from original Show HN 4 days ago:
[https://news.ycombinator.com/item?id=18205169](https://news.ycombinator.com/item?id=18205169)

~~~
jacobsheehy
Didn't have the website up then, was too early sorry about that! Thanks for
your feedback!

~~~
matt_the_bass
Awesome! Good work getting it launched. Good luck!

------
reckless
Are you also planning on open sourcing thr algorithms you're using to process
the data? I'd be interested in helping out on the ML side for any sensor
fusion or image classification.

~~~
jacobsheehy
Yes that is part of the plan! I will send you a message when the time gets
closer. Thanks!

Edit: or you can email me to stay in touch, jacob@allclearweather.com.

Cheers!

------
simon1573
Looking forward to a global release! Best of lucks :-)

------
jacobsheehy
Here is some context. This is a hobby project.

I make All Clear Weather (Android, US-only for now, see below for why) because
I demand better of weather apps. Many of the top weather apps on Android do
nothing but display sometimes-outdated weather data in poorly formatted
tables. Frankly the top 20 look exactly the same! I'm making a new weather app
because it's 2018 and the category is stagnant. Here is what I am passionately
dedicated to bringing to a weather app to make it interesting and useful at
the same time:

1) Your phone probably has a barometer in it! Most Android phones do because
Google likes to use it as an altimeter to give you better indoor directions.
So with iPhones also having barometers (for the iPhone health app), there are
now billions of phones around the world are carrying barometers that next to
no weather apps are using.

With All Clear you can see the sensor data from the weather sensors in your
phone and know that it is being analyzed and processed into 'virtual weather
stations' so it can be eventually included in the main weather models. You'll
know this because you'll be able to see it happen in the app as I make
progress. Here is a quick graph I made:
[https://imgur.com/gallery/IPj6w7N](https://imgur.com/gallery/IPj6w7N) of
Hurricane Florence's pressure recorded by phones in North Carolina a few weeks
ago. It may take some imagination to see how the data could be useful (it
needs cleaning, privacy protection, quality control, etc) but you can see the
data in there and I know it can be extracted into useful numerical weather
model inputs.

2) All the phones in the world have cameras. These cameras are taking photos
of the sky and the weather all day long but nobody is using this data in
weather forecasting. That's obviously because it is difficult to extract
meaningful numeric weather data from a photo, but I believe it can be done.
All Clear has a feature to let users send in photos of the sky with tags, and
I am going to use this data as a training data set to a machine learning
classifier. If it works, it should be able to automatically tag weather
information in any outdoor photo - assuming the training dataset is good and
my ML is good.

3) Current conditions in a lot of weather apps are out of date, since the
source data is out-of-date. A lot of the US MADIS stations update only hourly,
and they can miss updating in intervals when significant weather happens.
Sometimes they update and sometimes they don't. But it's so frustrating to
look outside and see it is raining while your weather app just happily denies
the existence of that rain and tells you it's just partly cloudy. So I let you
fix it, for yourself and for others nearby. If enough people all submit the
same weather condition, contrary to the station reports, the app will start
showing that condition as the Truth for current conditions in that area.
(safeguards built in to protect against cheating).

4) I am dedicated to open source. While the whole app isn't open source yet
(it will be!) the main sensor code that does the barometric pressure data
collection along with other sensors, is open source on GitHub:
[https://github.com/JacobSheehy/AllClearSensorLibrary](https://github.com/JacobSheehy/AllClearSensorLibrary).
I would encourage other weather app developers to enable features in their
apps to aide in the crowdsourcing of potentially very useful weather data that
is otherwise idle in phones.

5) It's in the details. The US NOAA Forecast API that the app uses only
returns its text forecasts with F and mi built-in as units. I wrote an
algorithm (regexes mostly) to convert the text forecast units on-the-fly so
you can be reading it and see it as C and km if you like, it's seamless. Well
there is one error I know about, sometimes it will say "quarter of an inch of
rain" which is a bit of a pain to parse, but I'm working on that too!

\-----

Alright, there's what All Clear can do that other weather apps can't.

And a note about it being US-only. I tried to make this international from the
get-go, and wasted months using IBM's API from WU that barely worked at all.
Then they yanked the payment form off the web the week I was going to publish
the app and buy their $400/month package. No longer an option. Now you have to
wait for a phone call, might be weeks, and the prices are like $3,000 for the
equivalent data (or something, hard to say). If you want to read some of the
drama about this, check out this WeatherUnderground thread:
[https://apicommunity.wunderground.com/weatherapi/topics/weat...](https://apicommunity.wunderground.com/weatherapi/topics/weather-
underground-api-changes) where people are describing the confusion as WU cuts
off their keys and makes the service dramatically more expensive.

So the app is using the US NOAA data for free and I am investigating
alternatives for international weather data.

Here is All Clear Weather on Google Play (US):
[https://play.google.com/store/apps/details?id=com.allclearwe...](https://play.google.com/store/apps/details?id=com.allclearweather.android)

~~~
draugadrotten
The idea to use crowdsourced data and sensors is great. Using pictures I can
see as very...risky.

Are you prepared to handle pictures that are inappropriate (non-weather) or
even illegal (perhaps of military locations or underage) You can compare for
example with the Strava app [https://techcrunch.com/2018/01/28/strava-exposes-
military-ba...](https://techcrunch.com/2018/01/28/strava-exposes-military-
bases/?guccounter=1)

Going to Europe pictures could also make any picture with humans in them
subject to GDPR requests and that may not be what you want either.

Thoughts?

~~~
chrismeller
Photos of the actual user would, of course, be subject to all the other GDPR
requirements around that user's data, but random other people that happen to
be in those photos should have no impact on the GDPR scope unless you're
attempting to identify them. From [1] on consent:

"Recital 51 notes that photographs will qualify as biometric data only when
they are processed “through a specific technical means allowing the unique
identification or authentication of a natural person.”"

So Facebook trying to match your face to tag you in a photo counts, but the
usage in this weather app doesn't appear to.

[1]: [https://iapp.org/news/a/top-10-operational-impacts-of-the-
gd...](https://iapp.org/news/a/top-10-operational-impacts-of-the-gdpr-
part-3-consent/)

~~~
uberneo
How about if you are just identifying that whether its a Male or Female and
their ages without uniquely identifying that it was 'John Doe'.. Does GDPR
will still have problems with that ?

~~~
chrismeller
I don't believe that would fall afoul under this particular section, but I
know there are specific sections that deal with anything regarding gender,
ethnicity, sexual preference, religious affiliation, etc. that you might have
to worry about. I'm pretty sure you could still do it without any kind of
consent (which is what the case here was related to), but the gender data you
collect from it might have restrictions on use or other caveats you'd have to
deal with.

Honestly I'd have to do some specific research and then probably contact a
lawyer to have them do some research just to CYA (or rather, mine).

