
Machine learning can boost the value of wind energy - andreamichi
https://deepmind.com/blog/machine-learning-can-boost-value-wind-energy/
======
dmichulke
I have been working in the same field, quite a few years in my own forecasting
company and then later on as a side project.

Not wanting to steal the thunder of Deepmind, but I feel should mention my
site which is currently beta as in "works for a few electricity power load and
supply forecasting scenarios" and quite well so. Also, it's out there, cheap
and fast.

[https://ausblick.cryptoport.net](https://ausblick.cryptoport.net)

Unfortunately, it's only in German but the basic rule is:

\- upload an excel (or csv) file

\- wait a minute or so (for a typical dataset of say 3 years of hourly data)

\- get back the results

There are a few design rules how the excel file should look like, e.g.,

\- a date/time series per sheet (it accepts most English and European formats)
and as many number series as you want (always use a decimal point instead of
comma here)

\- one series has to have consecutive missing numbers at the end, this will be
the target value

Other than that, it should just work.

Feel free to comment.

PS: The examples might be the best explanation -
[https://ausblick.cryptoport.net/examples](https://ausblick.cryptoport.net/examples)

~~~
linkingday
What prediction algorithm are you using for this? Not sure if it's mentioned
elsewhere on the site

~~~
dmichulke
No, it's not mentioned. It's currently a mix of very simple techniques

\- a number of smart features (usually a few k) depending on the series (using
lags, aggregates, curve fits, combinations of features, ...)

\- an iterative algorithm that selects features using maximum relevance (~
correlation with the target) / minimum redundancy and adds them to the model

\- simple pca and ridge regression (because it's fast)

\- a few optimizations of the final model (removing features, selecting a
better ridge regression alpha with CV, ...)

The stack is pure Clojure / Clojurescript.

~~~
profunctor
How did you find developing your models in clojure? I would love to switch
away from python for ml but it seems to just have the ecosystem.

~~~
dmichulke
Actually, I use both but my production models are in Clojure.

I often end up implementing minor things myself using lower level abstractions
(e.g., Linear Regressions or PCA with whitening using Matrix libraries) and I
check the results and/or try new things using scikit-learn.

So in general, I'd say I do the programming (outputing intermediate CSVs,
tests, web service, thread handling, UI, ...) in Clojure(Script), and try
other approaches (e.g., other models/parameters/...) in Python.

I'm quite happy with this pipeline but probably to some extent because I
really love to understand how things work and nothing pushes you to learn as
much as a missing function in your ML library :-)

------
Mizza
No paper, no code, no comparison to other models - hell, they didn't even
label the graphs. This is just Google PR spam.

~~~
giacaglia
The optimization for their datacenters used a 3 layer neural network with 6
neurons. I wonder if this is much more complicated than that...

~~~
sgillen
You can still do cutting edge work with 3 small layers, for example most of
the advances in robotic control (using neural networks) use networks with
similar sizes.

------
whatislovecraft
> Using a neural network trained on widely available weather forecasts and
> historical turbine data, we configured the DeepMind system to predict wind
> power output 36 hours ahead of actual generation.

Interesting. Something I've long wondered about:

\- Many wind farms are located in very specific places that have measured high
winds and are otherwise good places to locate a wind farm.

\- Global climate change is changing local climates too. Are the studies that
were done 10-20 years ago to discover ideal wind farm locations no longer as
accurate as they used to be?

\- How are wind farm locations decided today? Is there is skill in predicting
where a wind farm might be optimally located in 5 years rather than where it
should be located today? Or if not (since probably not) what are the right
risk mitigation actions to take if you are planning a wind farm but unsure
about location?

\- Since Machine Learning is useful in improving the value of the existing
wind power, could it also be useful in this endeavor, finding optimal wind
power generation locations?

\- Lastly, is this something a person could research at home with open data
and code? Just curious. :)

------
shereadsthenews
A demonstration of the astonishing value of a relatively small government
program: weather observation and forecasting.

~~~
HeWhoLurksLate
Weather forecasting is _not_ a small program, NOAA & NWS are both pretty big,
but they're not the largest.

Actually, scratch that. On the scale of government, they're more medium size.

------
afpx
Is this a substantial improvement over existing neural network models? I’ve
seen models that were being used to trade renewables on PJM’s market that
seemed at least as good as these. And, similar models are currently being used
to manage DERs in micro-grids.

I don't mean to downplay these results. I’m not an expert. Just mentioning it
in case someone at deepmind sees this and wants to reach out to PJM. I believe
those models were created almost 10 years ago.

~~~
giacaglia
It is probably not. Their neural network to optimize their warehouse
efficiency had 6 neurons. The neural network used here is probably pretty
small too

------
straws
Reminds me of some of the excellent advice in Bret Victor's essay about global
warming:

[http://worrydream.com/ClimateChange/](http://worrydream.com/ClimateChange/)

The investment is not only in the machinery, but in tools to model, observe,
and predict efficiency.

------
astanway
If anyone is interested in this type of work, our team at Amperon is hiring
data engineers, data scientists, and front end engineers. We’re funded by SV
Angel and Notation Capital and we’ve already got traction in ERCOT (Texas).

More info: [https://medium.com/@astanway/introducing-
amperon-2cded368284...](https://medium.com/@astanway/introducing-
amperon-2cded3682847)

~~~
RhodesianHunter
I'm not seeing a data engineer role by title or description in the angelist
site linked to at the bottom of that post, is that the right spot?

Is remote (Texas) viable?

------
tobias2014
How exactly does this need machine learning? Naively I would assume that it's
somewhat directly correlated with wind speed (and little more, maybe
direction) and a very simple model based on that data will give you good
predictions.

~~~
sgillen
I think determining the wind speed and direction 36 hours in advance is the
tricky bit.

~~~
joe_the_user
Weather forecasting is difficult problem but it's problem that as far as I
know has been worked on intensely for quite a while.

Weather forecasting is based on physics-based models that, so far as I know,
are essentially correct with the fundamental barrier to prediction being a
combination of random effects and the difficulty of solving the complex
equations numerically.

Deep learning involves training a heuristically approximate a system based on
past data. It has been used to emulate various human-learning behaviors like
recognizing images or recognizing good or bad position in games. Essentially,
the visible successes are in "we don't know but can extrapolate roughly from
data" (or we "know" but can't easily program it, in the case of image
recognition). So I'd be surprised if deep learning do a good or better job in
situation where _we do know_ how things work, we already running algorithm
which is correct - the situation with weather.

------
JHonaker
I didn't see any comparison to other simpler methods. I'd like to see how well
a time series model works.

------
qsolo
I was super excited when I read the headline and saw that deepmind published
the article. I was hoping for something along the lines of applying
reinforcement learning on the turbine or park level to optimize power output.
Finding the optimal pitch angles, better yawing algorithms (yaw misalignemnts
are one of the most common causes of underperforming wind turbines), or
finding optimal yaw directions for the entire wind park (downwind wake effects
can have a negative impact on other turbines)...

Unfortunately this is really only about predicting wind speeds to then
forecast power output. IIRC, grid codes in some countries are starting to
require 24 hour, 15 minute interval forecasts for renewables? I've also seen
this feature being advertised by other companies over the years as well,
although I don't recall seeing any numbers on accuracy for any of the models.

------
formalsystem
> Our hope is that this kind of machine learning approach can strengthen the
> business case for wind power and drive further adoption of carbon-free
> energy on electric grids worldwide. Researchers and practitioners across the
> energy industry are developing novel ideas for how society can make the most
> of variable power sources like solar and wind. We’re eager to join them in
> exploring general availability of these cloud-based machine learning
> strategies.

Would be nice to know a bit more detail? Which power grids was this approach
tested on? How does this approach work exactly? Are we talking about DNN on
time series? The graphs also really need labels to inform.

~~~
confounded
>> _... join them in exploring general availability of these cloud-based
machine learning strategies._

... is probably the operative part. Possibly uncharitable, but this appears to
be ‘thought leadership’ marketing generally supporting the idea of GCP managed
services.

------
brainpool
Of course it can, but only on the margin as this article suggests. The real
potential is in figuring out where to put them, what to put in the spots, and
most importantly, who to choose to build them and support them given the
aforementioned variables. There may also be some advances to be made in the
actual design, but I do not believe machine learning is going to help so much
there.

------
angry_octet
"Statistics are useful for forecasting."

Meteorologists have long used a metric, forecast skill, to evaluate different
techniques. It would be good if the researchers used standard benchmarks.

ML is probably much more useful in understanding the demand for power, the
reaction to weather and current events. But even still, it seems a stretch.

------
ychen306
Perhaps some people are missing the point here?. The post is not about the
technical challenge/novel but rather how _existing_ ML techniques can play a
part in making reusable energy more efficient.

