
Show HN: Euro 2016 predictions using Bayesian inference - lum
http://kickoff.ai
======
lum
For those who are interested in the details: we model the strength of teams
using the well-know Elo model (used e.g., for official chess ratings).

We innovated on two aspects of the traditional Elo model, in which every team
has an independent parameter:

1) We actually model the strength of _players_ instead of teams. This makes it
possible to learn from games that were played in championships between clubs,
and transfer this knowledge to games between countries

2) There is a not-so-widely-known connection between Elo-type comparison
models and Gaussian process classification. We leverage this, and get a full
posterior distribution for each team's strength. (Information on the
uncertainty of our estimates helps a lot in coming up with sensible
predictions)

If anyone wants to know more (explanations on the web page are very
superficial at the moment), please drop me a line!

~~~
antouank
But how does that make sense in a game where you can have a draw? By
definition 100% cannot cover only home/away. Draw must be factored in. So
doesn't that make all the Elo assumptions false? Just wondering how you can
take draw into consideration...

~~~
lum
Excellent point - this one is definitely on our todo list. There are several
simple extensions of the Elo model that take draws into consideration (i.e.,
give a non-zero probability to draws), for example the Rao-Kupper model. There
are only minimal changes needed w.r.t. the original model, but still we didn't
manage to make the changes in time for this version of the site.

In short: at its core, the "Elo assumption" postulates that every team can be
represented by a real number (that can be interpreted as the _strength_ of the
team), and that the probability of the outcome depends on on the difference in
strength. In the vanilla Elo model, the outcome is binary, but it's easy to
make it ternary.

~~~
tfgg
Have you considered using a TrueSkill-alike with extensions for scores/teams,
such as PoissonOD?

[http://research.microsoft.com/pubs/193839/sbsl_ecml2012.pdf](http://research.microsoft.com/pubs/193839/sbsl_ecml2012.pdf)

A few years ago I had a similar idea for trying to build team models, so you
can make a better guess at the performance of national teams, since they don't
play very often, or for league teams due to transfers at the start/during the
season, but hadn't got as far as you :)

~~~
lum
We did some preliminary experiments in this direction. Basically, we tried to
do a regression on the score difference instead of using only binary outcomes.
In our experiments it didn't improve the predictive accuracy - but there are
many more things to try. It does feel a bit wasteful not to take score data
into account.

Nice that you had some similar ideas :-)

------
alexroan
Great base for future development of the idea.

Some things to take into account though:

\- Friendly games are sometimes about fitness rather than winning at all costs

\- Weather greatly effects results. For example, in heavy rain and high wind,
the likelihood of the game yielding more than 2.5 goals reduces massively.

\- How does the algorithm take into account substitutions?

\- If a team in the group has already won 2 games, they may have already won.
In that case they may rest many players or try new tactics

\- Not all leagues (and levels of those leagues) are taken into account

Very much looking forward to seeing where this goes!

~~~
lum
Thanks a lot for the comments!

\- about friendly games: agreed, in the future we'd like to downweight them
(as well as downweight older games over newer ones)

\- about substitutions: at training time, we look at the number of minutes
each player has spent on the field. At test time, we assume there will be no
substitution. Could certainly be improved :-)

\- about teams that are already qualified: true, our model does not encode all
these contextual factors, and arguably they are very important for certain
games. If the lineup changes (e.g. star players resting) it does impact the
prediction.

------
gloves
Cool work! Appreciate it's not your mission to say 'this will or won't happen'
and is just a bit of fun.

But the great thing about football is that interesting as this is, I won't be
placing any bets off the back of it. Too many variables in human nature,
playing conditions, external factors, and maybe even _luck_. It's what keeps
the game interesting and millions watching.

It would be interesting as a question to wonder what parts of human behaviour
can be predicted accurately in this way though!

~~~
huherto
It is my personal opinion that soccer has a very high luck component. Often
four or five plays decide the game. Frequently enough it is even decided by a
referee call. It makes the game attractive because even an inferior team has a
decent chance against a superior team. It may be just a lucky shot and then
you close your defense.

~~~
semi-extrinsic
> It may be just a lucky shot and then you close your defense.

See Greece in WC 2010 as an example of this as a deliberate strategy.

------
Guigoz
Betting 2€ with my coworkers on the euro results. Will be using your
prognostics because I don't know anything about football.

MAKE ME RICH! \o/

------
mcjiggerlog
I think the fundamental assumption that a team's performance is the sum of the
individual players' past record is wrong.

If you studied how a pundit would judge outcomes for these games, I'm sure
they'd be talking about current form, whether the players play well together,
tactics, playing styles etc. Take England for example, they've given Rooney a
far higher score than either Vardy or Kane, even though any sane pundit is
clammering for Rooney to not start, saying the other strikers are on far
better form and that Rooney's presence detracts from the overall performance
of the team.

As a technical project it's pretty cool, I just doubt its accuracy. I guess
we'll soon find out!

~~~
secfirstmd
Man Utd fan here. Agree completely with assessment of Rooney.

------
pmontra
Football (it's Euro 2016) is a team sport so players are important but it's
also very important how they play together as a team. Sometimes there are
teams with average players that win competitions because they help each other
much more than teams with star players (example: Leicester this season).

I don't know if you can model this taking in account single players, even if
their performance in previous games could be an indicator about the strength
of the team. We will see soon how the model fares.

~~~
oneeyedpigeon
"We will see soon how the model fares."

I'm no statistician but "soon" seems unlikely; wouldn't you need a lot of
results in order to judge how good these predictions are? I'd like to know if
the model can use data capped at any time period, so that matches from the
past can be judged against what the model would predict.

~~~
pmontra
If they have data for the last 10 years they could use the model to predict
thousands of games. That would start to be statistically significant for
anybody.

However, I'm no statistician myself too but if the goal is to predict only 12
results how do we evaluate the predictions of the model? There should be a
difference between 0 and 12 successes.

~~~
pmontra
They are 5-2 after the first 3 days. The first game was a draw (72%-28%) and
the third one a win for Wales (39%-61%). Two days and five games to go.

------
d99kris
Here's another prediction data point, based on betting odds:

[http://eeecon.uibk.ac.at/wopec2/repec/inn/wpaper/2016-15.pdf](http://eeecon.uibk.ac.at/wopec2/repec/inn/wpaper/2016-15.pdf)

------
thom
This is similar to the stuff GoalImpact does:

[https://twitter.com/Goalimpact/status/740657055841816577](https://twitter.com/Goalimpact/status/740657055841816577)

It's true most predictive models work at the team level, but it's interesting
to point out that one of the more successful models over the last season was
Chad Murphy (@soccermetric)'s MOTSON, an SVM trained over a bunch of stats per
player:

[https://twitter.com/JamesWGrayson/status/732673663019802625](https://twitter.com/JamesWGrayson/status/732673663019802625)

------
Grue3
I can say right off the bat this is completely wrong. The sum of probabilities
of each team winning cannot be 100%. Why? Because draws exist. Worse yet, the
concept of "playing for a draw" based on your current position in the group
also exists. Also, the concept of "nothing to play for", when the team is
already guaranteed the first place in the group and might not play at full
strength in the final game.

~~~
r3bl
Yes, the draws exist, but every other argument you've posted cannot be applied
to the first round of group games. Chances are huge that all of the times are
going to want to start strong.

As the researchers said multiple times, they're going to be updating their
algorithms all of the time and take into account the things they've missed in
the current version.

~~~
tremon
_Chances are huge that all of the times are going to want to start strong._

Except when you're the lowest-ranked team, and starting your group with a
match against the favourite. In that case, you're likely to play for damage
control (e.g. aim for a draw, don't concede too many goals).

------
kidoptcus
It's an interesting prediction. The UI design is clear too!

Do you have the prediction for the final rankings and the winning %? Some
predict France and Germany are in top 2.

------
sbardle
The predictions look good to me except Wales vs Slovakia because of Gareth
Bale.

~~~
lum
You're right. We took Wales' last lineup, but it will very probably be a
different lineup against Slovakia. We'll update the prediction shortly before
the game, when we know who's going to play.

------
ctlaltdefeat
Are you guys confident that this can beat the market (i.e Betfair)? I would
highly suspect that the answer is "not at all". Does this affect you?

------
snake_plissken
Noice! I'm excited to see how the results turn out. Have you guys thought of
incorporating info other than player and match data, like money-lines?

------
NumberCruncher
Did you validate your model on former EC or WC data?

------
riffraff
interesting, the model seems very slightly confident that italy will win
against belgium, even though it recently lost 3-1, and most rankings put the
belgians ahead, and gut feelings say that italy's current team kinda sucks.

I really hope your model is right :)

EDIT: could it be you are underestimating the weight of player age?

~~~
vicow
We are not considering the player's age as a feature at all actually.

Concerning the predictions, they depend on the starting 11 for each team. We
took the most recent ones, i.e. during the last friendlies, which are probably
not the ones that will actually start the game!

------
lormayna
Did you take care about home advantage? I would be very happy to know more
about math of your model

~~~
vicow
We did, it helps quite a lot! Concerning the model, we use Gaussian process
classification with a logit link function, a linear kernel and Expectation
Propagation as inference method.

