
How Coaches and the NYT 4th Down Bot Compare - trevin
http://www.nytimes.com/newsgraphics/2013/11/28/fourth-downs/post.html
======
kosei
Tough to take everything into account, but I find it troubling that all non-
success/failure outcomes are not part of the calculation (ex: 8-yard gain on a
4th and 10; a fumble on a 4th and 2 run, etc). It simplifies the formula to
the layman, but I have to imagine that it adds enough additional variance to
matter, especailly considering we're talking in variances of less than half a
point in their stated example.

Also (and to their point), there is a ton of variation between offenses,
defenses, punters, running backs, etc. Using this as a rubric is a nice idea
for a rule of thumb for an armchair quarterback, but I'd strongly disagree
with using it to accurately criticize any decision within a couple points of
expected value.

~~~
altoz
This is especially true of 4th down plays. A surprise play-action pass on 4th
and 1 can go for an especially long gain and thus should have higher expected
value. There's also the fake-punt scenario, which I suspect tends to be more
successful than a regular 4th down.

~~~
kosei
Absolutely. And I'm sure if coaches started trying to run more 4th down plays,
that would dramatically change the algorithm as well.

------
slg
These type of articles never take into consideration the reason why coaches
don't go for it as often as a computer might. The coaches face a reverse moral
hazard when taking risks because the negative outcomes are multiplied for
them. Even if the statistics support their decisions, a coach will be fired
for taking too many risks and failing. Is gaining an extra 2% (or whatever) in
expected returns worth the risk of losing everything if the low(er) percentage
outcome does actually happen? It is the same idea behind the whole "no one
gets fired for buying IBM/Microsoft".

Until you get a coach's motivations more in line with the motivations of the
team, you won't get a coach that is as aggressive as these computer
simulations suggest they should be.

~~~
symbaton
People always say this, but it is actually true? Has any coach ever been fired
for missing too many fourth downs?

~~~
spinlock
Ask Nick Saban how trying to kick a field goal with 1 second left in the
Baylor game is working out for him. There are plenty of people saying that any
other coach would be fired.

~~~
javanix
That particular outcome had happened less than 5 other times in the history of
the modern NCAA. 99/100 times that game ends with either an Alabama win or
O.T.

He hardly made a controversial decision.

~~~
spinlock
The bar that I was watching the game in exploded when that happened. You
wouldn't believe the number of people calling him all sorts of names for
making that call. And, I think it was a good call too. The point is that being
risk averse is the right way for a coach to keep his job.

------
Zimahl
Why would you ever go for it on 4th and 1 inside your own 10 yard line? You
would put yourself at a severe disadvantage if you failed.

Since the data from this is from previous plays in the last 10 or so seasons
it is flawed. There probably isn't very little data of a team going for 4th
and 1 and that deep in their own territory. The only time this has probably
happened is in extraordinary circumstances - specifically in a game-winning
drive scenario with time expiring, where the defense is in 'prevent'. In these
cases it is highly successful in that situation but you don't have the time to
get the score.

~~~
altoz
I think statistically, if you punt from the 9 yard line, the other team is
going to get excellent field position (around the 50 yard line). The
difference between failing on 4th and 1 from your own 9 and the other team
getting the ball around the 50 yard line is 41 yards of field position.

That may sound like a lot, but I'd imagine the actual point difference is on
the order of 2 points.

The more interesting thing to consider would be whether taking a safety there
is ever a positive.

~~~
swang
No. Because you cost yourself 2 points and you give the ball to them.
Essentially you gave them 2 points at the cost of being able to "punt" the
ball at the 35.

~~~
twoodfin
Ah, but sometimes two points is not so bad if the clock is a factor and you're
about to turn over possession regardless. Down by one and facing a fourth down
against the Broncos from his own one yard line with three minutes to go, Bill
Belichick famously took an intentional safety[1]. The field position gain from
the free kick was worth the two points, and the Patriots went on to win.

[1]
[http://sports.espn.go.com/espn/wire?id=1653659](http://sports.espn.go.com/espn/wire?id=1653659)

~~~
swang
Ah interesting, I have never heard about this game specifically.

It seems though he gave up some amount of EV by giving up the 2 points to the
Broncos, thus a FG later on would only tie the game instead of winning it. But
I assumed this was offset by the risk of having to punt and the Broncos end up
getting a FG where then they'd be up by 4 points.

I don't know if Belichick was right to do this, but it paid off for him which
is why everyone pats him on the back.

Contrast this with the 2009 Colts/Pats game where he went for it on 4th and 2
on his own 28 and didn't make it.

~~~
twoodfin
Exactly. Punt from the 1 and the Broncos will have the opportunity to burn
clock and still likely kick a field goal, forcing you into a touchdown-
required situation. As it was, the free kick netted even more yards than
expected.

Going for it on 4th and 2 in the '09 game was the right decision, and few
knowledgeable commentators disparaged Belichick for it. Manning was dissecting
the Pats defense at that point in the game: He had a much greater chance of
scoring a touchdown than the Colts defense had of stopping the conversion.

Now the playcall...

------
thejteam
This was discussed some in
[https://news.ycombinator.com/item?id=6728821](https://news.ycombinator.com/item?id=6728821)

I've thought about it some since then and I'm also not convinced that expected
points is a good metric. That will maximize the expected point differential
per season but I am not convinced (although I could be wrong... I haven't put
pencil to paper to make my thoughts rigorous) that there are enough
possessions in the game on average to make this metric useful even in the
first half.

~~~
gerad
The best defense is keeping the other team's offense off the field. Going for
it on fourth down has a much higher probability of doing that than a punt
does.

------
timfrietas
This does not take into account an important factor that is somewhat
intangible, but real to the psychology of the game: point swings. If you go
for it on 4th and 1 at your own 1 and lose you are definitely down 3 points
and probably 7. This puts pressure on your offense--which just failed on a 4th
and 1--to somehow make it all the way down the field next time. That can be
demoralizing for that offense, paralyzing in fact. That is much harder to
model, but an important part of a correct model, I believe.

~~~
aero142
This actually seems measurable. Do teams that are behind perform worse on
average? Do teams perform worse than normal after failing a 4th down attempt?

~~~
twoodfin
The same statistics wonk has done a lot of work on "momentum" in games. His
conclusion is that there is no such thing:

[http://www.advancednflstats.com/2013/11/momentum-1.html](http://www.advancednflstats.com/2013/11/momentum-1.html)

That jibes with my own anecdotal experience watching a lot of football,
despite how often announcers like to cite it as a factor.

------
nova22033
It's the same thing with penalty kicks in soccer. Statistically, you are more
likely to score if you kick the ball down the middle. The goalie is going to
go one way or another and there's a 50% chance he'll guess right. So why don't
players kick it down the middle..because if they kick it down the middle and
the goalie manages to block the kick, they'll come off looking like idiots and
it would look real bad on the highlight shows.

------
vacri
There's an interesting hole in the data at 4 & 10 on the opponent's 36. In a
sea of punting and kicking, this one data point has actual coaches going for
it.

Also, looking at the overall pattern of the data, my gut tells me that the
optimal rubric is somewhere in-between the 'bot and the meatcoach. There are
some strange holes as well - at 4th&3, you should go for if you're on your own
8 or 9, but punt if you're on your own 7 or 10?

~~~
thejteam
That makes sense. At the 36, it is a 53 yard or so field goal. At the outer
range of most kickers and pretty low percentage, so you give the ball to the
opponent at the 40 something yardline with high probability. On the other
hand, its also close enough to be very difficult for a punter to pin in inside
the 5 or 10, so even if you fail you only give the opponent 16 yards over a
punt.

But the holes at the your own end you mentioned, that makes no sense at all.

------
Uhhrrr
Hm, so the data points for likelihood of getting another first down is based
on data from the fourth downs that coaches actually attempted? It may be that
the fourth downs they didn't attempt were not as propitious for some reason -
wind (which the authors mention), injuries on key players, lateral positioning
which doesn't favor one formation or another, etc.

~~~
cheald
The choice to go for a 4th down conversion is more likely than not primarily
decided by the current score and time left in the game. However, the success
of a 4th down conversion probably isn't heavily influenced by those factors
(other than perhaps player fatigue), which would suggest that coaches are
indeed too conservative.

~~~
001sky
_is more likely than not primarily_

This seems pretty loose. I'm not trying to be pedantic, but the theoretic
flaws in the data are such that you would want a pretty tight logic in your
model to have any level of comfort with it. The proportion of the state-space
that is unobserved seems ~large and critically relevant.

------
snowwrestler
The fundamental flaw with this analysis is that statistics are descriptive,
not predictive.

You can't reliably predict the future with statistics alone. If you could,
then technical analysis of the stock market would be the most successful
investing strategy. It is not.

Likewise, a lot of folks seem to think that taking an historical statistical
trend and extending it into the future is a scientific approach to football
coaching. It's not.

Scientists use statistics to understand evidence, but they formulate theories
in terms of root causes. In other words it's not enough to know that dropping
a hammer has a high correlation with acceleration of 10 meters per second
squared. If you want to predict how fast other objects in the future will
fall, you need a theory that describes why and how anything falls.

It doesn't look to me like the 4th down bot embodies or implements any
cohesive theory of football success. It just finds trends and dots the line
into the future.

------
reddog
Here is a high school coach from Little Rock that never punts, and always
onside kicks. His reasoning is sound and it seems to work out for him.

[http://www.youtube.com/watch?v=AGDaOJAYHfo](http://www.youtube.com/watch?v=AGDaOJAYHfo)

------
dfan
It is interesting what an emotional reaction people instinctively have to 4th
down. Going for it on 4th down and failing feels like a catastrophe. But
punting would have given the ball away anyway, so all you've done is give the
opponent an extra 40 yards or so. Which is bad, sure, but it's a hell of a lot
less bad than, say, throwing an interception on 1st and 10. But somehow giving
up the ball on downs has come to be seen as a massive failure, on par with an
regular turnover.

~~~
snowwrestler
Turning it over on downs is a turnover, as sure as fumbling is. It's not
accounted that way in the box score, but in terms of impact on the game it is
the same.

An interception is better or worse than turning it over on downs by the field
position delta. If it is returned past the line of scrimmage, then it's worse.
If it is not returned, it's better. A deep interception with no return is like
a punt in terms of impact on the game.

