When Analytics/Philosophy blind coaches to Situational Awareness

[Chaos]

I am a huge believer in making decisions based on analytics. But I don't thinnk football head coaches in general have demonstrated mathematical genius ( I am not saying I am).   The 4th and 3 play call with a 14 point lead in the NFC championship game, I think is an example of not properly understanding the math.  

 

I will start with an analogy.  Imagine a dice game, where the rules allow you to pick either of the following situations A) rolling 1-5 pays you even money B) rolling a 6 pays you 100-1.     If I tell you you get to roll the die 50 times, the expected return for A is that you would roll 1 to 5 5/6 or 42 times $42 dollars (assuming a $1 bet each time and the expected return on B is that you would roll 6 eight times and get a return of $800.   Any rational person would select B as there choice.    While the numbers are extreme this is similar to the 'analytics' of going for it on fourth and 3 on the opponents 20 yard line.  Over a large number of interations, the correct statistical decision is to go for it on fourth and 3. 

Now imagine if after your fifty rolls, you have collected you $800 based on your selection of B.  You made the wise decision.   However, lets say the rules of the game change (like in the second half of an NFC championship game).    Now there is only 1 roll of the die.  If you roll a 1-5 you get to keep your $800.  If you roll a 6, your $800 grows to $900.    A rational person would always select A in this scenario.  However I feel as though  Dan Campbell was confused by the excitement of winning with analytics over the course of a 17 game decision, and tried to apply a season long anaytic concept to a very small remaining set of plays.   And he made a mistake.  Not quite as extreme as my dice example.  But a field goal from that spot is probably 90% plus successful (2.7 points) , and a conversion was probably 70% likely (4.9 points expected value, if we assume the conversion leads to a TD 100% of the time).  The difference in expected points between 2.7 and 4.9 was not worth increasing the likelhood of zero from 10 to 30%. 
 

Edited January 29 by Chaos

[dave mcbride]

18 minutes ago, Chaos said:

I am a huge believer in making decisions based on analytics. But I don't thinnk football head coaches in general have demonstrated mathematical genius ( I am not saying I am).   The 4th and 3 play call with a 14 point lead in the NFC championship game, I think is an example of not properly understanding the math.  

 

I will start with an analogy.  Imagine a dice game, where the rules allow you to pick either of the following situations A) rolling 1-5 pays you even money B) rolling a 6 pays you 100-1.     If I tell you you get to roll the die 50 times, the expected return for A is that you would roll 1 to 5 5/6 or 42 times $42 dollars (assuming a $1 bet each time and the expected return on B is that you would roll 6 eight times and get a return of $800.   Any rational person would select B as there choice.    While the numbers are extreme this is similar to the 'analytics' of going for it on fourth and 3 on the opponents 20 yard line.  Over a large number of interations, the correct statistical decision is to go for it on fourth and 3. 

Now imagine if after your fifty rolls, you have collected you $800 based on your selection of B.  You made the wise decision.   However, lets say the rules of the game change (like in the second half of an NFC championship game).    Now there is only 1 roll of the die.  If you roll a 1-5 you get to keep your $800.  If you roll a 6, your $800 grows to $900.    A rational person would always select A in this scenario.  However I feel as though  Dan Campbell was confused by the excitement of winning with analytics over the course of a 17 game decision, and tried to apply a season long anaytic concept to a very small remaining set of plays.   And he made a mistake.  Not quite as extreme as my dice example.  But a field goal from that spot is probably 90% plus successful (2.7 points) , and a conversion was probably 70% likely (4.9 points expected value, if we assume the conversion leads to a TD 100% of the time).  The difference in expected points between 2.7 and 4.9 was not worth increasing the likelhood of zero from 10 to 30%. 
 

The other issue I always raise with regard to the situation is the opponent. The 49ers are not the Commanders. They finished first in defense last season and 3rd this season. 

[BarleyNY]

2 hours ago, Chaos said:

I am a huge believer in making decisions based on analytics. But I don't thinnk football head coaches in general have demonstrated mathematical genius ( I am not saying I am).   The 4th and 3 play call with a 14 point lead in the NFC championship game, I think is an example of not properly understanding the math.  

 

I will start with an analogy.  Imagine a dice game, where the rules allow you to pick either of the following situations A) rolling 1-5 pays you even money B) rolling a 6 pays you 100-1.     If I tell you you get to roll the die 50 times, the expected return for A is that you would roll 1 to 5 5/6 or 42 times $42 dollars (assuming a $1 bet each time and the expected return on B is that you would roll 6 eight times and get a return of $800.   Any rational person would select B as there choice.    While the numbers are extreme this is similar to the 'analytics' of going for it on fourth and 3 on the opponents 20 yard line.  Over a large number of interations, the correct statistical decision is to go for it on fourth and 3. 

Now imagine if after your fifty rolls, you have collected you $800 based on your selection of B.  You made the wise decision.   However, lets say the rules of the game change (like in the second half of an NFC championship game).    Now there is only 1 roll of the die.  If you roll a 1-5 you get to keep your $800.  If you roll a 6, your $800 grows to $900.    A rational person would always select A in this scenario.  However I feel as though  Dan Campbell was confused by the excitement of winning with analytics over the course of a 17 game decision, and tried to apply a season long anaytic concept to a very small remaining set of plays.   And he made a mistake.  Not quite as extreme as my dice example.  But a field goal from that spot is probably 90% plus successful (2.7 points) , and a conversion was probably 70% likely (4.9 points expected value, if we assume the conversion leads to a TD 100% of the time).  The difference in expected points between 2.7 and 4.9 was not worth increasing the likelhood of zero from 10 to 30%. 
 

 

Analytics should inform decisions not dictate them. In every situation there are nuances that are not factored in. Sometimes there are even assumptions inherent in the analysis that are not valid. Using analytics effectively requires understanding those things. 

[Cash]

Excellent post. And a great point about sample size. 
 

I’ve seen this a couple times, but not many: what the models should be spitting out is the “break even” probability for going for it. (And with all of the assumptions, approximations, and missing variables, they should probably be rounding to the nearest 10%.) If I’m a coach, that’s what I want to know. Do I need an 80-90% chance of success to be worth going for it, or is it closer to 50/50? And comparatively, how much do I trust my offense/playcall/QB against their defense right now? 

 

And on another note, the advantage of being up 3 scores vs 2 is significant. If the model (or the coach making decisions based on the model) didn’t take that into account, that’s a major problem. 

[dave mcbride]

1 minute ago, Cash said:

Excellent post. And a great point about sample size. 
 

I’ve seen this a couple times, but not many: what the models should be spitting out is the “break even” probability for going for it. (And with all of the assumptions, approximations, and missing variables, they should probably be rounding to the nearest 10%.) If I’m a coach, that’s what I want to know. Do I need an 80-90% chance of success to be worth going for it, or is it closer to 50/50? And comparatively, how much do I trust my offense/playcall/QB against their defense right now? 

 

And on another note, the advantage of being up 3 scores vs 2 is significant. If the model (or the coach making decisions based on the model) didn’t take that into account, that’s a major problem. 

Yes, the three score lead vs. two score lead is definitely a huge factor. It's not like they were trying to go from 17-14 to 20-14.

[DCOrange]

2 hours ago, dave mcbride said:

The other issue I always raise with regard to the situation is the opponent. The 49ers are not the Commanders. They finished first in defense last season and 3rd this season. 

San Fran was near dead last defensively in similar situations this season. Detroit was near the best offensively. One, the play worked and the WR just dropped it. The other, Goff had multiple guys he could have delivered the pass to and just missed it. Sucks but it seems like it was a fine decision in the moment and a fine play choice too. Kicking the field goal would have been a fine decision too.

Edited January 29 by DCOrange

[dave mcbride]

3 hours ago, DCOrange said:

San Fran was near dead last defensively in similar situations this season. Detroit was near the best offensively. One, the play worked and the WR just dropped it. The other, Goff had multiple guys he could have delivered the pass to and just missed it. Sucks but it seems like it was a fine decision in the moment and a fine play choice too. Kicking the field goal would have been a fine decision too.

I see the argument, but I just don't see how you don't trust your kicker to make a 45-yard FG in moderate conditions that puts you up 3 scores. in a game where you're facing a defense that's perennially  top 5. As for the drop, it was a moderately tough catch and Goff was pressured (and Goff was 0-8 under pressure last night). And as for the analytics argument (not saying you're making that, btw), I'll repeat what analytics OG Billy Beane famously said: "My sh*t don't work in the playoffs."

Edited January 29 by dave mcbride

[DCOrange]

13 minutes ago, dave mcbride said:

I see the argument, but I just don't see how you don't trust your kicker to make a 45-yard FG in moderate conditions that puts you up 3 scores. in a game where you're facing a defense that's perennially  top 5. As for the drop, it was a moderately tough catch and Goff was pressured (and Goff was 0-8 under pressure last night). And as for the analytics argument (not saying you're making that, btw), I'll repeat what analytics OG Billy Beane famously said: "My sh*t don't work in the playoffs."

Apparently re: the kicker, their kicker is literally the worst in NFL history from that range and he hadn't made a kick outside of a dome all season lol. Seems they didn't really take the position seriously because they always intended to go for it a ton this year and therefore didn't have faith when the time came where they might want a good kicker.

[dave mcbride]

19 minutes ago, DCOrange said:

Apparently re: the kicker, their kicker is literally the worst in NFL history from that range and he hadn't made a kick outside of a dome all season lol. Seems they didn't really take the position seriously because they always intended to go for it a ton this year and therefore didn't have faith when the time came where they might want a good kicker.

Huh? Badgley is 37-48 lifetime between 40 and 49 and made 9 of his last 11 (and also 2 of 3 from 50+) going back to last season. https://www.pro-football-reference.com/players/B/BadgMi00.htm

[HurlyBurly51]

Do analytics differentiate between a week 5 game against the Panthers and one in which a berth in the Super Bowl is on the line?  I think not.  Use your brain.

[Nextmanup]

Any chance you could re-word the thread title so that is makes sense from a grammatical perspective?

Thank you.

 

 

[Chaos]

22 minutes ago, Nextmanup said:

Any chance you could re-word the thread title so that is makes sense from a grammatical perspective?

Thank you.

 

 

Yes, there is a chance.

[DCOrange]

1 hour ago, dave mcbride said:

Huh? Badgley is 37-48 lifetime between 40 and 49 and made 9 of his last 11 (and also 2 of 3 from 50+) going back to last season. https://www.pro-football-reference.com/players/B/BadgMi00.htm

 

[HurlyBurly51]

1 minute ago, DCOrange said:

 

Wasn't it a 45 yarder?  Seems Nate is massaging his stats to support his case.

[Logic]

I agree with the general premise of this post.

I will go further and say that, in my opinion, Sean McDermott does a great job balancing analytics and "feel" and making good decisions in high pressure moments. He's routinely in the top three of the "goes for it on 4th when he should" chart, but doesn't allow himself to be a prisoner to it. There have been games when we, as fans, would prefer a little more aggression, but he trusts the way his defense is playing, so he's a touch more conservative, and it has paid off more often than not. Then, more recently -- especially in the second half of this season and in the playoffs -- he has made aggressive decisions when the analytics are 50-50, because he senses that his offense is hot or could use the jolt of confidence that such a decision sometimes gives a team.

I do not think McDermott is a perfect coach. Far from it. But credit where credit is due, I think he does a great job balancing analytics and game feel and, more often than not, making intelligent and "correct" decisions.

[dave mcbride]

8 minutes ago, HurlyBurly51 said:

Wasn't it a 45 yarder?  Seems Nate is massaging his stats to support his case.

He is. 

[Chaos]

10 minutes ago, Logic said:

I agree with the general premise of this post.

I will go further and say that, in my opinion, Sean McDermott does a great job balancing analytics and "feel" and making good decisions in high pressure moments. He's routinely in the top three of the "goes for it on 4th when he should" chart, but doesn't allow himself to be a prisoner to it. There have been games when we, as fans, would prefer a little more aggression, but he trusts the way his defense is playing, so he's a touch more conservative, and it has paid off more often than not. Then, more recently -- especially in the second half of this season and in the playoffs -- he has made aggressive decisions when the analytics are 50-50, because he senses that his offense is hot or could use the jolt of confidence that such a decision sometimes gives a team.

I do not think McDermott is a perfect coach. Far from it. But credit where credit is due, I think he does a great job balancing analytics and game feel and, more often than not, making intelligent and "correct" decisions.

Maybe. But he does not seem to understand when to call timeouts, and when wasting a time out is likely to create a problem later on. 

[Simon]

8 hours ago, Chaos said:

But a field goal from that spot is probably 90% plus successful

This is patently absurd and bears no relation to reality

 

8 hours ago, Chaos said:

and a conversion was probably 70% likely

Detroit's success rate this season on 4th/3 or less is over 77%

 

19 minutes ago, HurlyBurly51 said:

Wasn't it a 45 yarder? 

The first one was 45; I think the second one was around 48.

 

In both instances, both kicking and going were acceptable options; neither call was "wrong" in either instance.

Unless of course Badgley was struggling in pregame to make 45yrd kicks on that ***** grass; if that was the case, then going for it was absolutely correct.

From my perspective, I would have gone for the first one to keep pressure on SF, maintain possession to burn clock and hunt for a TD that might finish the game.

The second one I would probably have kicked to tie the game (again, dependent on how much Badgley was struggling pregame).

[Chaos]

6 minutes ago, Simon said:

This is patently absurd and bears no relation to reality

 

Detroit's success rate this season on 4th/3 or less is over 77%

 

The first one was 45; I think the second one was around 48.

 

In both instances, both kicking and going were acceptable options; neither call was "wrong" in either instance.

Unless of course Badgley was struggling in pregame to make 45yrd kicks on that ***** grass; if that was the case, then going for it was absolutely correct.

From my perspective, I would have gone for the first one to keep pressure on SF, maintain possession to burn clock and hunt for a TD that might finish the game.

The second one I would probably have kicked to tie the game (again, dependent on how much Badgley was struggling pregame).

The word "probably" is pretty critical in understanding my post.  What we do know is that fully adjusted for the game pressure  of the NFL Championship game, the Lions are not very good at converting 4th and 3. 

Edited January 30 by Chaos

[Simon]

21 minutes ago, Chaos said:

The word "probably" is pretty critical in understanding my post. 

 

So you did a deep dive with a bunch of mathematics to make a declared conclusion that it was not worth it.

Yet your entire mathematical argument hinges on "probably".

 

Perhaps a game like football doesn't readily lend itself to rigid mathematical analysis.