Over 29 years of fanhood Posted June 22, 2019 Posted June 22, 2019 15 hours ago, Logic said: Stumbled across this on Twitter today. In a nutshell, the chart shows how often teams made the decision on 4th down that matched what the New York Times 4th Down bot (which is built based on what decisions analytics suggest give a team the best statistical chance at winning) suggested. The higher you are on the chart, the more often you made a "good" choice on 4th down. This chart is just for the 2018 season. It shows the Bills as the 6th "smartest" or "most frequent makers of correct decisions" on 4th down. I find it interesting because so many people claim that McDermott is an awful 4th down decision maker. At least one source, the one mentioned here, disagrees. There it is again... Bill and Panthers...
Ronin Posted June 22, 2019 Posted June 22, 2019 15 hours ago, Logic said: Stumbled across this on Twitter today. In a nutshell, the chart shows how often teams made the decision on 4th down that matched what the New York Times 4th Down bot (which is built based on what decisions analytics suggest give a team the best statistical chance at winning) suggested. The higher you are on the chart, the more often you made a "good" choice on 4th down. This chart is just for the 2018 season. It shows the Bills as the 6th "smartest" or "most frequent makers of correct decisions" on 4th down. I find it interesting because so many people claim that McDermott is an awful 4th down decision maker. At least one source, the one mentioned here, disagrees. We ranked 31st (of 32 that would be) in converting 4th downs. We were ahead of Rosen and Arizona. I'm not sure what "good decisions" means, but in terms off converting 4th downs, we sucked, royally. In other unfavorable news, we ranked 30th in Scoring %, that is the percentage of drives resulting in offensive scores. We ranked 29th in drives ending in offensive TOs. 28th in average number of plays/drive. 30th in average number of plays/drive. 26th in average time/drive 30th in average number of points/drive 1 hour ago, buffalobloodfloridahome said: I think we as fans should realize last year was a time to test things out and see what works. Go for broke, because we knew we were outmatched on our oline and needed every opportunity to run game clock and keep the other offense off the field. This year we will see a much more normal play calling and risk taking as long as our offense has improved as dramatically as it should. We played for broke last year with low expectations and a rookie QB Let's hope so, but it's still a very hefty climb to merely average.
dickleyjones Posted June 22, 2019 Posted June 22, 2019 this analysis is problematic. all it actually shows is how the team's decisions lined up with an algorithm. that's it. you cannot draw any more conclusions than that. there is nothing that shows the bot is correct. maybe the bot is wrong 100% of the time. that would mean we are one of the worst decision makers (for example). analysis is fun, but far too many people draw conclusions from it when there is no basis for it. decision making is far more complicated than making a decision tree and following it. 2
Ronin Posted June 22, 2019 Posted June 22, 2019 7 hours ago, MJS said: Sure, but it's a pretty small sample size. Teams don't go for it on 4th down all that often. Differences between them are murky from the small sample size. It's better to make the right decision in the long run. If your team isn't good enough, well that sucks and you'll lose games regardless of whether or not you punt it on 4th down. That is correct. The range for 4th-down attempts runs from a high of 25 (Denver) to a low of 8 (Chargers). 29 teams rank between 23 and 12 attempts/game. That's on the magnitude of approximately 1/game. We had 18, converted 7.
oldmanfan Posted June 22, 2019 Posted June 22, 2019 Not much relevance to this. Maybe in the middle of the field but there is no way any NFL HC is going for it on 4th and 1 deep in their own side of the field.
Ronin Posted June 22, 2019 Posted June 22, 2019 9 minutes ago, dickleyjones said: this analysis is problematic. all it actually shows is how the team's decisions lined up with an algorithm. that's it. you cannot draw any more conclusions than that. there is nothing that shows the bot is correct. maybe the bot is wrong 100% of the time. that would mean we are one of the worst decision makers (for example). analysis is fun, but far too many people draw conclusions from it when there is no basis for it. decision making is far more complicated than making a decision tree and following it. Good points. On analysis however, one has to know which analyses are relevant and which are not. That's the problem with these metrics departments on teams these days, they overthink things and use irrelevant, at least relatlively speaking, metrics/data/analyses.
Just Joshin' Posted June 22, 2019 Posted June 22, 2019 32 minutes ago, dickleyjones said: this analysis is problematic. all it actually shows is how the team's decisions lined up with an algorithm. that's it. you cannot draw any more conclusions than that. there is nothing that shows the bot is correct. maybe the bot is wrong 100% of the time. that would mean we are one of the worst decision makers (for example). analysis is fun, but far too many people draw conclusions from it when there is no basis for it. decision making is far more complicated than making a decision tree and following it. Correct - how do we know the algorithm is correct? There are always potential bias built into the logic. People assume the algorithm is right which is not true.
Thurman#1 Posted June 22, 2019 Posted June 22, 2019 47 minutes ago, Ronin said: We ranked 31st (of 32 that would be) in converting 4th downs. We were ahead of Rosen and Arizona. I'm not sure what "good decisions" means, but in terms off converting 4th downs, we sucked, royally. In other unfavorable news, we ranked 30th in Scoring %, that is the percentage of drives resulting in offensive scores. We ranked 29th in drives ending in offensive TOs. 28th in average number of plays/drive. 30th in average number of plays/drive. 26th in average time/drive 30th in average number of points/drive Just the percentage of converting 4th downs doesn't tell you too much. You'd also need to know how difficult the situations were. Not converting a 4th and one with reasonable consistency is an indicator of inability. Not converting a 4th and 18 with consistency is more an indicator of how often you were so far behind in a game that you would be trying to convert a 4th and 18. And yes, our offense was bad last year. Which has a lot to do with how very few resources we put into improving the offense the last two years (outside of bringing in Josh Allen). The unit we put a lot of resources into performed quite well last year. This year we had a ton of money under the cap thanks to their hard and painful work in clearing up the cap. And they put a lot of resources into the offense. I, personally, expect an improved offense this year, with the improvements to the OL in particular and the offense in general.
Thurman#1 Posted June 22, 2019 Posted June 22, 2019 (edited) 31 minutes ago, Just Joshin' said: Correct - how do we know the algorithm is correct? There are always potential bias built into the logic. People assume the algorithm is right which is not true. It's very likely a smart algorithm indeed. It's been tried with great success. Yes, it was in high school, but the team using it have had terrific success. Look up Pulaski Academy. Their coach doesn't never punt, but almost never. He punted in his most famous game (there's a documentaray about it, against Highland Park). But it was 4th and 30 from their own 16. The math said punt, so he punted. https://www.si.com/college-football/2015/09/09/why-college-football-coach-should-adopt-no-punt-always-onside-kick-philosophy I wouldn't do what Kelley does, but I'd sure take more risks than most NFL coaches are willing to assume. It makes sense. NFL coaches are too risk-averse on this issue and a few others as well. Edited June 22, 2019 by Thurman#1
formerlyofCtown Posted June 22, 2019 Posted June 22, 2019 16 hours ago, TPS said: The Bills ranked 31st in their 4th down conversion rate in 2018. I think what this states is they were good at making the right decision on 4th and 5 or less yards. I think one of the reasons McD and Beane re-did the O-line is they did not win the line of scrimmage on most short yardage runs, with the exception of Allen and his sneaks. I expect them to be much better with some of the beasts they brought in.... https://www.teamrankings.com/nfl/stat/fourth-down-conversion-pct I think a lot of guys arent getting that this doesnt mean we were good at converting on fourth. 1
formerlyofCtown Posted June 22, 2019 Posted June 22, 2019 4 minutes ago, Thurman#1 said: It's very likely a smart algorithm indeed. It's been tried with great success. Yes, it was in high school, but the team using it have had terrific success. Look up Pulaski Academy. Their coach doesn't never punt, but almost never. He punted in his most famous game (there's a documentaray about it, against Highland Park). But it was 4th and 30 from their own 16. The math said punt, so he punted. https://www.si.com/college-football/2015/09/09/why-college-football-coach-should-adopt-no-punt-always-onside-kick-philosophy I wouldn't do what Kelley does, but I'd sure take more risks than most NFL coaches are willing to assume. It makes sense. Patriots are middle of the pack. I havent looked but I bet they are among the highest to convert. I think whether you think your O has the ability to convert is the biggest factor in going for it.
Locomark Posted June 22, 2019 Posted June 22, 2019 I would prefer we become the 6th most FIRST DOWN decision making team.... 1
Buffalo Boy Posted June 22, 2019 Posted June 22, 2019 17 hours ago, Inigo Montoya said: Hopefully with an upgraded offensive line to go with our athletic 6'5" -- 240 lb quarterback, we will have even more success on 4th and short. It's also easier to go for it on 4th down when you know you have a great defense to help bail you out if you don't get it. QB keepers are all well and good. Having an o line that can line up on 4th and short and everyone knows the RB is getting it and you still win most of those situations.....? That is preferable.
Rock'em Sock'em Posted June 22, 2019 Posted June 22, 2019 (edited) 17 hours ago, Albany,n.y. said: The main reason the Bills rank so high is because Allen on a QB sneak is automatic on 4th & a yard or less. They rank high because their decisions tended to match analytics optimums. But those assume average offense against average defense based on the score and time remaining in the game or half. Every team would then augment those baseline analytics with coaching decisions in cases when you have an all-pro punter or kicker, or if your short yardage offense is notably good (Allen 1 yard sneaks), game-time injuries (like to a kicker), weather, or other facts that make the particular circumstance deviate from "averages". I wonder if there is any correlation to making close to "optimal" decisions based only on analytics and outcome. Notably, Pats* were in the middle of the road. Edited June 22, 2019 by Rock'em Sock'em
syhuang Posted June 23, 2019 Posted June 23, 2019 (edited) On 6/21/2019 at 5:45 PM, Mr. WEO said: Nope. They were actually awful on 4th down conversions, even if the robot told them to "go for it". Next to last in conversions (an anemic 38.9%). At some point, a HC has to say "we're not really very good at this, even if it seems like the right call"...and punt. You can't just use 4th down conversion percentage and said they should just punt even the "right call" by the bot is to go for it. Let's look at each 4th down attempt in the season where Bills converted 7 out of 18 attempts (38.89%). (1) Among the 18 attempts, half of them were when trailing near the end of the game or trailing by large score. You probably can still use punting argument in some of these plays, but not most of them when considering the score and situation. 4th&8 at own 39, 14:02 3rd, tailing 0-26, failure (peterman incomplete pass) 4ht&4 at opp 20, 10:59 4th, tailing 5-31, failure (Anderson 3 yard pass) 4th&2 at opp 32, 3:52 4th, tailing 6-25, failure (Anderson incomplete pass) 4th&18 at opp 43, 0:48 4th, tailing 6-25, failure (Peterman 9 yard pass) 4th&1 at opp 33, 5:13 3rd, tailing 3-31, success (Peterman 1 yard run) 4th&9 at opp 31, 2:36 3rd, tailing 3-31, failure (Peterman incomplete pass) 4th&16 at opp 31, 2:00 4th, tailing 9-41, failure (Peterman sacked) 4th&11 at opp 30, 1:05 4th, tailing 17-21, failure (Allen incomplete pass) 4th&3 at Opp 25, 2:23 2nd, tailing 0-14, failure (Allen incomplete pass) (2) Of the remaining attempts, 6 can be categorized as 4th & short as following. Bills converted 5 of them. Using overall 38.9% 4th down conversion to claim they should have punted in these situations is simply wrong consider their 4th&short conversion rate is actually high. 4th&1 at opp 1, 12:14 2nd, leading 17-0, success (Allen 1 yard run) 4th&1 at opp 38, 13:13 1st, 0-0, success (Ivory 3 yard run) 4th&1 at opp 28, 7:40 2nd, leading 7-3, success (Ivory 2 yard run) 4th&2 at opp 13, 3:19 3rd, leading 7-13, failure (Allen 1 yard run, fumble) 4th&1 at opp 41, 1:50 4th, leading 14-13, success (Allen 2 yard run) 4th&1 at Opp 8, 10:46 3rd, 14-14 success (Allen 3 yard run) (3) For the remaining 4th down attempts, one is the famous Bojoquez botched play in TEN game when he thought it's a fake. Coach didn't call for 4th down attempt and actually called for FG. Another one is a trick play where Logan Thomas executed a fake punt and converted the 4th down. Maybe they should have called for punting in that situation, but that's what trick play does with the surprise element. The last 4th down attempt is Allen took a kneel in the end of the last game when leading 42-17. There is really no point to argue whether they should have punted there. 4th&7 at opp 12, 5:03 2nd, leading 7-3, failure (Bojorquez incomplete pass, FG formation, TEN game, miscommunication) 4th&6 at own 47, 7:26 1st, leading 7-0, success (L. Thomas 15 yard pass, punt formation) 4th&4 at Opp 24, 1:21 4th, leading 42-17 failure (Allen -1 kneel) Edited June 23, 2019 by syhuang 2
Thurman#1 Posted June 23, 2019 Posted June 23, 2019 (edited) 13 hours ago, formerlyofCtown said: Patriots are middle of the pack. I havent looked but I bet they are among the highest to convert. I think whether you think your O has the ability to convert is the biggest factor in going for it. I disagree. Ability to convert should be a factor, but IMO the second or third-most important. Situation is most important. 4th down conversion rates are statistically all over the place, due to the small sample size and the difference in likelihood of conversion in different situations, such as a 4th and 1 being far easier than a 4th and 18. 3rd down conversion rates are far more indicative of ability to convert, due to larger sample. But situation is still a huge factor. And 3rd down conversion rates go from 29% to 47% last year. There are none in single digits and nobody over 75%. They're gathered in a reasonably small range. Not to be too bad of a bore, but how much of the difference might be due to ... for example, only ... say, the Colts having an average 3rd down of 3rd and three while the Cards might have an average 3rd down of 3rd and five. Made up those numbers ... if anyone can be bothered to find the real ones, that'd be great. Point is, though, situation is by far the most important variable. Oh, and the Pats were 9th in 3rd down conversions. But 25th in 4th down conversions, at 50%, probably because they are a bit more situationally aggressive than most teams, and for good reason. Fewer times to need to make desperate conversion attempts, but more willing to try in non-desperate situations where the distance is a bit longer than most coaches would be comfortable with. Nice post, Syhuang!!! Edited June 23, 2019 by Thurman#1
2020 Our Year For Sure Posted June 23, 2019 Posted June 23, 2019 9 hours ago, syhuang said: You can't just use 4th down conversion percentage and said they should just punt even the "right call" by the bot is to go for it. Let's look at each 4th down attempt in the season where Bills converted 7 out of 18 attempts (38.89%). (1) Among the 18 attempts, half of them were when trailing near the end of the game or trailing by large score. You probably can still use punting argument in some of these plays, but not most of them when considering the score and situation. 4th&8 at own 39, 14:02 3rd, tailing 0-26, failure (peterman incomplete pass) 4ht&4 at opp 20, 10:59 4th, tailing 5-31, failure (Anderson 3 yard pass) 4th&2 at opp 32, 3:52 4th, tailing 6-25, failure (Anderson incomplete pass) 4th&18 at opp 43, 0:48 4th, tailing 6-25, failure (Peterman 9 yard pass) 4th&1 at opp 33, 5:13 3rd, tailing 3-31, success (Peterman 1 yard run) 4th&9 at opp 31, 2:36 3rd, tailing 3-31, failure (Peterman incomplete pass) 4th&16 at opp 31, 2:00 4th, tailing 9-41, failure (Peterman sacked) 4th&11 at opp 30, 1:05 4th, tailing 17-21, failure (Allen incomplete pass) 4th&3 at Opp 25, 2:23 2nd, tailing 0-14, failure (Allen incomplete pass) (2) Of the remaining attempts, 6 can be categorized as 4th & short as following. Bills converted 5 of them. Using overall 38.9% 4th down conversion to claim they should have punted in these situations is simply wrong consider their 4th&short conversion rate is actually high. 4th&1 at opp 1, 12:14 2nd, leading 17-0, success (Allen 1 yard run) 4th&1 at opp 38, 13:13 1st, 0-0, success (Ivory 3 yard run) 4th&1 at opp 28, 7:40 2nd, leading 7-3, success (Ivory 2 yard run) 4th&2 at opp 13, 3:19 3rd, leading 7-13, failure (Allen 1 yard run, fumble) 4th&1 at opp 41, 1:50 4th, leading 14-13, success (Allen 2 yard run) 4th&1 at Opp 8, 10:46 3rd, 14-14 success (Allen 3 yard run) (3) For the remaining 4th down attempts, one is the famous Bojoquez botched play in TEN game when he thought it's a fake. Coach didn't call for 4th down attempt and actually called for FG. Another one is a trick play where Logan Thomas executed a fake punt and converted the 4th down. Maybe they should have called for punting in that situation, but that's what trick play does with the surprise element. The last 4th down attempt is Allen took a kneel in the end of the last game when leading 42-17. There is really no point to argue whether they should have punted there. 4th&7 at opp 12, 5:03 2nd, leading 7-3, failure (Bojorquez incomplete pass, FG formation, TEN game, miscommunication) 4th&6 at own 47, 7:26 1st, leading 7-0, success (L. Thomas 15 yard pass, punt formation) 4th&4 at Opp 24, 1:21 4th, leading 42-17 failure (Allen -1 kneel) Really good post
Recommended Posts