Draft Success Measured by AV - Part III

[JGMcD2]

5 minutes ago, BullBuchanan said:

no, I read it. Your model is flawed because it's based on a known bad value.

This doesn't work. It's definitely an admirable attempt, but the underlying data isn't any good.|

 

 

"What I did was found the average value for a player drafted in each round (1-7) in each year (2017-2019) and the calculated what I am calling the Net Drafted Accumulated Value (NETDrAV) for each pick in each round. I only compared each draft to itself. I then found the Total Net Drafted Accumulated Value (TOT_NETDrAV) for each team in each draft and ranked them against each other. Rather than just looking at how much raw value the Bills brought in as compared to the 31 other teams, this gives an idea of how much extra value they extracted in each round as compared to the 31 other teams in the league. 

 

TOT_NETDrAV is the AV that a team has benefitted from directly on their team as compared to players and teams in their draft class.

 

TOT_NETCarAV is just the pure amount of value that players drafted by a given team have produced as compared to players and teams in their draft class."

Known bad value according to whom?

 

What value should be used to measure instead?

Edited February 8, 2023 by JGMcD2

[Big Turk]

2 hours ago, JGMcD2 said:

Elam is TBD. Showed signs of being a good/great pick - we knew it was going to be an adjustment. 

 

Rousseau was actually a REALLY good value for where they picked. They can't really control much else. Only way the Bills would have gotten a significantly better player was if they picked in the top 12 picks that year. 

 

Oliver was over-drafted but isn't a bad player - not awful value but not great either. 

 

Edmunds is better than OK. Not going to turn this into a Tremaine Edmunds thread.

 

Allen was a home run. 

 

I'd have to look, but I'd argue RD1 has been better than OK when compared to the rest of the league. 

 

It really is amazing how people think every other team drafts superstar players in the first round every year.

[JGMcD2]

3 minutes ago, Big Turk said:

 

It really is amazing how people think every other team drafts superstar players in the first round every year.

Highest concentration of them is in the top 10. 

 

You're lucky if you find one after that in RD1... you're REALLY lucky if you find one after RD1. 

[Coach Tuesday]

Beane generally has drafted high floor players, many at non-premium positions.  It skews the overall results in his favor, because many of his draft picks are still in the league.  But other than Allen he has yet to hit a grand slam at a premium position in the draft.  Rousseau is trending that way.  Oliver has been too inconsistent.  Epenesa and Basham and Ford are trending toward busts.  Elam is an unknown but still feels like a panic pick.  He drafted one great offensive linemen and traded him away for peanuts.  Other than that, he's drafted a bunch of serviceable players at non-premium positions who remain in the NFL.  Straight down the fairway - except last year's draft, which was a disaster at the worst possible time.  He needs to start hitting on high-ceiling premium players or the talent gap between the Bills and their rivals is going to widen.

[BullBuchanan]

59 minutes ago, JGMcD2 said:

Known bad value according to whom?

the wAV of guard John Miller is 31 and the guard Wyatt Teller has a wAV of 29
the wAV of WR Sammy Watkins is 48 and the wAV of WR Justin Jefferson is 39

I'd say the model is flawed to everyone who has ever watched football.
 

59 minutes ago, JGMcD2 said:

What value should be used to measure instead?

Great question. A model needs to start with questions, and you asked some good ones while calculating your formula but you need to backtest it to be sure that it gives accurate results. The more accurate you try to be, the less reliable it's likely to be, so you need a reliable grade of performance first and foremost. There are multiple other problems that need to be solved for. An example of some:

If you're weighting against draft position, you need to consider the difference of value between pick value within a round as the #1 overall pick is much different than the #32 pick and they don't average out over time since it's based on performance, not chance. You should really be grading performance against that position int he draft, but statistical significance will be impossible to obtain since there haven't been enough #1 picks, and you'd have to grade performance for all players ever at that slot, and there may be position bias.

If you're grading performance, you need to make sure you're grading performance against games played, expected games played and role. If you drafted Breece Hall and he was an all pro for half the season and then went down due to injury, you need to be careful about how that compares to a guy that played all 16 games but was less impressive. Likewise if a DB was drafted in the 6th round but was a standout special teamer, you need to ensure he ranks comparably against a player who played a lot more downs in a DB role but didn't perform well.

Edited February 8, 2023 by BullBuchanan

[Big Turk]

5 minutes ago, BullBuchanan said:

the wAV of guard John Miller is 31 and the guard Wyatt Teller has a wAV of 29
the wAV of WR Sammy Watkins is 48 and the wAV of WR Justin Jefferson is 39

I'd say the model is flawed to everyone who has ever watched football.
 

Great question. A model needs to start with questions, and you asked some good ones while calculating your formula but you need to backtest it to be sure that it gives accurate results. The more accurate you try to be, the less reliable it's likely to be, so you need a reliable grade of performance first and foremost. There are multiple other problems that need to be solved for. An example of some:

If you're weighting against draft position, you need to consider the difference of value between pick value within a round as the #1 overall pick is much different than the #32 pick and they don't average out over time since it's based on performance, not chance.

If you're grading performance, you need to make sure you're grading performance against games played, expected games played and role. If you drafted Breece Hall and he was an all pro for half the season and then went down due to injury, you need to be careful about how that compares to a guy that played all 16 games but was less impressive. Likewise if a DB was drafted in the 6th round but was a standout special teamer, you need to ensure he ranks comparably against a player who played a lot more downs in a DB role but didn't perform well.

 

AV is a sum of their career total, I think you are simply not understanding it.  Watkins has a 51 AV and 9 seasons, meaning he averages 5.7 AV per year.  Jefferson has a 41 AV and 3 seasons meaning he has an average of 13.7. If Jefferson simply puts up similar numbers, he would have almost 123 AV in the same time period as Watkins.  

 

Trying to use Career AV doesn't work unless you are comparing the same number of years. You would need to take AV/year if you are trying to use it compare players in the way you are trying to use it for, otherwise it makes no sense.

[BullBuchanan]

2 minutes ago, Big Turk said:

 

AV is a sum of their career total, I think you are simply not understanding it.  Watkins has a 51 AV and 9 seasons, meaning he averages 5.7 AV per year.  Jefferson has a 41 AV and 3 seasons meaning he has an average of 13.7. If Jefferson simply puts up similar numbers, he would have almost 123 AV in the same time period as Watkins.  

 

Trying to use Career AV doesn't work unless you are comparing the same number of years. You would need to take AV/year if you are trying to use it compare players in the way you are trying to use it for, otherwise it makes no sense.

I understand it just fine. I'm using the weighted AV.

Using AV of the player's rookie year alone would be a far worse way to judge the performance of a draft. The chiefs would get a 0 for Mahomes in 2017 despite drafting the best QB in 20 years

[JGMcD2]

2 minutes ago, BullBuchanan said:

I understand it just fine. I'm using the weighted AV.

Using AV of the player's rookie year alone would be a far worse way to judge the performance of a draft. The chiefs would get a 0 for Mahomes in 2017 despite drafting the best QB in 20 years

Ok but I'm not using weighted AV?

 

I'm using DrAV...

[BullBuchanan]

12 minutes ago, JGMcD2 said:

Ok but I'm not using weighted AV?

 

I'm using DrAV...

Which is a metric you invented right?

You're using this definition to determine what players had good seasons vs poor ones using AV

"Offense

Every team gets this many points to divvy up among its offensive players:

team_offense_points = 100 * (team offensive points per drive) / (league average offensive points per drive),

where

offensive points per drive = (7*(rushTD+passTD) + 3*FG) / (rushTD + passTD + turnovers + punts + FGA)

Offensive line

As a unit, the offensive line for a given team will share this many points:

team_points_for_o_line = 5 / 11 * team_offense_points

For each offensive lineman (and fullback and tight end), we define:

individual_points = [(games played) + 5*(games started)*(pos_multiplier)] * (all_pro_multiplier),

where pos_multiplier = 1.2 for tackles, 1.0 for guards and centers, 0.3 for fullbacks, and 0.2 for tight ends,

and all_pro_multiplier = 1.9 for first-team AP all-pro, 1.6 for second-team AP all-pro, and 1.3 for a pro bowler who was not first- or second-team all-pro. [NOTE: all_pro_multiplier is for tackles, guards, and centers only, not fullbacks or tight ends.]

Finally, each individual player receives this many points:

approx_value = (individual_points) / (sum of individual_points for all players on team) * (team_points_for_o_line)"

Did you actually read that when you came up with this and decided it was viable? John Miller's second season had a 9 - which is better than all of Wyatt teller's years but 1 where he had 10. A model that grades like that can't be relied upon as an indicator of success. it's MASSIVELY skewed in favor of offensive points scored rather than individual contribution.

Spencer Brown registered a 9 this year, and I think we can all agree he's atrocious. that score of 9 would have outperformed 7 of Orlando Pace's seasons.

Edited February 8, 2023 by BullBuchanan

[JGMcD2]

11 minutes ago, BullBuchanan said:

Which is a metric you invented right?

You're using this definition to determine what players had good seasons vs poor ones using AV

"Offense

Every team gets this many points to divvy up among its offensive players:

team_offense_points = 100 * (team offensive points per drive) / (league average offensive points per drive),

where

offensive points per drive = (7*(rushTD+passTD) + 3*FG) / (rushTD + passTD + turnovers + punts + FGA)

Offensive line

As a unit, the offensive line for a given team will share this many points:

team_points_for_o_line = 5 / 11 * team_offense_points

For each offensive lineman (and fullback and tight end), we define:

individual_points = [(games played) + 5*(games started)*(pos_multiplier)] * (all_pro_multiplier),

where pos_multiplier = 1.2 for tackles, 1.0 for guards and centers, 0.3 for fullbacks, and 0.2 for tight ends,

and all_pro_multiplier = 1.9 for first-team AP all-pro, 1.6 for second-team AP all-pro, and 1.3 for a pro bowler who was not first- or second-team all-pro. [NOTE: all_pro_multiplier is for tackles, guards, and centers only, not fullbacks or tight ends.]

Finally, each individual player receives this many points:

approx_value = (individual_points) / (sum of individual_points for all players on team) * (team_points_for_o_line)"

Did you actually read that when you came up with this and decided it was viable? John Miller's second season had a 9 - which is better than all of Wyatt teller's years but 1 where he had 10. A model that grades like that can't be relied upon as an indicator of success. it's MASSIVELY skewed in favor of offensive points scored rather than individual contribution.

Spencer Brown registered a 9 this year, and I think we can all agree he's atrocious.

I didn't create DrAV... it's listed on PFR. I'm finding the value a team is extracting compared to their peer in a given draft. Not comparing across drafts... it's in comparison to the available talent pool in a given draft. 

 

Again, if you have a better publicly available stat let me know. Be sure to let publications like the Washington Post and The Athletic know too... 

[BullBuchanan]

3 minutes ago, JGMcD2 said:

I didn't create DrAV... it's listed on PFR. I'm finding the value a team is extracting compared to their peer in a given draft. Not comparing across drafts... it's in comparison to the available talent pool in a given draft. 

 

Again, if you have a better publicly available stat let me know. Be sure to let publications like the Washington Post and The Athletic know too... 

PFF's rating would be far better than attempting to asses draft pick performance based on overall team performance as AV appears to be doing. That at least grades a player independently.

AV may actually be a useful counter metric as it tells you what players performed on strong teams so a player with a high PFF and a low AV may be better than one with a High/High

With respect to journalists using these values, they aren't statisticians. As I mentioned, you're free to create whatever model you want comprised of whatever metrics you deem relevant, but it's the questions you ask when creating it and the insights you glean from it that matter and come under scrutiny.

In this case, I think the formula for AV isn't telling you which players performed the best, so it isn't a useful metric when considering who had a strong draft.

Edited February 8, 2023 by BullBuchanan

[CookieG]

Im just going ot mention this with a smile.  Seems appropriate, considering the Eagles are in the SB and many consider them the most comeplete roster in the league.

 

It was written in November of 2020 by some Philly "beat writer". Does it sound familiar?

 

1. In 2013, the Eagles drafted Lane Johnson and Zach Ertz. Since then, Howie Roseman’s drafts have landed the Eagles exactly one Pro Bowl player, and that’s Carson Wentz. The only defensive player Roseman has ever drafted who’s made a Pro Bowl is Fletcher Cox, the 12th pick in the 2012 draft. One defensive Pro Bowler in nine drafts (not counting this year). He’s never found a defensive Pro Bowler after the 12th pick overall. Wentz is also the only Pro Bowler Roseman has drafted who’ll be in his 20s on opening day next year. It’s not just J.J. Arcega-Whiteside instead of D.J. Metcalf. This has been happening for years. Scrounging up functional starters from the late rounds or the practice squad or the waiver wire is fine. But you need stars to win big. Studs. Elite players. And Howie’s failure to deliver star power to this roster has made it virtually impossible for the Eagles to compete on a regular basis with the NFL’s best teams. Maybe Miles Sanders or Dallas Goedert becomes a full-fledged star, but right now there isn’t a single elite player in his 20s on this roster, and that’s a damning indictment of Roseman’s draft record. Every GM has misses. Every GM has big misses. But you can’t afford to have the big misses without the big hits. Doug Pederson has been awful. Wentz has been terrible. But when you look at the Eagles’ decline, it all starts in the GM’s office.

 

 

https://www.nbcsports.com/philadelphia/eagles/howie-rosemans-shocking-draft-record-and-more-roobs-random-observations

[JGMcD2]

19 minutes ago, BullBuchanan said:

PFF's rating would be far better than attempting to asses draft pick performance based on overall team performance as AV appears to be doing. That at least grades a player independently.

AV may actually be a useful counter metric as it tells you what players performed on strong teams so a player with a high PFF and a low AV may be better than one with a High/High

With respect to journalists using these values, they aren't statisticians. As I mentioned, you're free to create whatever model you want comprised of whatever metrics you deem relevant, but it's the questions you ask when creating it and the insights you glean from it that matter and come under scrutiny.

In this case, I think the formula for AV isn't telling you which players performed the best, so it isn't a useful metric when considering who had a strong draft.

Publicly available... are you shelling out $200 for me to get a PFF subscription?

 

Weren't you also complaining about All-Pro not being factored in... must have conveniently glossed over the Pro-Bowl and All-Pro multiplier used in AV.

Edited February 8, 2023 by JGMcD2

[BullBuchanan]

14 minutes ago, JGMcD2 said:

Publicly available... are you shelling out $200 for me to get a PFF subscription?

 

Weren't you also complaining about All-Pro not being factored in... must have conveniently glossed over the Pro-Bowl and All-Pro multiplier used in AV.

No, i didn't complain about that. I noted the overall value of an all pro being the same as a player who is one of the worst players in the league.

Edited February 8, 2023 by BullBuchanan

[mjt328]

Being a GM gets progressively harder with success.

Lower draft picks.  Less money.  Fewer holes mean you must focus on needs, instead of going BPA every round.

 

Brandon Beane did a great job pulling the Bills out of the basement, and making us a Super Bowl contender.  No doubt.

He still needs to prove he can KEEP us at the top. 

 

Most teams can only contend for 3-4 years, and then must step back and reload.  It's a rarity for teams to remain elite for a decade (like the Patriots).  Unfortunately, the recent draft trend suggests we aren't going to sustain what we've seen the last few years.

[newcam2012]

9 hours ago, Big Turk said:

 

Hate to break the news to you, but McD is not only NOT on the hot seat, it's probably about as ice cold as it could get.  You are dreaming if you think he is going anywhere without multiple losing seasons with a good team.

Oh I agree his seat is cold. I'd agrue that his seat should start getting warmer if this team doesn't progress. McD has had 6 years of Allen who is arguably one of the top 3 QBs in the league. 

 

The Bills led by McD have failed to progress deep into the playoffs. Arguably, Coach McD blew that chance in 13 seconds vs KC. Hate to bring that up but I think it carries validity. This year his team was flat, unprepared, and laid an egg vs Cinci in a huge home field playoff game. Excuses or not much should fall on the coach. 

 

Let's look at KC and Philly for examples since both are in the Super Bowl. KC built up their Oline almost immediately and protected their elite QB. Bingo a Super Bowl win and likely more to follow.

 

Philly drafted young QB and surrounded him with excellent weapons and an elite defense. Bingo a Hurts Super Bowl appearance in year three. 

 

The Bills have floundered their opportunities to advance. They have squandered their opportunities to improve. Thus, they can't overcome injuries. They have glaring weaknesses in the trenches and at WR.  

 

Not sure how fans no matter how optimistic you are can't see this organization is not elite. The wait til next year mentality isn't working. The injury excuse isn't working. The players will develop excuse isn't working. The cap space issues. That's what they are selling us fans. Listen to Beane's presser. He pretty much summarizes that. I just can't and won't accept that. 

 

Yet, this organization is going to run it back. How can Bills fans be satisfied with that? Changes, schemes, philosophy, attitude, etc... have to be made. Teams like KC and Cinci have passed the Bills. Other teams are right there or better. It's time for the organization to get aggressive and assertive not passive. Running it back is passive. 

 

Now is not the time for fans to be complacent. Are you satisfied with nice regular season records? Making playoff appearances only to exit early? Perhaps many are based on the previous Bills history. 

 

As for myself, I want at least a Super Bowl appearance and than a Super Bowl win. Currently, the direction of this team makes that feat  distant and unlikely. For that reason, all the coache's seats including McD shouldn't be ice cold!

Edited February 9, 2023 by newcam2012

[JGMcD2]

1 hour ago, Coach Tuesday said:

Beane generally has drafted high floor players, many at non-premium positions.  It skews the overall results in his favor, because many of his draft picks are still in the league.  But other than Allen he has yet to hit a grand slam at a premium position in the draft.  Rousseau is trending that way.  Oliver has been too inconsistent.  Epenesa and Basham and Ford are trending toward busts.  Elam is an unknown but still feels like a panic pick.  He drafted one great offensive linemen and traded him away for peanuts.  Other than that, he's drafted a bunch of serviceable players at non-premium positions who remain in the NFL.  Straight down the fairway - except last year's draft, which was a disaster at the worst possible time.  He needs to start hitting on high-ceiling premium players or the talent gap between the Bills and their rivals is going to widen.

 

Since Beane took over drafting in 2018:

• 61 players who have entered league have been named to at least one All-Pro team
• Remove Special Teams players from that group then the number drops to 43 players named to at least one All-Pro team
• Account for only premium positions (QB, WR, T, DE, CB) and the numbers drops to 22 players named to at least one All-Pro team
  • QB - 3
  • WR - 7
  • T - 3
  • DE - 3
  • CB - 6
     
• 3 teams have drafted multiple All-Pro players at premium positions since 2018
  • Atlanta - Calvin Ridley (2018) and AJ Terrell (2021)
  • Dallas - Micah Parsons (2021), Trevon Diggs (2020) and CeeDee Lamb (2020)
  • San Francisco - Deebo Samuel (2019) and Nick Bosa (2019)
     
• 2 GMs have drafted multiple All-Pro players at premium positions since 2018
  • Dallas - Jerry Jones (Will McClay)
  • San Francisco - John Lynch
    • Thomas Dimitroff drafted Calvin Ridley and Terry Fontenot drafted AJ Terrell
       
• Based on the Bills draft position from 2018 - 2022 Brandon Beane had the opportunity to select the following players without sacrificing acquiring another All-Pro Player at a premium position:
   
  • Josh Allen (HE DID!)
  • DK Metcalf (Over Ed Oliver and Cody Ford)
  • Calvin Ridley (Over Tremaine Edmunds) - Suspended from NFL
  • Jaire Alexander (Over Tremaine Edmunds)
  • Deebo Samuel (Over Ed Oliver)
  • Maxx Crosby (Over Ed Oliver, Cody Ford, Devin Singletary and Dawson Knox)
  • J.C. Jackson (Entire 2018 Draft excluding Josh Allen)
  • A.J. Brown (Over Ed Oliver and Cody Ford) - Traded away before becoming an All-Pro w/ Tennessee 

 

Let's not pretend like these premium players grow on trees and are easy to identify. 2/3 of them were 1st round picks that we didn't even have a chance to acquire, and the group listed above had an ADP of the 50th pick... meaning everyone passed on them at least once... including other top tier GMs. 

[Coach Tuesday]

2 minutes ago, JGMcD2 said:

 

Since Beane took over drafting in 2018:

• 61 players who have entered league have been named to at least one All-Pro team
• Remove Special Teams players from that group then the number drops to 43 players named to at least one All-Pro team
• Account for only premium positions (QB, WR, T, DE, CB) and the numbers drops to 22 players named to at least one All-Pro team
  • QB - 3
  • WR - 7
  • T - 3
  • DE - 3
  • CB - 6
     
• 3 teams have drafted multiple All-Pro players at premium positions since 2018
  • Atlanta - Calvin Ridley (2018) and AJ Terrell (2021)
  • Dallas - Micah Parsons (2021), Trevon Diggs (2020) and CeeDee Lamb (2020)
  • San Francisco - Deebo Samuel (2019) and Nick Bosa (2019)
     
• 2 GMs have drafted multiple All-Pro players at premium positions since 2018
  • Dallas - Jerry Jones (Will McClay)
  • San Francisco - John Lynch
    • Thomas Dimitroff drafted Calvin Ridley and Terry Fontenot drafted AJ Terrell
       
• Based on the Bills draft position from 2018 - 2022 Brandon Beane had the opportunity to select the following players without sacrificing acquiring another All-Pro Player at a premium position:
   
  • Josh Allen (HE DID!)
  • DK Metcalf (Over Ed Oliver and Cody Ford)
  • Calvin Ridley (Over Tremaine Edmunds) - Suspended from NFL
  • Jaire Alexander (Over Tremaine Edmunds)
  • Deebo Samuel (Over Ed Oliver)
  • Maxx Crosby (Over Ed Oliver, Cody Ford, Devin Singletary and Dawson Knox)
  • J.C. Jackson (Entire 2018 Draft excluding Josh Allen)
  • A.J. Brown (Over Ed Oliver and Cody Ford) - Traded away before becoming an All-Pro w/ Tennessee 

 

Let's not pretend like these premium players grow on trees and are easy to identify. 2/3 of them were 1st round picks that we didn't even have a chance to acquire, and the group listed above had an ADP of the 50th pick... meaning everyone passed on them at least once... including other top tier GMs. 

Sure if All Pro is your measuring stick.  

[JGMcD2]

1 minute ago, Coach Tuesday said:

Sure if All Pro is your measuring stick.  

Well what's the measuring stick then Coach... you didn't give one. 

[BullBuchanan]

Nobody tell PFR.
 

6. OL Spencer Brown

(Photo by Bryan M. Bennett/Getty Images)

Overall grade: 53.4

NFL positional ranking: 76th of 84 qualifying

https://billswire.usatoday.com/lists/pro-football-focus-buffalo-bills-rodger-saffold-spencer-brown/?utm_campaign=trueanthem&utm_medium=trueanthem&utm_source=facebook&fbclid=IwAR0akCLnbsphAyZ7kwevDYtCG9C4NxdkVqOqeBBuG1LmwY7SkOjX7H7n7EM



vs Tristan Wirfs, 2nd Team all pro and Pro Bowler

Edited February 8, 2023 by BullBuchanan