Once and for all....PFFffffffft

[Royale with Cheese]

3 minutes ago, BullBuchanan said:

 

And here lies the problem. PFF isn't saying a player is good or bad. They are providing a grade based on what a player does on each play. It's YOU and others who are ascribing a meaning to that grade.

PFF isn't saying that Jordan Davis is a top 10 DT. They are saying that within their grading system he has the 10th highest score. It's not at all a complete measure of a football player. It's one data point that measures the impact of each and every play of a player. It's entirely possible for you to think that Jordan Davis is "terrible" because of whatever reasons, while still having a higher aggregate score using PFFs metrics than most other players. If you were forced to break down why you thought Davis was terrible this year, I'm sure you'd find an answer as to why he still has a high PFF grade

This isn't just PFF. It goes for all data from all sources. You can't just take two results from a data source and automatically say that the higher result is the better thing.

As for what I know and don't know - I know how PFF says they score players and I work with data and insights professionally, so I think that's probably more experience than most people have. It doesn't make me an expert in PFF, but the types of foaming at the mouth posts raging against PFF don't require me to be an expert.

 

LOL then this entire thing doesn't make any sense then.

 

Within their grading system, he has the 10th highest score?  Is it not based on performance or something lol.

How is he scoring so high on PFF when he's doing nothing on the field?  Is no effort and walking on the field a metric they score favorably or something?

 

https://www.themirror.com/sport/american-football/philadelphia-eagles-bench-draft-pick-267699

 

"Davis in particular has come under scrutiny as former NFL offensive lineman Brian Baldinger delivered a brutal assessment of the physical condition he believes the 23-year-old is in.

“It is clear that [he] is out of shape,” the 64-year-old PHLY Eagles podcast. "I gotta believe they fine him every week. He’s overweight, and it shows. He’s not pursuing the ball, he’s nothing in the pass rush. So if he’s going to play with the effort and be overweight like that, then you think that they’re a man short in rotation.

“He’s not effective right now, they need another guy inside in their defensive tackle rotation. I’m not sure why he keeps dressing, he's ineffective. When plays are going on, he’s basically walking on the field, it’s bad effort.”

[Generic_Bills_Fan]

I think pffs analysis works pretty well for some positions/schemes/etc and it’s not as useful for others 

 

[Beck Water]

2 hours ago, BarleyNY said:

 

For ratings to be useful people have to understand what they really mean, which includes what they’re actually measuring as well as their limitations. I like a lot of what PFF does, but their grading has limited utility. It’s better with some positions than others, but is still limited to what players are asked to do rather than what they’re capable of. For instance, a QB like Cousins who plays a much more limited role in his team’s offense, can more easily score a higher grade than a QB like Allen, who much more is asked of.

 

Sam Monson recently mentioned this on a podcast when PFF’s grading came up. He used Dodson as an example of a player who graded very highly due to him doing well in a very limited role. It makes sense that coaches would be more interested in a player’s overall impact on a game and would factor in things like the difficulty of the assignment, the ability of the player, the contract of the player, etc. Each has their use and a value, but they aren’t going to be the same. 

 

I could be wrong here and again, hope I will be corrected but I think actual coaches grading game film are not considering things like the contract of the player, the difficulty of the assignment, and the ability of the player when grading.  Those are things which are taken into account when "valuing the positions" and "ranking the players against positional value" which Beane has talked about the FO and coaches doing after the season.

 

I redirect to what @Buffalo716 said upthread since I know he knows something about it:

https://www.twobillsdrive.com/community/topic/252639-once-and-for-allpffffffffft/page/3/#comment-8885301

 

The coaches grade on assignment, technique, and effort.  Did you "bring it" on that play?  Did you use good technique?  And did you "do your job"?  If a player (like AJ Klein filling in for Milano in 2020) is giving maximal effort and using sound technique, but simply can't complete his assignment because he's being asked to do things beyond his physical abilities, then the coaches need to see that and adjust his assignment to something within his physical capabilities while he's on the field, but it's not taken into account in the grade.

[BullBuchanan]

23 minutes ago, Royale with Cheese said:

 

LOL then this entire thing doesn't make any sense then.

 

Within their grading system, he has the 10th highest score?  Is it not based on performance or something lol.

How is he scoring so high on PFF when he's doing nothing on the field?  Is no effort and walking on the field a metric they score favorably or something?

 

https://www.themirror.com/sport/american-football/philadelphia-eagles-bench-draft-pick-267699

 

"Davis in particular has come under scrutiny as former NFL offensive lineman Brian Baldinger delivered a brutal assessment of the physical condition he believes the 23-year-old is in.

“It is clear that [he] is out of shape,” the 64-year-old PHLY Eagles podcast. "I gotta believe they fine him every week. He’s overweight, and it shows. He’s not pursuing the ball, he’s nothing in the pass rush. So if he’s going to play with the effort and be overweight like that, then you think that they’re a man short in rotation.

“He’s not effective right now, they need another guy inside in their defensive tackle rotation. I’m not sure why he keeps dressing, he's ineffective. When plays are going on, he’s basically walking on the field, it’s bad effort.”

I already explained this to you, you just didn't want to hear it. It's clear you'd rather just make something up to believe.

[Royale with Cheese]

2 minutes ago, BullBuchanan said:

I already explained this to you, you just didn't want to hear it. It's clear you'd rather just make something up to believe.

 

Just because you explained it, doesn't mean it made sense.

 

This is straight from PFF.  Their "metrics" states that Jordan Davis performed very well in his role.  

And it grades Ed Oliver as average.

Explain this.

 

"PLAYER RANKINGS

The grades allow for easy player comparisons, whether using an overall grade or a facet grade. While we believe the grade is an excellent baseline for how well a player performed his given role, we also believe the using the entire context is crucial when evaluating players. Perhaps a slot receiver had to play more than expected on the outside or a nose tackle was forced to play more three-technique than his coaching staff initially desired. Both players are being evaluated based on what they did, and that context is important when using the grades. PFF+ allows users to sort by player grades, but they can also see the simple and advanced stats that tell the story for each player."

 

https://www.pff.com/grades

[BullBuchanan]

1 hour ago, hondo in seattle said:

 

So, wait...  You're saying your smarter than us PFF critics because we don't understand "the intrinsic nature of player evaluation" but you do?   I'm not sure why you're taking low shots at the intelligence of fellow mafioso, but I disagree.

 

Most of us don't like the notion that you can reduce a player to a set of numbers.  Let me give another example.  Maybe Diggs is playing with broken ribs.  The injury will lower his grade.  But it won't lower the way a coach evaluates him as a player.  PFF lacks context.  Their graders don't know what the position coaches have been telling the player, the playcall, the assignment, the assignment of the guy next to the graded player which may effect his decision-making, his injury status, and so on.  

 

Additionally, how PFF grades each play and how they weigh (or don't weigh) different things is ultimately subjective.   Let's talk about weighting for a moment.  Let's say the Bills call a run to the right.  Shakir is lined up on the left and completely muffs his block. So PFF gives him a -2 grade, even though it didn't impact the run.  The next play Shakir makes a tremendous one-handed grab and dances through the defense for a dazzling 63-yard TD that wins the game with just seconds left.   PFF grades him a +2, their highest grade.  Shakir's PFF average for those two plays is ZERO.  But a coach would call him the hero of the game.  Weighting matters. 

 

The idea that you can entirely remove subjectivity from football evaluation is absurd.  Maybe PFF knows this, maybe not.  I don't care.  What I know is that their attempt to reduce players to a metric is flawed.  And while many of their grades seem accurate, some clearly are not.  

 

I don't hate PFF for trying.  In fact, when the Bills acquire a FA, one of the first things I do is look at his PFF grade for a ballpark estimate of his value.  But if I wanted a better evaluation, I'd talk to a coach.  

 

PFF accounts for weighting, and they would never give Shakir a -2 for just a missed block. If you look at their rubric, those numbers are reserved for explosive big plays. That said, you're absolutely right that it doesn't account for a player that's a big outlier. Let's say you have a QB that throws 2 passes in the dirt or out of the stadium and then throws a hail mary TD. He proceeds to do this every drive of his career. He'd be the all-time greatest QB ever, but he'd likely have a worse score than a guy that dinks and dunks it down the field and only occasionally scores. Now of course in reality things aren't that extreme, so if you have a guy that plays mostly solid, but has huge screwups you probably end up with a higher score than you'd think or vice-versa.

That doesn't make the system or analysis flawed. Every system has limitations. The flaw is using it to ascribe value that was never intended. You need to be real with what the data tells you and what it doesn't. It should be one piece of the puzzle. More often than not data should make you ask questions, not tell you answers. It's a very common mistake. Even c-level executives do it in fortune 500 companies - even when we tell them not to.

3 minutes ago, Royale with Cheese said:

 

Just because you explained it, doesn't mean it made sense.

 

This is straight from PFF.  Their "metrics" states that Jordan Davis performed very well in his role.  

And it grades Ed Oliver as average.

Explain this.

 

"PLAYER RANKINGS

The grades allow for easy player comparisons, whether using an overall grade or a facet grade. While we believe the grade is an excellent baseline for how well a player performed his given role, we also believe the using the entire context is crucial when evaluating players. Perhaps a slot receiver had to play more than expected on the outside or a nose tackle was forced to play more three-technique than his coaching staff initially desired. Both players are being evaluated based on what they did, and that context is important when using the grades. PFF+ allows users to sort by player grades, but they can also see the simple and advanced stats that tell the story for each player."

 

https://www.pff.com/grades

Man, all you had to do was read one more sentence.

Edited January 18 by BullBuchanan

[Beck Water]

19 hours ago, Einstein said:

 

You really think NFL teams would pay millions of dollars for useless data? (.....)

The other excuse is "teams pay for the data, not the grades". How do you think PFF grades the player? On the data!

 

You earned *respect* for your Good Samaritan action of sending a snowplow to the drive of a brother fan with a snowblower.

 

But I know it's been explained to you over and over that the data PFF pays for is not their grades.  It would be nice if you would acknowledge this.

 

In particular, in the first link, there are a LOT of data in those fields - of which player grades are only two fields.  Ultimately useful player grades are based on 1) knowledge of the play call and the player's assignment in the called play 2) perception of how they executed their assignment, their technique and effort.  The first is not data PFF has.  The second is not objective data per se though coaches who know their players and scouts who watch a ton of film can be very consistent in this assessment.

 

When former players such as Kurt Warner, JT O'Sullivan etc are putting together film they frequently caveat their critique "I don't know what the reads were in this play, for me it would be....".  That's because even these guys who have forgotten more football than the rest of us will know, are aware that they don't understand the details of the play.  And sometimes they're caught out by that (Warner calling out Reggie Gilliam as the first read on the throw where Knox got mugged)

 

 

Edited January 18 by Beck Water

[Einstein]

Just now, Beck Water said:

But I know it's been explained to you over and over that the data PFF pays for is not their grades.  It would be nice if you would acknowledge this.
 

I did address this. Here is the link: https://www.twobillsdrive.com/community/topic/252639-once-and-for-allpffffffffft/?do=findComment&comment=8886231

 

 

 

[Royale with Cheese]

1 minute ago, BullBuchanan said:

Man, all you had to do was read one more sentence.

 

When it's that big of a discrepancy, its legitimacy can be questioned.  

It's great that you defend PFF player grades but have no idea how they grade them.  

 

So when you have something as bizarre as Ed Oliver being graded as average, I think actually rated him below average and Jordan Davis performing well....I think it's pure BS.

 

In your opinion, outside of PFF, has Ed Oliver been below average this year in his play?

Is Brian Baldinger completely off his rocker with his assessment of Jordan Davis's play?

[hondo in seattle]

25 minutes ago, BullBuchanan said:

That doesn't make the system or analysis flawed. Every system has limitations. The flaw is using it to ascribe value that was never intended. You need to be real with what the data tells you and what it doesn't. It should be one piece of the puzzle. More often than not data should make you ask questions, not tell you answers. It's a very common mistake. Even c-level executives do it in fortune 500 companies - even when we tell them not to.

 

 

I do think analytics has a place in player evaluations.  And I believe it'll get better with time. 

 

But the bolded remark is a great observation.  

[BullBuchanan]

33 minutes ago, Royale with Cheese said:

 

When it's that big of a discrepancy, its legitimacy can be questioned.  

It's great that you defend PFF player grades but have no idea how they grade them.  

 

So when you have something as bizarre as Ed Oliver being graded as average, I think actually rated him below average and Jordan Davis performing well....I think it's pure BS.

 

In your opinion, outside of PFF, has Ed Oliver been below average this year in his play?

Is Brian Baldinger completely off his rocker with his assessment of Jordan Davis's play?

I don't have a professional assessment of Ed Oliver. I know he's made some great plays, but I haven't broken down his tape and looked every one of his plays to build an aggregate score. While he has made great plays, i do know he hasn't looked like prime Aaron Donald or JJ Watt. Not that that means anything.

As for Brian Baldinger's assessment. It's one guy with a super subjective opinion. Not sure if he's completely off in his assessment, because he doesn't really say that much that takes away from PFFs assesment. He can be a limited factor in pass rush due to his weight  and still perform well on his assignments without performing to potential or desired impact.

[Royale with Cheese]

16 minutes ago, BullBuchanan said:

I don't have a professional assessment of Ed Oliver. I know he's made some great plays, but I haven't broken down his tape and looked every one of his plays to build an aggregate score. While he has made great plays, i do know he hasn't looked like prime Aaron Donald or JJ Watt. Not that that means anything.

As for Brian Baldinger's assessment. It's one guy with a super subjective opinion. Not sure if he's completely off in his assessment, because he doesn't really say that much that takes away from PFFs assesment. He can be a limited factor in pass rush due to his weight  and still perform well on his assignments without performing to potential or desired impact.

 

Baldinger says he shouldn't be dressing.  So yes, it's completely the exact opposite of what PFF's assessment is.  Baldinger is saying he's useless, PFF says he's doing well.

[BullBuchanan]

Just now, Royale with Cheese said:

 

Baldinger says he shouldn't be dressing.  So yes, it's completely the exact opposite of what PFF's assessment is.  Baldinger is saying he's useless, PFF says he's doing well.

If you want to find a talking head saying a thing about any player, you can absolutely find it.

[Beck Water]

19 hours ago, Einstein said:

PFF employs former NFL players, especially at Tier 2 and 3 of grading. The reason why PFF grades take a couple day to update after a game is because they are not graded by one player. It gets graded by a Tier 1 analyst, then a Tier 2 analyst, and then sometimes a Tier 3 analyst. 

People will often write "they don't know the play call". These people clearly think that football is rocket science. It's not. We can see on All-22 what the play is. Especially by mid season when teams are running the same plays over and over. And no, we do not accept the idea that maybe this particular team told their left tackle to get pancaked on a counter-run because the coach told him to. No, he just got beat.

 

I already addressed your first and last sentence, but I'd like to address these points.  Again, you deserve a Shout! for your good Samaritan action taking care of a guy's snow removal crisis.  You still need to be responsive of people's points.

 

Let's delve into PFF graders and do some simple math.  PFF states they employ 600 full and part time analysts but less than 10% are trained to the level where they grade plays and 2-3% percent to the level of reviewing and finalizing grades.  OK, so that means they have less than 60 people grading plays and 12-18 people reviewing and finalizing them.  We'll use 60 and 12-18 in our assessment.

 

In a typical game there are average 63 offensive and 63 defensive plays.  That's 126 plays per game for each team, or 4032 plays per week.  Each of those plays involves 11 players PFF gonna grade, so 44352 player plays per week to grade.

 

If less than 10% of their 600 analysts grade plays, that means  each analyst is grading at least 739 plays over a span of a couple days.  For the data to be useful for teams, they would need the grades at latest Tuesday after the game - Monday for their own players, since they review film and do corrections on Monday; Tuesday for their next opponent since that's when the coaches create the game plan for the next week, which they install with the team Wednesday.

 

So less than 60 analysts would be reviewing 44352 plays in 24 hrs so that 12-18 people can review and finalize them.  Let's say they work a 12 hr shift (brutal, but people do it).  That would say they're reviewing 61.6 plays per hour.  They say "Each grade is reviewed at least once, and usually multiple times, using every camera angle available, including All-22 coaches’ tape."  2 All-22 views, usually 3 or 4 broadcast camera views, so what - 10 seconds per view?

 

Then there are those senior analysts.  Let's say they only review 25% of the plays or 11088 plays, as the other 75% are considered straightforward.  Again, to be useful to the team, these data are needed by Tuesday am at latest, so let's say we have 12 or 18 people reviewing those plays in another 12 hrs, that's 51 - 77 plays per hour.

 

Let's contrast this with the NFL, where, let's say on offense, the Bills average 65.5 plays per game on offense.  The OL coach has an assistant coach (and possibly an intern or another offensive assistant) and 5 players in on every play, so and his assistant have 328 plays to review and grade.  That's half the number of plays, for twice the number of people per play, by people who know the play call and the assignment to start with.  And that's the busiest room on offense.  The WR coaches are grading 180 plays per game since 2 or 3 WR sets.  The RB coach is grading an average of 71 plays per game since the Bills do very few 2 back sets.  etc.

 

Just on sheer arithmetic, it doesn't make sense to me that teams would rely on PFF to grade their players.

 

Now: to your statement "we can all see on All-22 what the play is".  I would like to remind you of a discussion that took place earlier this season between yourself, @Buffalo716, and @HoofHearted regarding what the defensive coverage was on a specific play thus what the assignment responsibilities were.  You're obviously someone who knows some ball, but does the phrase "I run the coverage you’re referring to and it’s not played the way you think it’s played. If you’d like we can go to PMs and I can teach it to you" ring a bell?  Yes, football isn't rocket science, but there are a lot of subtleties, and those subtleties impact what each player is actually supposed to be doing.  You even ran into that here about 2 months ago, so I'm not sure how (if you're intellectually honest in discussion) it's so challenging for you to acknowledge this.

The point is even guys who know ball and understand coverages can have to look at a play a couple times to understand what the coverage and thus the roles and responsibilities were supposed to be before zeroing in on how well each filled their assignment.

[Royale with Cheese]

4 minutes ago, BullBuchanan said:

If you want to find a talking head saying a thing about any player, you can absolutely find it.

 

"A thing".  Yes of course, you can find several million of them.

 

But to say someone is so bad they shouldn't be dressing?  We can find that?

Let me see if I can find a talking head saying that Aaron Donald is so ineffective that he shouldn't be dressing.  You think I can find it?

[BullBuchanan]

5 minutes ago, Royale with Cheese said:

 

"A thing".  Yes of course, you can find several million of them.

 

But to say someone is so bad they shouldn't be dressing?  We can find that?

Let me see if I can find a talking head saying that Aaron Donald is so ineffective that he shouldn't be dressing.  You think I can find it?

Probably yes. By the way, Davis had 2.5 sacks while only playing 45% of snaps. based on that alone, I'm guessing there's more than a tad of hyperbole coming from your boy.

Edited January 18 by BullBuchanan

[Royale with Cheese]

5 minutes ago, BullBuchanan said:

Probably yes. By the way, Davis had 2.5 sacks while only playing 45% of snaps. based on that alone, I'm guessing there's more than a tad of hyperbole coming from your boy.

Your evidence that he wasn't bad because he had 2.5 sacks?

 

Ed Oliver played 50% of snaps and had 9.5 sacks.

 

Let me guess....PFF is more accurate with Davis than they were with Oliver.  

 

You know Oliver played well this year but you won't come out and say it because of the stupid statement below.  Sticking to your guns.

 

On 6/4/2023 at 1:51 AM, BullBuchanan said:

it was 6.5, but that doesn't really matter. What matters is it's A LOT less money and a lot more production. Oliver has never even lived up to his rookie contract, and then they give him an top level extension. Total clown move from Beane. It's starting to look like they're already writing off this season and planning for 2024 and beyond. We're going to be fielding a team this year that has equal or lesser talent at nearly every position. Unless you thought we were a guard and a 220lb mlb away from a championship, of course.

 

 

[BullBuchanan]

2 minutes ago, Royale with Cheese said:

Your evidence that he wasn't bad because he had 2.5 sacks?

 

Ed Oliver played 50% of snaps and had 9.5 sacks.

 

Let me guess....PFF is more accurate with Davis than they were with Oliver.  

 

You know Oliver played well this year but you won't come out and say it because of the stupid statement below.  Sticking to your guns.

 

 

nope.

[Royale with Cheese]

3 minutes ago, BullBuchanan said:

nope.

I guess we will disagree about Oliver.

I think it was a great extension, you still think Beane is a clown for extending him.

[Scott7975]

1 hour ago, Einstein said:

I did address this. Here is the link: https://www.twobillsdrive.com/community/topic/252639-once-and-for-allpffffffffft/?do=findComment&comment=8886231

 

 

 

 

 

Thats not addressing it.  To me it just looks like you are doubling down on saying teams are paying for player grading.  This is like saying you paid for a 12 course fine dining meal for the Andes mint you get with the check.