Once and for all....PFFffffffft

[Jay_Fixit]

14 hours ago, Einstein said:

 

You really think NFL teams would pay millions of dollars for useless data? The idea that you think it's some schmuck sitting at home is part of the problem.

 

PFF employs former NFL players, especially at Tier 2 and 3 of grading. The reason why PFF grades take a couple day to update after a game is because they are not graded by one player. It gets graded by a Tier 1 analyst, then a Tier 2 analyst, and then sometimes a Tier 3 analyst. 

People will often write "they don't know the play call". These people clearly think that football is rocket science. It's not. We can see on All-22 what the play is. Especially by mid season when teams are running the same plays over and over. And no, we do not accept the idea that maybe this particular team told their left tackle to get pancaked on a counter-run because the coach told him to. No, he just got beat. 

The other excuse is "teams pay for the data, not the grades". How do you think PFF grades the player? On the data!

Ladies and gentlemen, I present Mrs PFF.

[4merper4mer]

The PFF ratings on the Thanksgiving game when Tre’ White got hurt, so healthy Tre’ White, had him ranked as the 98th best CB in the NFL.  I agree with JJ Watt about half as frequently as I agree with a broken clock but this is the time.

[boyst]

When pff was coming or years ago I had friends in sports management at BGSU. They offered jobs to students who passed a few basic tests to chart and grade players per game. These were students who may not even watch the game or didn't even know football.

 

Pff is ***** 

[4merper4mer]

1 minute ago, boyst said:

When pff was coming or years ago I had friends in sports management at BGSU. They offered jobs to students who passed a few basic tests to chart and grade players per game. These were students who may not even watch the game or didn't even know football.

 

Pff is ***** 

Bowling Green has an orange logo.  

[boyst]

33 minutes ago, 4merper4mer said:

Bowling Green has an orange logo.  

And brown!

[DCOrange]

13 hours ago, BlazinBill said:

IMO having a response to criticism like that just screams lacking belief in what you do-it’s really not productive. If PFF believes in what they do a simple no response says more- to me anyway .

 

 

It’s just an easy way for them to get more clicks.

[HoofHearted]

15 hours ago, Beck Water said:

I'm not a scout, but I'm not sure that's the way "most NFL coaches grade" and would welcome some input on this - any input from guys who've coached at the college or HS level @HoofHearted @Buffalo716?

 

Either way, I think the main point is that the coaches (and scouts) are typically people who have a deep and nuanced knowledge of the game.  They're people who can diagnose the defensive coverage based on subtle cues like footwork and body position.  We saw this kind of thing with Tony Romo when he started his broadcast career, where he could accurately read the defense and predict what the offense was going to do and the network told him to tone it down.  The ability of these people with a lifetime football background, playing and coaching, to deduce the play call, or the coverage, or the protection call, after looking at film, is on a different level.

 

It's my understanding that PFF uses a lot of graders in Ireland and now India whose understanding of the game is taking a few training sessions.  Now they claim that only 10% of their employees finalize grades or something like that, and that everything is reviewed, but 🤷‍♂️?

 

Some critiques of PFF:

https://sportank.com/media/post/is-pff-reliable-are-they-all-that-credible

 

I picked these because neither are out to just dismiss or trash PFF.

PFF grades are for fans. They serve no purpose to teams.

[BarleyNY]

17 hours ago, Einstein said:

Is this really surprising? 

You can have two NFL coaches grade a player completely differently.

Two GM's too.

That's why some will trade down in the draft and others will trade up to get the player that the other left there.

It doesn't mean PFF sucks. It means HIS coach graded it differently.

 

For ratings to be useful people have to understand what they really mean, which includes what they’re actually measuring as well as their limitations. I like a lot of what PFF does, but their grading has limited utility. It’s better with some positions than others, but is still limited to what players are asked to do rather than what they’re capable of. For instance, a QB like Cousins who plays a much more limited role in his team’s offense, can more easily score a higher grade than a QB like Allen, who much more is asked of.

 

Sam Monson recently mentioned this on a podcast when PFF’s grading came up. He used Dodson as an example of a player who graded very highly due to him doing well in a very limited role. It makes sense that coaches would be more interested in a player’s overall impact on a game and would factor in things like the difficulty of the assignment, the ability of the player, the contract of the player, etc. Each has their use and a value, but they aren’t going to be the same. 

[DCOrange]

13 hours ago, JoPoy88 said:

I 100% agree with you. If PFF put out more publicly on how these graders are trained and instructed, it might alleviate some of this bewilderment for some of their grades and hatred for them in general. But they don’t do that. Wonder why?

They have done this several times over the years. Unsurprisingly, most of the people (not saying you, just the general people that go crazy on Twitter and such) that complain about them don't really care to invest time in reading/listening/watching. The link below includes guides as to how they grade the different position groups.

 

It does not include answers regarding who is physically doing the grading, who reviews the graders, etc., but I believe that's been answered elsewhere as well. If I remember right there's like a three-level review process on every game.

 

https://profootballfocussupport.zendesk.com/hc/en-us/articles/360029567633-How-does-PFF-grade-each-position-

 

 

Edited January 18 by DCOrange

[JoPoy88]

10 minutes ago, DCOrange said:

They have done this several times over the years. Unsurprisingly, most of the people (not saying you, just the general people that go crazy on Twitter and such) that complain about them don't really care to invest time in reading/listening/watching. The link below includes guides as to how they grade the different position groups.

 

It does not include answers regarding who is physically doing the grading, who reviews the graders, etc., but I believe that's been answered elsewhere as well. If I remember right there's like a three-level review process on every game.

 

https://profootballfocussupport.zendesk.com/hc/en-us/articles/360029567633-How-does-PFF-grade-each-position-

 

 

I’ve read all this in the past, along with Matthew Coller’s book. The example clips they give do help add some understanding of it. 
 

Still need to know who’s doing these evaluations and how they are qualified and vetted. 
 

There are still too many head-scratching grades on a semi-regular basis with this outfit. 

30 minutes ago, HoofHearted said:

PFF grades are for fans. They serve no purpose to teams.

exactly. The player grades are an entertainment product. They’re glorified power rankings.

[DCOrange]

3 minutes ago, JoPoy88 said:

I’ve read all this in the past, along with Matthew Coller’s book. The example clips they give do help add some understanding of it. 
 

Still need to know who’s doing these evaluations and how they are qualified and vetted. 
 

There are still too many head-scratching grades on a semi-regular basis with this outfit. 

At the end of the day, I don't think it's wise for people to treat PFF grades as "gospel" as I believe JJ Watt said it. As best as they try, grading football players is ultimately a subjective exercise, regardless of if it's a nerd or a coach doing the grading. If you're willing to pay for what IMO is a very expensive subscription, I believe you are able to get a somewhat detailed breakdown of what went into each player's grade in a given week but that's just not an additional expense that I need in my budget lol.

 

Anecdotally, I feel like when there's occasional viral moments like this where they do come out and explain a specific rating, it generally makes sense. I'm sure there are plays that can be nitpicked but I appreciate when they show their work on stuff like Stroud vs. Love and that one game several years ago when Aaron Rodgers threw for like 400 yards and 5 TDs but received a pretty mediocre grade because it was basically all off of screen passes and busted coverages. I generally feel like I agree with or at least understand situations where a player has great box score stats but a relatively low grade (Jordan Phillips 9.5 sack season with us is one I remember off the top of my head). But obviously I'm only really looking at players that I care about; I can't say I have any idea if Peter Skoronski's run blocking grade is accurate or not.

[Einstein]

7 minutes ago, BarleyNY said:

 

For ratings to be useful people have to understand what they really mean, which includes what they’re actually measuring as well as their limitations. I like a lot of what PFF does, but their grading has limited utility. It’s better with some positions than others, but is still limited to what players are asked to do rather than what they’re capable of. For instance, a QB like Cousins who plays a much more limited role in his team’s offense, can more easily score a higher grade than a QB like Allen, who much more is asked of.

 

Sam Monson recently mentioned this on a podcast when PFF’s grading came up. He used Dodson as an example of a player who graded very highly due to him doing well in a very limited role. It makes sense that coaches would be more interested in a player’s overall impact on a game and would factor in things like the difficulty of the assignment, the ability of the player, the contract of the player, etc. Each has their use and a value, but they aren’t going to be the same. 

 

I’ve noticed that whenever it comes to PFF, people turn to confirmation bias. They will latch onto anything that proves their side in the debate. So let’s simply look at the facts.

 

”NFL teams don’t care about PFF grading” - Except every single NFL team pays PFF, as well as 80+ college teams. Guess they see PFF as a charity!

 

”PFF grades serve no purpose to teams” - Except teams and players use these grades in contract negotiations. “You'd be amazed that Pro Football Focus is used in almost every contract negotiation known to man. Because either the team likes our grade because it helps them negotiate, or the player likes the grade because it helps them negotiate. A lot of times, we get asked by agents and by teams, 'All right, give us the strength of this guy' or 'Give us the weakness of this guy to help us negotiate.'” - Chris Collinsworth

 

”Well yeah the teams pay PFF but it’s just for their data, not for their grades” - How do you think PFF grades? From their data…

 

“But they just hire a bunch of schmucks to grade” - The final grade is approved or denied by former NFL coaches. To quote PFF, “about 40 or 50 former coordinators and head coaches that have been in the league that have the final say. So we have four different people look at every single play that takes place, and the former coaches have all the veto power.”

 

I guess we could just ask some NFL coaches too… 

 

“Pro Football Focus has done a nice job with their stats and their analysis. I think they’re pretty accurate with things.” - Andy Reid, Chiefs head coach

 

“I think it’s impressive the work they do” - Gary Kubiak, head coach when the Denver Broncos won the Super Bowl

 

"I don't think there's any question that they do a great job, and I think they're a valuable resource for us." - Kyle Shanahan, San Francisco 49ers head coach

 

“We attributed all three special teams return TDs in 2017 to PFF data.” – FBS Special Teams Coach

 

 

 

So, let me put this all together and get this straight. NFL teams see no value in a product that pay a ton of money for, which is graded by former NFL coaches (which to my knowledge, no one on this forum is a former NFL coach), but the fans on this forum know better.

Ok.

 

[JoPoy88]

3 minutes ago, DCOrange said:

At the end of the day, I don't think it's wise for people to treat PFF grades as "gospel" as I believe JJ Watt said it. As best as they try, grading football players is ultimately a subjective exercise, regardless of if it's a nerd or a coach doing the grading. If you're willing to pay for what IMO is a very expensive subscription, I believe you are able to get a somewhat detailed breakdown of what went into each player's grade in a given week but that's just not an additional expense that I need in my budget lol.

 

Anecdotally, I feel like when there's occasional viral moments like this where they do come out and explain a specific rating, it generally makes sense. I'm sure there are plays that can be nitpicked but I appreciate when they show their work on stuff like Stroud vs. Love and that one game several years ago when Aaron Rodgers threw for like 400 yards and 5 TDs but received a pretty mediocre grade because it was basically all off of screen passes and busted coverages. I generally feel like I agree with or at least understand situations where a player has great box score stats but a relatively low grade (Jordan Phillips 9.5 sack season with us is one I remember off the top of my head). But obviously I'm only really looking at players that I care about; I can't say I have any idea if Peter Skoronski's run blocking grade is accurate or not.

That’s all perfectly fine - so long as it’s made clear that these things are purely subjective analyses. These are people watching tape and assigning grades on a scale that they created. 
 

My main problem with them is these two sides of their business are conflated together and now everything is shrouded under PFF’s data analytics. And they really don’t seem interested in speaking up and differentiating the two to the public. 

[Sierra Foothills]

17 hours ago, Punch said:

Not to defend PFF, but "literally putting a coaches grade next to a PFF grade" doesn't really prove much considering how many meatheads are coaching in the NFL. Coaches are constantly wrong when evaluating talent and situation. 

 

Yes player evaluation is highly subjective... even for NFL coaches.

 

17 hours ago, Einstein said:

Is this really surprising? 

You can have two NFL coaches grade a player completely differently.

Two GM's too.

That's why some will trade down in the draft and others will trade up to get the player that the other left there.

It doesn't mean PFF sucks. It means HIS coach graded it differently.

 

Subjective even for NFLscouts and GMs.

 

17 hours ago, Buffalo716 said:

Their grades are highly subjective because they do not know the play call

 

Pro scouts who work for NFL teams and scouting syndicates also don't usually know the play call.

 

16 hours ago, MJS said:

Wow! I'm a huge fan of JJ Watt now. Absolutely savage. I love it.

 

Dude is a meathead and he spurned the Bills so strikes 2 and 3.

 

16 hours ago, eball said:

You really think some schmuck sitting at home "grading" a play when he doesn't know the play call or the individual player responsibilities is the same as a coach grading a play?

 

1) One can't assume the competency of PFF evaluators. Some have worked at high levels in the NFL as coaches and players.

 

2) Your argument dictates that only the handful of people who know the play call are justified in judging the results. This argument then disqualifies NFL coaches and scouts who usually don't know the play call.

 

16 hours ago, BADOLBILZ said:

Just like scouts grades.    Totally subjective because they do not know the play call.  

 

Exactly.

 

16 hours ago, Einstein said:

to be honest, I disagree with several of PFF grade. Ed is just one of them.

but keep in mind that two NFL coaches may disagree with each other on their own grading.so, in my opinion, that’s not that unusual.

 

Correct.

 

15 hours ago, Mango said:

I don't disagree on this topic, but JJ Watt is also a dummy. 

I have no other value to offer to this conversation. 

 

JJ Watt is Gronk without the rings.

 

14 hours ago, Punch said:

Meanwhile, PFF grades are "molded by the NFL" as our old friend Matthew Coller tweeted (I'll reiterate for the third time, I do not necessarily wholeheartedly endorse PFF):

 

 

Andre Whitworth, Paul Alexander and Bobby Slowik? Not bad.

 

Critics of PFF are assuming the caliber of the evaluators without actually knowing who they are.

 

14 hours ago, hondo in seattle said:

The play a couple of weeks ago when Josh threw to Gabe in the endzone and was picked...  How do you grade that?  Did Josh just make a bonehead play under duress?  Or did Gabe go to the wrong spot based on the defense?   How does the PFF grader know?  Even if the grader is a former All-Pro, he can't know.  He can only guess.

 

So PFF is the only entity capable of this type of error?

 

How about QC coaches and advance scouts? Most scouting evaluations, including those done by NFL scouts are done without knowing for sure each players assignment.

 

The same goes for esteemed and highly-credible evaluators like Greg Cosell.

 

Critics of PFF seem like they don't understand the intrinsic nature of player evaluation... or maybe they just want to complain about something.

 

 

[Sierra Foothills]

8 minutes ago, JoPoy88 said:

That’s all perfectly fine - so long as it’s made clear that these things are purely subjective analyses. These are people watching tape and assigning grades on a scale that they created. 
 

My main problem with them is these two sides of their business are conflated together and now everything is shrouded under PFF’s data analytics. And they really don’t seem interested in speaking up and differentiating the two to the public. 

 

To the bolded... if that has to be explained to people they most likely won't understand the explanation.

 

[BADOLBILZ]

1 hour ago, HoofHearted said:

PFF grades are for fans. They serve no purpose to teams.

 

As they shouldn't.    Teams pay a whole lotta' people whose professional ceiling without the NFL is otherwise "gym teacher" to do their own research and offer their own opinions.

 

Every industry has this.   In my experience, the few coaches, scouts and the GM I've known..........not particularly smart people and often with very poor recollection.   But in some cases they've had great success.  

 

When you are IN an industry you have an understanding that from within all that matters is your results there.   I've experienced it too.   People who couldn't make a dime in any of the areas I've had success in often knew more about the minutia of them than I have.   That was worthless to me........but I didn't have to defend myself against it like people in the NFL do.

 

 And PFF grades are all about that minutia.   As Monson said,  they don't grade like coaches.   

 

It's also why a meathead like JJ Watt can rant about the only bad plays are plays that have ACTUAL bad results.   Avid fans often have very unrealistic expectations of these people in the NFL.   It's not life and death or rocket science.   It's entertainment.   The plans are slapped together with purpose but hastily from week to week.  The sample sizes are small. And frankly,  these aspects help make it so unpredictable and entertaining.  

 

As I said earlier..........we should all just be glad they don't play 162 games each.   Because THEN the data gets real and changes everything.

 

[hondo in seattle]

55 minutes ago, Sierra Foothills said:

...

 

Critics of PFF seem like they don't understand the intrinsic nature of player evaluation... or maybe they just want to complain about something.

 

 

 

So, wait...  You're saying your smarter than us PFF critics because we don't understand "the intrinsic nature of player evaluation" but you do?   I'm not sure why you're taking low shots at the intelligence of fellow mafioso, but I disagree.

 

Most of us don't like the notion that you can reduce a player to a set of numbers.  Let me give another example.  Maybe Diggs is playing with broken ribs.  The injury will lower his grade.  But it won't lower the way a coach evaluates him as a player.  PFF lacks context.  Their graders don't know what the position coaches have been telling the player, the playcall, the assignment, the assignment of the guy next to the graded player which may effect his decision-making, his injury status, and so on.  

 

Additionally, how PFF grades each play and how they weigh (or don't weigh) different things is ultimately subjective.   Let's talk about weighting for a moment.  Let's say the Bills call a run to the right.  Shakir is lined up on the left and completely muffs his block. So PFF gives him a -2 grade, even though it didn't impact the run.  The next play Shakir makes a tremendous one-handed grab and dances through the defense for a dazzling 63-yard TD that wins the game with just seconds left.   PFF grades him a +2, their highest grade.  Shakir's PFF average for those two plays is ZERO.  But a coach would call him the hero of the game.  Weighting matters. 

 

The idea that you can entirely remove subjectivity from football evaluation is absurd.  Maybe PFF knows this, maybe not.  I don't care.  What I know is that their attempt to reduce players to a metric is flawed.  And while many of their grades seem accurate, some clearly are not.  

 

I don't hate PFF for trying.  In fact, when the Bills acquire a FA, one of the first things I do is look at his PFF grade for a ballpark estimate of his value.  But if I wanted a better evaluation, I'd talk to a coach.  

 

Edited January 18 by hondo in seattle

[Shaw66]

Man, I freakin love this!   I'm so happy to hear and see Watt saying that.   

 

I don't remember if it was on video, but I remember seeing or reading a piece of an interview with Kyle Williams when someone asked about his or someone else's rating on PFF or some other system.  First, Kyle said, "What's that?"   He didn't even know any of these outside rating systems existed.   So, someone explained to him what it was, and Kyle said, "Are you kidding?"

 

He went to say that it was totally absurd to think that some people outside the team reviewing game film can evaluate a player's performance.   He said they don't know the play call, they don't know the adjustments that may have been made during the week, they don't know the audibles, they don't the assignment on the play call, the weekly-adjusted play call, or the audibled play.   They don't know anything about what the player is actually supposed to be doing to make the play work, and they want us to believe that their rating actually means something.   How can it possibly mean something if they don't know what the player was supposed to do in the first place?

 

Kyle was incredulous.   And that's exactly what Watt is saying here.  Good for him. 

[BullBuchanan]

16 hours ago, Royale with Cheese said:

 
Okay so you with PFF grade that Ed Oliver has been below average this year. 

They have Jordan Davis as a top 10 DT and he’s been terrible this year.

 

Ed Oliver, below average.

 

15 hours ago, JoPoy88 said:

There is nothing empirical about PFF player grades. Jesus Christ.

 

Edit: and this is the biggest problem with PFF - they’re liars (or at least, manipulators.) PFF holds themselves out as leaders in analytical football data. and they do in fact do a lot of that - that’s what they sell to teams. But PFF player grades are not that, never have been. That’s their fan facing product. That’s to drive those clicks and engagements. And they rely on fans like ol’ Bull here, who’s never met a topic he knew nothing about he wouldn’t comment on, to not know the difference. And they’re succeeding.

And here lies the problem. PFF isn't saying a player is good or bad. They are providing a grade based on what a player does on each play. It's YOU and others who are ascribing a meaning to that grade.

PFF isn't saying that Jordan Davis is a top 10 DT. They are saying that within their grading system he has the 10th highest score. It's not at all a complete measure of a football player. It's one data point that measures the impact of each and every play of a player. It's entirely possible for you to think that Jordan Davis is "terrible" because of whatever reasons, while still having a higher aggregate score using PFFs metrics than most other players. If you were forced to break down why you thought Davis was terrible this year, I'm sure you'd find an answer as to why he still has a high PFF grade

This isn't just PFF. It goes for all data from all sources. You can't just take two results from a data source and automatically say that the higher result is the better thing.

As for what I know and don't know - I know how PFF says they score players and I work with data and insights professionally, so I think that's probably more experience than most people have. It doesn't make me an expert in PFF, but the types of foaming at the mouth posts raging against PFF don't require me to be an expert.

[Matt_In_NH]

19 hours ago, Punch said:

Not PFF specific, but it seemed almost the entire analytics crowd hated Josh Allen when he was coming out of college. Now, by large, the analytics community is constantly defending him whereas most of the criticism he receives comes almost exclusively from former players and talking heads--- typically, the least informed sports commentators wrt anything but the most casual biased stereotypes.

Talking heads get paid because they have an opinion they back up emphatically and will use whatever data they have to support it while ignoring data that refutes their point.  Unfortunately that approach works and people love to respond to it and talk about the take as right or wrong.  But in doing so it encourages more  of the same.  It’s like the guy I. The bar that just shoots from the hip with a string opinion.   It’s an opinion and lacks substance but it gets a reaction and that pays.