Interesting article regarding analytical grading sites like PFF

[CorkScrewHill]

Details how Raiders players moved away from caring about these site’s grades when they realized it didn’t align with how their coaches were grading them based on the analytics site not understanding what their actual responsibilities were on a given play.

 

https://sports.yahoo.com/raiders-focus-coaches-grades-rather-005549328.html

 

I know this has been debated back and forth (especially given Lewan’s contention that players pay to increase their ratings, but the part about the graders not really understanding the individual’s responsibilities on a given play hit home to me.

[TH3]

Definitely paying off for them

[eball]

Yeah, this is what a lot of us have been saying for a while.  I think the negative grades are much more subject to criticism, but I’ve even seen some positive grades I’ve questioned after watching the game.

 

I just don’t think there is any accountability by these services as to the quality of their analysts — many of whom have other full time jobs and “review” game film as a side hustle.

 

[LyndonvilleBill]

Maybe I'm off, but when I look at the grades, I don't look at them individually.  I only look at them to compare. So 47.2 means nothing individually,  but if you compare 2 players at the same position and one is at 70 and the other at 50, it does give an idea of who played better. 🤔

[djp14150]

2 hours ago, CorkScrewHill said:

Details how Raiders players moved away from caring about these site’s grades when they realized it didn’t align with how their coaches were grading them based on the analytics site not understanding what their actual responsibilities were on a given play.

 

https://sports.yahoo.com/raiders-focus-coaches-grades-rather-005549328.html

 

I know this has been debated back and forth (especially given Lewan’s contention that players pay to increase their ratings, but the part about the graders not really understanding the individual’s responsibilities on a given play hit home to me.

 

this is the problem in sports analytics.  Team sports of football and hockey have real problems in isolating responsibility.

 

thr Bille standard is using just 2LBs and %5 DBs so this can screw up these general analysis models

 

sure you can measure parts of th3 game likeQB, RB based on where they run, WR and their catches and routes.  In OL andD it’s much harder because the6 need to work together. No matter what D you call, there is an offensive play that can exploit it.

 

shead of the play, you do t know they’d call so if a receiver is open in a zone then who is to blame or if theCB is playing a zone releasing a receiver toa safety, but the safety doesn’t cover the receiver then who do you blame for that?

[CorkScrewHill]

51 minutes ago, LyndonvilleBill said:

Maybe I'm off, but when I look at the grades, I don't look at them individually.  I only look at them to compare. So 47.2 means nothing individually,  but if you compare 2 players at the same position and one is at 70 and the other at 50, it does give an idea of who played better. 🤔

The problem is as pointed out in the article the grade is made without the context of knowing what the players assignment was on the various plays. If the grader guesses the assignment correctly the scoring would be fine, but in many cases they are in fact just guessing so comparing grades would not work

[Dibs]

Never forget that analytics always starts with anal.

Edited October 27, 2022 by Dibs

[Big Turk]

2 hours ago, eball said:

Yeah, this is what a lot of us have been saying for a while.  I think the negative grades are much more subject to criticism, but I’ve even seen some positive grades I’ve questioned after watching the game.

 

I just don’t think there is any accountability by these services as to the quality of their analysts — many of whom have other full time jobs and “review” game film as a side hustle.

 

 

Agreed. And with so many foreigners who likely didn't grow up watching or understanding football how accurate can they be?

 

Would be like me starting to watch rugby which I've never played or really even watched and then start trying to grade players

 

Edited October 27, 2022 by Big Turk

[Beck Water]

3 hours ago, CorkScrewHill said:

Details how Raiders players moved away from caring about these site’s grades when they realized it didn’t align with how their coaches were grading them based on the analytics site not understanding what their actual responsibilities were on a given play.

 

https://sports.yahoo.com/raiders-focus-coaches-grades-rather-005549328.html

 

I know this has been debated back and forth (especially given Lewan’s contention that players pay to increase their ratings, but the part about the graders not really understanding the individual’s responsibilities on a given play hit home to me.

 

That's something some of us here have been saying for years.

 

PFF touts that 31 of 32 teams (or maybe it's 32 of 32) use their data, but the data that teams use is probably data about tendencies - what formations in different down and distance and game situations, what plays are called from those formations, how often and when a LB drops into coverage or blitzes, that sort of thing that a person given a short training course can actually determine accurately - not their player grades.  And those data are probably good, and help teams by allowing them to put manpower into decoding tactics and not into grunt work.

 

I can't find it, but Eric Wood did an interview (shortly after he retired) talking about how especially on the OL, those grades chap their grits because they're just not accurate.  He gave specific examples of a play where Cog blew his assignment, Wood tried to pick it up and whiffed, the coaches knew who screwed up but Cog laughed that Wood would get the blame for his mistake (the Bills won, so they could laugh).  Sure enough, Wood was graded poorly by the analytics sites.

 

 

 

Edited October 27, 2022 by Beck Water

[US Egg]

Random ratings and overkill stats add nothing, nor detract, from my enjoying the NFL product.

[Bleeding Bills Blue]

5 minutes ago, Beck Water said:

 

That's something some of us here have been saying for years.

 

PFF touts that 31 of 32 teams (or maybe it's 32 of 32) use their data, but the data that teams use is probably data about tendencies - what formations in different down and distance and game situations, what plays are called from those formations, how often and when a LB drops into coverage or blitzes, that sort of thing that a person given a short training course can actually determine accurately - not their player grades.  And those data are probably good, and help teams by allowing them to put manpower into decoding tactics and not into grunt work.

 

I can't find it, but Eric Wood did an interview (shortly after he retired) talking about how especially on the OL, those grades chap their grits because they're just not accurate.  He gave specific examples of a play where Cog blew his assignment, Wood tried to pick it up and whiffed, the coaches knew who screwed up but Cog laughed that Wood would get the blame for his mistake (the Bills won, so they could laugh).  Sure enough, Wood was graded poorly by the analytics sites.

 

 

 

 

Right - when a guy comes in unblocked...  Is it the center for setting incorrect protection?  Did someone miss their assignment or call?  Did the QB mis-read the blitz, or not know that guy was the hot?  

[Thurman#1]

He's saying that the players have access to better grading, in that they can find out from their coaches how well they think they're doing. 

Duh. Of course they should focus more on what the coaches think.

 

Thing is, while those players have access to their coaches' grades, we don't. Nor do even the other teams.

 

Again, every team in the NFL subscribes to PFF for their info. If you can't find out what a guy's coaches think, you have to rely on film study and those who do it. PFF does it pretty well, or the teams wouldn't subscribe. 

 

Does that mean you have to believe every single thing they put out? Hell, no. Some of their pundit commentary in particular is pretty weird stuff. But their grades are pretty good, basically.

[Thurman#1]

9 minutes ago, Bleeding Bills Blue said:

 

Right - when a guy comes in unblocked...  Is it the center for setting incorrect protection?  Did someone miss their assignment or call?  Did the QB mis-read the blitz, or not know that guy was the hot?  

 

 

You can generally tell most of that stuff with thoughtful film study. Not with 100% accuracy of course, but with quite good accuracy. If you couldn't do that, teams wouldn't bother studying tape.

 

It's hindsight and every single instant of the play is on tape. 

 

If a guy comes unblocked and an unengaged lineman lunges at him but misses and all the other linemen are engaged, it's not difficult to figure out what goes on, for instance. Most times looking at elimination, you can get a very good idea. Not always. The coaches of that team do certainly know better than anyone else. I wonder why guys were listening to anyone else in the first place, with access to the coaches?

[MJS]

1 hour ago, djp14150 said:

 

this is the problem in sports analytics.  Team sports of football and hockey have real problems in isolating responsibility.

 

thr Bille standard is using just 2LBs and %5 DBs so this can screw up these general analysis models

 

sure you can measure parts of th3 game likeQB, RB based on where they run, WR and their catches and routes.  In OL andD it’s much harder because the6 need to work together. No matter what D you call, there is an offensive play that can exploit it.

 

shead of the play, you do t know they’d call so if a receiver is open in a zone then who is to blame or if theCB is playing a zone releasing a receiver toa safety, but the safety doesn’t cover the receiver then who do you blame for that?

So, think about Tremaine Edmunds. He leads the defense and gets everyone lined up correctly, communicates checks, etc. Then maybe Matt Milano makes a great tackle, but he was in the right position because of Edmunds. That doesn't show up as Edmunds actually doing anything. In fact, he might get a bad grade on that play even though he set up the good play with his recognition and leadership.

 

There is just way too much teamwork going on to really get a good enough idea to assign a grade based on how a player is performing from a bird's eye view.

 

Also, it really bothers me that they use their ridiculous numbering system. They are looking at a player and assigning a number. They use a quantitative measurement to express their qualitative analysis. To the point they are even using decimal places. It is no different than looking at a player and assigning gold stars, but it looks more legitimate by using numbers, as if they are using some scientific method of analysis, when all they are doing is looking at it and arbitrarily assigning a number. It is ridiculous and misleading. They should have stuck to letter grades like others use, because that is more aligned to what they are actually doing.

[MJS]

21 minutes ago, Thurman#1 said:

 

 

You can generally tell most of that stuff with thoughtful film study. Not with 100% accuracy of course, but with quite good accuracy. If you couldn't do that, teams wouldn't bother studying tape.

 

It's hindsight and every single instant of the play is on tape. 

 

If a guy comes unblocked and an unengaged lineman lunges at him but misses and all the other linemen are engaged, it's not difficult to figure out what goes on, for instance. Most times looking at elimination, you can get a very good idea. Not always. The coaches of that team do certainly know better than anyone else. I wonder why guys were listening to anyone else in the first place, with access to the coaches?

The team studies film of themselves because they know the assignments and exactly what they did right and wrong. They watch film on opponents to find tendencies, like what an individual might do differently on a pass play versus a run play, or what a team likes to do on certain down and distances, or how hard a player might react to play action or a pump fake, etc. What presnap movement or personnel might be telling them about the play being called. What types of pass rush moves a player uses, or what stunts they like to call. They aren't watching the film to see how good or bad a player is and to assign a grade. They don't need to know the opponent's individual assignments to look at that kind of stuff.

Edited October 27, 2022 by MJS

[Tortured Soul]

Even that article says the Raiders use outside grades in free agency, meaning that, when the organization doesn't have access to inside info, those grades are as good as anything the team can do itself. Suggests value to me.

[djp14150]

27 minutes ago, MJS said:

The team studies film of themselves because they know the assignments and exactly what they did right and wrong. They watch film on opponents to find tendencies, like what an individual might do differently on a pass play versus a run play, or what a team likes to do on certain down and distances, or how hard a player might react to play action or a pump fake, etc. What presnap movement or personnel might be telling them about the play being called. What types of pass rush moves a player uses, or what stunts they like to call. They aren't watching the film to see how good or bad a player is and to assign a grade. They don't need to know the opponent's individual assignments to look at that kind of stuff.

 

i agree

 

you watch other teams tapeyou look for patterns or trends.

 

on offense you might see tendencies a lineman does if it’s a pass or run.  If you have the same set with one a pass and one a run are there small signals thr players do to signal one.

 

in their defensive schemes you can see some patterns in man vs zone, one high vs two high safety.

 

they do look at individual players to see patterns.  If Miller lines up on the right side vs the left side.  If a CB is on the right side or left side. A teams top CB could be playing against Diggs when it’s man scheme but not when it’s zone.

 

Is the OLB lined up relative to the end of the line could say something on their D scheme.

 

 

[Mr. WEO]

4 hours ago, CorkScrewHill said:

Details how Raiders players moved away from caring about these site’s grades when they realized it didn’t align with how their coaches were grading them based on the analytics site not understanding what their actual responsibilities were on a given play.

 

https://sports.yahoo.com/raiders-focus-coaches-grades-rather-005549328.html

 

I know this has been debated back and forth (especially given Lewan’s contention that players pay to increase their ratings, but the part about the graders not really understanding the individual’s responsibilities on a given play hit home to me.

 

 

Well the Raiders suck, so how are their coaches rating them?  Participation ribbons and orange slices for them all?

[LyndonvilleBill]

2 hours ago, CorkScrewHill said:

The problem is as pointed out in the article the grade is made without the context of knowing what the players assignment was on the various plays. If the grader guesses the assignment correctly the scoring would be fine, but in many cases they are in fact just guessing so comparing grades would not work

Maybe, but if you go to the PFF site they explain this as well. Nobodies grades will be perfect as there are too many variables.

 

https://www.pff.com/grades

 

 

[K-9]

As I said last week the sheer enormity of the task and the time constraints involved don’t lend itself to quality work.
 

PFF has around 60 graders according to their own website. Given an average of 60 plays per team per game on each side of the ball, those 60 people have to analyze over 84,400 individual player snaps each week. As a result, I just can’t take their analyses with anything less than a huge boulder of salt.