Case Study: 1,000 pass attempts determines a young QBs career path

[Ramius]

Good work. Some interesting analysis that I will file in the memory to use at a later date when thinking about QB's.

 

The only question I have is rather complex. If a QB like Chad Pennington gets his 1,000 passes in how does he compare to a guy like Carson Palmer who got 1,000 passes at probably the same rate? Or both of those guys compare to a Mark Sanchez? Philip Rivers?

 

I ask because I am curious how far after 1,000 passes did some of these "starters" last? By all accounts Pennington, Palmer and Sanchez lost out in their starting position earlier then many expected them to when they likely hit their 1,000 mark.

Good work. Some interesting analysis that I will file in the memory to use at a later date when thinking about QB's.

 

The only question I have is rather complex. If a QB like Chad Pennington gets his 1,000 passes in how does he compare to a guy like Carson Palmer who got 1,000 passes at probably the same rate? Or both of those guys compare to a Mark Sanchez? Philip Rivers?

 

I ask because I am curious how far after 1,000 passes did some of these "starters" last? By all accounts Pennington, Palmer and Sanchez lost out in their starting position earlier then many expected them to when they likely hit their 1,000 mark.

 

Pennington hit 1,000 attempts in his 5th season, where he had a 91.0 rating. Pennington's career rating with close to 2500 attempts is 90.1 If i get some time in the future, perhaps i'll look up the number of total attempts thrown by QBs. Sanchez has close to 1900 attempts. But you're right, he hit 1,000 sttempts during his 3rd season and didn't last past his 4th. Rivers has thrown 4100 attempts.

 

I was trying to stress that you really need to see 1,000 sttempts to get a good look at a young QB, because how he performs in the season prior doesn't necessarily mean he's going to be good/bad.

 

 

Thanks for working the numbers.

 

My question is what is the percentage of QBs that make the 1000 attempts threshold? Clearly, there are backup QBs that never get any opportunity. But, what I'm asking about is the percentage of guys like Trent Edwards and JP Losman, who never reached the 1000 attempts threshold. And, I don't think many people would argue that they need to see him start another season to realize what they got in Trent.

 

PS: Matt Leinart is another one-time NFL starter that didn't make the plateau. (OTOH, Vince Young and Tarvaris Jackson both did, the latter in his 6th season.)

 

I had 28 QBs by my count (now up to 31, because i realized i had forgotten Ponder, so i had 29 QBs originally) and from 2001-2010 there were 145 QBs drafted. So 21% of the QBs threw 1,000 attempts.

 

I tried to look at QBs that at least started somewhat consistently, so their "year" ratings consisted of a large number of pass attempts.. Tavaris Jackson just crossed the 1,000 attempt threshold in his 6th season (14 starts with seattle). He had a rating of 79.2 that year, to go with his career rating of 78.5. He's a tougher case because he didn't start consistently, so its tough to assign a "prior year" rating when he only threw 58 pass attempts. I also forgot Kyle Orton.

 

Adding in those 2 QBs really doesn't change the correlation. It's still a 0.71.

 

Fitzpatrick was the only Bills QB to hit the 1,000 attempt plateau.

[3rdand12]

Though I am not one to critique any of the methods, or the revelations gleaned from such data ..

I think 1000 tosses is a nice round figure to cover most peaks and valleys for a rookie coming up .

And i can appreciate the underlying but specific message statement for us gnostics.

or is it agnostics .

 

i ferget

Edited February 27, 2014 by 3rdand12

[dubs]

 

Kind of like that, except for Ramius actually did his own research instead of cherry picking others' to generate a pop science theory and make himself the darling of a certain set of intellectuals.

 

Haha. Well, I think Gladwell's point was that practice and reputation is a larger influence in success than given credit for. I'd argue that it got blown out of proportion by others.

 

To liken it to this was only to say that both theories would suggest that it takes a while for people to reach their plateau, 1000 pass attempts, 10,000 hours. But thanks for arguing!

[CardinalScotts]

great research and interesting read...BUT...so many things can change from head coaches to offensive coordinators, good defense bad defense, easy schedule tough schedule. MLB scouts say it takes 1500 at bats to tell what type of player someone is going to be- THAT I agree with

[RuntheDamnBall]

Haha. Well, I think Gladwell's point was that practice and reputation is a larger influence in success than given credit for. I'd argue that it got blown out of proportion by others.

 

To liken it to this was only to say that both theories would suggest that it takes a while for people to reach their plateau, 1000 pass attempts, 10,000 hours. But thanks for arguing!

No doubt. I work in public radio and it's enervating to hear them tease the next segment where Mr. Gladwell is going to educate us about why creativity happens.

 

I think I'd rather listen to a belt sander for 30 minutes.

[justnzane]

What college did you learn quantitative methods at so I can take it off the list of potential college for my kids?!!!! The value of r can be between -1 and 1. Values closer to -1 and 1 are considered strong negative or positive correlations, respectively. It its generally accepted than any score less than -.70 or greater than .70 have very strong negative or positive correlation, respectively. So you might want to brush up on your stats a bit.

 

Outside of that, I appreciate this type of thinking as data junkie/analyst myself. The only issue I would have is that you're not measuring a "career" properly. I know this was a quick-and-dirty idea and it leads to bigger questions, but it might be best to do this with the population of retired QBs (or retired QBs since 1980 or justifiable selection to capture the more recent NFL) rather than currently active QBs. In this manner, the results would be more robust. Anyway, I think the findings this way might not see an r as high but would still be strong.

While you are correct that .71 would be considered a stronger correlation, it is hardly definitive. You may raise a good point that using data from retired QB's may be useful; however, I'd have to say that longevity of a career may also skew the data compounded with changes in rules that alter the passing game. For instance, Peyton Manning's 2nd season rating was only a 90.7 while his career rating is a 97.2. Thus, there is a deviation from the mean of 6.7% which is quite large. Because of rule changes, I'd bet that it would be best just to focus on draftees of the last 6 years for a valid comparison.

[MDH]

Because of rule changes, I'd bet that it would be best just to focus on draftees of the last 6 years for a valid comparison.

 

Yeah, but you're not going to have much data to work with if you stick with only the last 6 years. And by the time you do have enough data to make a meaningful analysis more rule changes will likely have occurred. I think Ramius hit the sweet spot in picking when to set the cut off. Nice work Ramius, I appreciate your work.

[HamSandwhich]

It’s amazing this forum, I’ll tell ya. Everyone now is a research scientist and is arguing that because they have some remembrance of correlation and how it is interpreted, their opinion is unquestionable. But there are two problems here 1) the strength of the correlation coefficient, across the hard and soft sciences, although sometimes using different terms, has the same interpretation despite the above-comment to the effect that it won’t get you published “much” because it’s “generally” not accepted (by the way that statement is internally contradictory with your argument because it indicates that in some cases works are published with these scores, thereby invalidating your argument that). That wasn’t the argument anyway; the argument is the interpretation of the value of r. I didn’t say it was “very strong” I said it was strong and it took me all of 3 minutes to find two sources in the so-called hard sciences that suggest anything from .70 to .89 is consider “high” (Asuero et al, 2006, journal: Critical Reviews in Analytical Chemistry) and .70 to .89 “strong” correlation (Fowler et al, 2005, Practical Statistics for Field Biology). Nice try with the attempted personal misdirection that insinuates my kids aren’t good enough for the hard sciences as a way to shield your mistake. If you want to talk precision and stats, then write what you mean and say it without terms like “generally” and “much” as a way to not commit to a stance you seem committed to. You won’t get published at all with errors. 2) With everyone focused on the interpretation of r to flex their stats muscle, no one wanted to take a shot at his research design, which tells me a lot. It tells me you took a stats course once but have minimal understanding of the limitations of research design, and in particular this one. I’m not here to slam it, I think it’s an excellent exploratory analysis that leaves enough questions for further development and I applaud the OP. This is where your mettle as a researcher should have been focused, not on what is an obvious continual debate over the interpretation of r, which we’ll never agree.

 

I talked about the limitations of the design. Look back

[Sisyphean Bills]

I had 28 QBs by my count (now up to 31, because i realized i had forgotten Ponder, so i had 29 QBs originally) and from 2001-2010 there were 145 QBs drafted. So 21% of the QBs threw 1,000 attempts.

 

I tried to look at QBs that at least started somewhat consistently, so their "year" ratings consisted of a large number of pass attempts.. Tavaris Jackson just crossed the 1,000 attempt threshold in his 6th season (14 starts with seattle). He had a rating of 79.2 that year, to go with his career rating of 78.5. He's a tougher case because he didn't start consistently, so its tough to assign a "prior year" rating when he only threw 58 pass attempts. I also forgot Kyle Orton.

 

Adding in those 2 QBs really doesn't change the correlation. It's still a 0.71.

 

Fitzpatrick was the only Bills QB to hit the 1,000 attempt plateau.

 

I guess what I am driving at is why the magic number 1000?

 

To get to 1000, the guy has to play. He also has to play for several seasons. (At least 2, even in a pass-ecstatic scheme.) Getting to the plateau, it seems logical that the QB has to "show something". He must be minimally competent, his organization has to be invested and back him, his team has to win games and have a modicum of stability, etc.

 

As an analogy, it is like reaching the base camp. Not everyone can get there and there is a multitude of factors why. Also, not everyone that gets to base camp ascends the rest of the way to the summit. Your correlation suggests that how one did reaching base camp is 0.71 correlated to his mountain climbing career. On the other hand, (and I know you know this) one shouldn't take the result to mean it must be applied to an individual. That is not everyone needs to be given the opportunity to hike to base camp, including (oh, I don't know, let's say...) a chain-smoking, one-legged, naked, blind man so that one can properly conclude if he can climb to the summit.

 

PS: I consider it very likely that EJ Manuel is given every opportunity to make this plateau with the Bills.

[Mr. WEO]

Pennington hit 1,000 attempts in his 5th season, where he had a 91.0 rating. Pennington's career rating with close to 2500 attempts is 90.1 If i get some time in the future, perhaps i'll look up the number of total attempts thrown by QBs. Sanchez has close to 1900 attempts. But you're right, he hit 1,000 sttempts during his 3rd season and didn't last past his 4th. Rivers has thrown 4100 attempts.

 

I was trying to stress that you really need to see 1,000 sttempts to get a good look at a young QB, because how he performs in the season prior doesn't necessarily mean he's going to be good/bad.

 

 

 

 

I had 28 QBs by my count (now up to 31, because i realized i had forgotten Ponder, so i had 29 QBs originally) and from 2001-2010 there were 145 QBs drafted. So 21% of the QBs threw 1,000 attempts.

 

I tried to look at QBs that at least started somewhat consistently, so their "year" ratings consisted of a large number of pass attempts.. Tavaris Jackson just crossed the 1,000 attempt threshold in his 6th season (14 starts with seattle). He had a rating of 79.2 that year, to go with his career rating of 78.5. He's a tougher case because he didn't start consistently, so its tough to assign a "prior year" rating when he only threw 58 pass attempts. I also forgot Kyle Orton.

 

Adding in those 2 QBs really doesn't change the correlation. It's still a 0.71.

 

Fitzpatrick was the only Bills QB to hit the 1,000 attempt plateau.

 

There were only 28 QBs between 2000-10 who attempted 1000 or more passes?

[Cash]

I guess what I am driving at is why the magic number 1000?

 

To get to 1000, the guy has to play. He also has to play for several seasons. (At least 2, even in a pass-ecstatic scheme.) Getting to the plateau, it seems logical that the QB has to "show something". He must be minimally competent, his organization has to be invested and back him, his team has to win games and have a modicum of stability, etc.

 

As an analogy, it is like a reaching the base camp. Not everyone can get there and there is a multitude of factors why. Also, not everyone that gets to base camp ascends the rest of the way to the summit. Your correlation suggests that how one did reaching base camp is 0.71 correlated to his mountain climbing career. On the other hand, (and I know you know this) one shouldn't take the result to mean it must be applied to an individual. That is not everyone needs to be given the opportunity to hike to base camp, including (oh, I don't know, let's say...) a chain-smoking, one-legged, naked, blind man so that one can properly conclude if he can climb to the summit.

 

PS: I consider it very likely that EJ Manuel is given every opportunity to make this plateau with the Bills.

 

Interesting analogy. And definitely a good point about 1,000 as a "magic number". Of course, there isn't anything magical about that number of attempts, it's just a nice round number that humans like to work with, and it happens to fall at a fairly convenient spot in assessing a QB's progress. The results would probably be very similar using 931 or 1,234 attempts. (If they weren't, then that's a sign that maybe this is just a random occurrence that showed up due to the small sample involved.) There's probably a way to re-design this study using some kind of range or continuum of pass attempts, but none of us are getting paid for this, nor do we have any influence over any team's actual decision-making, so I don't really see a huge gain from complicating things.

 

Anyway, assuming we accept that there's really something here, what's the takeaway? I don't really agree with the Gladwell/10,000 hours analogy. Yes, more reps/attempts/practice time/etc. will help a young player get better, but not everyone will get good enough to succeed in the NFL, even if he takes every snap for 5 straight years. What Ramius is really talking about is that before a player gets (roughly) to 1,000 attempts, it's tough to really predict if his current play is indicative of his future play. Once he gets there, you have a pretty good idea of what you're going to get. Combining that with Sisyphean Bills' point, I'd say that if you have a young QB with some promise -- he has the physical tools, seems to be liked/respected as a leader by teammates or seems like he will be once he starts winning, shows some flashes of brilliance or at least stretches of competence, and isn't abysmal -- it might actually be worth your while to commit to him long enough to get him to that 1,000 attempt area. Conversely, don't sign him to a big extension before he gets there, even if he's played well. And I think EJ Manuel is a great case study for the former, because he looks like he has all the tools, but certainly had his share of flaws as a rookie. I doubt he'll get particularly close to 1,000 attempts even if he stays healthy next year, but this study makes me think that it's worth committing to him for 2015 as well, regardless of his performance in 2014. If he's not good enough in 2015, then it's time to cut bait and move on. (Probably with a new head coach, because if EJ sucks the next 2 years, our record will probably stink, and Marrone's out the door.)