What does “good enough to win” really mean?

By
Updated: July 1, 2013

What does it mean when we say a quarterback “played well enough to win”? WCSN statisticians undertook to find out.

Football is a sport of clichés. One of those clichés that frequently gets bandied about is “enough to win” — as in, “the quarterback did enough to win” or “the defense did enough to win.” Usually it’s followed by a “but” and an explanation of why the team lost anyway: “the defense collapsed in the last two minutes” or “the kicker missed a last-second field goal attempt.”

We all know that quarterbacks typically get too much credit when the team wins and take too much of the blame when it loses. So how can we know when a quarterback truly has done “enough to win”? Can we somehow define this nebulous concept statistically, such that if a quarterback performs to a certain standard in a game, we can confidently state that he has done his fair share and that if the team loses, the loss is attributable to failure on the part of some other unit?

In other words, what does “good enough to win” mean?

For the purposes of this discussion, we will define “good enough to win” as a level of performance that gives a team at least a 51 percent probability of winning. Now our task is to determine how well a quarterback must perform in a game to give his team better than a coin flip’s chance of winning.

Fabulous four

As a starting point, we looked at every game played by the four starting quarterbacks of the NFC North — Aaron Rodgers, Jay Cutler, Matthew Stafford, and Christian Ponder — between 2008-2012. This gave us 192 games’ worth of data, more than enough for a pilot project. All data was collected from the Game Logs page for each quarterback as posted on NFL.com, which list the final scores and win/loss outcomes for every game.

To determine quarterback performance, we turned to our trusty tool, the Complete Quarterback Rating (CQBR), which has proven itself so useful in a variety of applications. We sorted all the games by CQBR and grouped them into “bins” by CQBR range. We then tallied up the total wins and losses in each range to determine win probability.

Between 2008 and 2012, there is a strong positive correlation between performance of NFC North quarterbacks, as measured by Complete Quarterback Rating (CQBR), and their team’s probability of winning a game. Quarterbacks who posted CQBRs < 0.0 never won, while those who posted CQBRS > 110.0 never lost.

The histogram above makes it obvious that, at least for these NFC North quarterbacks, there is a strong positive correlation between quarterback performance and the team’s probability of winning the game. Between 2008 and 2012, teams helmed by current NFC North quarterbacks never won when the quarterback posted a CQBR of less than zero, while they won every time when the quarterback posted a CQBR of at least 110.0.

Lining up

By looking at the chart, we can estimate that a quarterback would probably need to put up a CQBR of around 60.0 or so to give his team at least a 51 percent chance of winning. And indeed, if we plot win probabilities against the midpoints of the bin ranges (e.g., the midpoint of the 20 to 29.9 bin range is 25.0), we get a beautiful line:

This linear regression shows the strong positive relationship between quarterback efficiency (CQBR) and win probability. From the equation of the trendline (y = 0.7517x + 8.8276, R² = 0.92), we can calculate that a quarterback must produce a CQBR of 56.1 to give his team a 51% chance of winning the game.

Plugging our desired probability of 51 percent into the equation of this trend line reveals that a quarterback who produces a CQBR of 56.1 has “done enough to win the game”; that is, given his team a better-than-coin-flip’s chance of winning.

What’s really exciting about this chart is that it shows that CQBR is almost, in and of itself, a measure of a team’s win probability. If a quarterback puts up a zero, his team has essentially no chance of winning. If he rates around a 50, his team has approximately a 50-50 chance of winning. If he has a great day and generates a CQBR of 100 or so, his team is almost guaranteed to win.

This would seem to make CQBR more useful than anyone could have imagined.

Casting a wider net

These results seem promising indeed, but would they hold up for the entire league?

After all, it’s hard to avoid the suspicion that this sample is probably not representative of the NFL as a whole — if for no other reason than it features a whole lot of Aaron Rodgers, reflected in the data set’s markedly high average CQBR of 64.79.

In other words, we could not ignore the possibility that our findings were skewed by Rodgers’ unworldly performances. The fact that these four quarterbacks play each other so often might also have an influence.

Nevertheless, the trend apparent above was so strong that we were confident it would show up in a leaguewide sample, even if it turned out not to be neither as neat or nor as pretty.

Resampling

So the next step was to look at a representative sample from the league as a whole. We wanted to focus on starting quarterbacks, because we wanted to look at trends over time, and backups have a tendency to introduce outliers into the situation. (Witness Matt Flynn’s extraordinary Week 17 performance against the Detroit Lions in 2011).

In order to ensure that only starting quarterbacks were represented, we limited the sample to those quarterbacks who started at least 12 games in a season between 2008 and 2012. That gave us a starting list of 120 quarterbacks. Compiling 120 quarterback-seasons’ worth of data would have been prohibitively time consuming, however, so we had to cull the list.

To prevent extremely high- or low-rated quarterbacks from being more likely to appear in the final sample, we randomly sorted the quarterbacks by season. We then selected every fifth quarterback-season to produce a much more manageable 24 quarterback-seasons for a total data set of 375 quarterback-games.

Using standard statistical outlier tests, we scrubbed the data set and threw out all games in which the quarterback attempted fewer than 11 or more than 58 passes. This helped eliminate anomalous games like Carson Palmer’s Week 16 (2012) performance against the Carolina Panthers, in which he posted a CQBR of 125.33 on three passing attempts — yet the Oakland Raiders lost the game. This left us with a sample size of 365 games, still more than enough for our purposes.

One of the outcomes of our sampling technique was that certain quarterbacks, like Ben Roethlisberger of the Pittsburgh Steelers and Jay Cutler of the Chicago Bears, are slightly overrepresented in the data set, while quarterbacks like Aaron Rodgers may be slightly underrepresented. But the method is certainly free of any intentional bias.

Average CQBR for the final data set was 58.8 with a median of 60.9 and standard deviation of 36.8. Considering the fact that the average CQBR for NFL starting quarterbacks the past five seasons has been right around 58, we can be reasonably certain that this is a representative sample.

Obvious trend

Just as with the original NFC North sample, the relationship between quarterback performance and a team’s win probability for this leaguewide sample is immediately apparent. Indeed, the results could not be more clear: as CQBR rises, so does a team’s probability of winning.

There is a strong positive relationship between quarterback performance as measured by CQBR and a team’s win probability for NFL quarterbacks between 2008 and 2012. With only one exception, quarterbacks who post CQBRs of less than zero never win, while quarterbacks who post CQBRS greater than 120 always win.

Quarterbacks who post a CQBR of less than 20.0 give their team’s very little chance to win, and those who put up CQBRs of less than zero almost never win. The only exception was a Week 13 (2012) game between the New York Jets and the Arizona Cardinals, in which Mark Sanchez put up an appalling CQBR of -25.43 on 21 attempts — and yet the Jets managed to eke out a 7-6 victory. Clearly, in this case, Sanchez did not “do enough to win” and was bailed out by the defense.

Conversely, when a quarterback puts up a CQBR of at least 100.0, his team wins roughly 85 percent of the time, and if he has a superb day with a CQBR over 120.0, his team wins every time.

As before, it appears by looking at the histogram that a quarterback needs to produce a CQBR of around 60 if he wants to give his team a good chance to win. Again, we can confirm this by plotting win probability against the midpoints of the bins, which gives us this lovely curve:

There is a strong relationship between quarterback performance as measured by CQBR and a team’s probability of winning. Using the equation of the trend line (y = 0.7256x + 6.0893, R² = 0.98), we can calculate that a quarterback must produce a CQBR of 61.9 to give his team a 51% chance of winning.

Good enough to win

Plugging our goal of 51 percent into the formula of the trend line reveals that, between 2008 and 2012 at least, an NFL quarterback needed to finish a game with a CQBR of at least 61.9 to give his team a better than 50-50 chance of winning. As mentioned previously, the league mean CQBR for starting quarterbacks during this period was roughly 58, so this represents a performance that is slightly above average.

To put this another way, a quarterback doesn’t have to have a spectacular day in order to do his part. He simply has to get the job done. After he has done his work, it’s up to the defense and special teams to seal the deal.

So the next time someone says that a quarterback “did enough to win the game,” run the numbers. If his CQBR for that game is at least 61.9, you can agree that indeed, the quarterback has done his part and that some other facet of the team — probably the accursed defense — has failed him.

But what happens when both quarterbacks put up extremely high numbers in the same game? What can we say then? Well, that is a fascinating question for future investigations!

CQBR’s cousin

Since CQBR seems to correlate so well with winning, we began to wonder how well the NFL passer rating would fare in comparison. We will reveal our findings tomorrow.

About the author(s)

Rourke Douglas Decker covers the Green Bay Packers beat for Water Cooler Sports. He resides with his family in the Twin Cities. He can be reached for questions or comments at . Connect with .

529 comments
bp.
bp.

Has everyone discussed he fact that Ashley Fox is a dumbass dumbfuck already?

Mr. Horse
Mr. Horse

NFL Balance: The Most Effective Offenses Since 1970

No surprise we see the dominance of the pass again. The top 20 pass-dependent teams still went a staggering 230-69-2(.767) with six Super Bowl appearances in spite of their imbalance. Meanwhile the top 20 run-dependent teams only went 93-220-1 (.298) with the 2012 Vikings being the lone playoff appearance. They went one-and-done with back up Joe Webb pretending to play quarterback.  

If you are going to be one-dimensional, it pays to be the one that matters most in football.

Benjamin Rajile
Benjamin Rajile

What has four wheels and flies?
  

A garbage truck.

bp.
bp.

Sample size too small.


Also:


LKP 40 10
LKP 40 10

THE TRUTH NUKES WILL LAUNCH AGAIN!!!!!!!!!!!!!

Mr. Horse
Mr. Horse

2008-2012 HOME FIELD ANALYSIS, Vegas style

Mr. Horse
Mr. Horse

Five Years of PFF Grades: Top 10 Linebackers 


9. Desmond Bishop, GB 

10. Lance Briggs, CHI 


Top 10 Overall: Run Defense 

5. E.J. Henderson, MIN 

7. Chad Greenway, MIN 


Top 10 Overall: Coverage 

2. Brian Urlacher, CHI

4. Ben Leber, MIN & STL 

5. Thomas Davis, CAR 

9. Stephen Tulloch,  TEN & DET


Mr. Horse
Mr. Horse

“Even if teams knew about the alleged punch to the head of the restaurant employee, it remains difficult if not impossible to project murder from this type of misbehavior.” -Mike Florio

G & G
G & G moderator

 No, but I'm with you

bp.
bp.

 But honestly, this lost credibility early:

"An elite quarterback is rarely an advantage to your team’s running game. We know the Green Bay Packers  hardly ever bother putting the ball into the hands of someone not named Aaron Rodgers, and for good reason. It’s similar to the Miami career of Dan Marino in that regard."


Green Bay has been fairly balanced under Fat Mike.  2011 stands out, and you can't really blame him for that.  But GB was something like 14 in the league in run-pass balance last year.  They just don't produce when they run.

bp.
bp.

 Rourke is going to poop down your throat now.

bp.
bp.

 Divine.

bp.
bp.

 not surprised to see Urlacher that high on the coverage list.  I don't think CHI fans are fully ready for an MLB who doesn't work those drop zones the way Urlacher did.

Rourke
Rourke

 I was away for 10 days, so  I am not apprised of such political matters.

Mr. Horse
Mr. Horse

  He noted that running the ball into a brick wall for 2 yards isn't real effective. 

Mr. Horse
Mr. Horse

  I disagree with some of the comments he makes but the stats are the stats. 

Benjamin Rajile
Benjamin Rajile

   

They ran it like 44% of the time last year. Cold Hard Football IDIOTS is more like it!

Rourke
Rourke

 Why? There is no question that the NFL is a passing league and has been for decades. Even in the era of Sid Luckman, the teams that dominated the pass typically dominate the league.

bp.
bp.

  You can tell the Saints run up the score all the time.

Doctor ϟ Professor ☧
Doctor ϟ Professor ☧ moderator

  

It was because he went back and edited his 2012 preditions article to make it look like he was correct and something else to that extent.


nvm..I remember someone said they told you about it.

Pat Fenis, Esq.
Pat Fenis, Esq. moderator

  You missed a lot and yet nothing at all.

bp.
bp.

  Well.  That escalated quickly.

G & G
G & G moderator

 Let's rape her!

BP,you are first

Benjamin Rajile
Benjamin Rajile

  

Harris had a decent amount of success but I don't think they ever really trusted him for some reason.

bp.
bp.

  And dropped one really deep safety along with overall deep coverage on Rodgers all year.  And he can get a little obsessed with the long ball, and I thought generally failed to show the patience necessary to make such defenses pay.  And that is the trouble with not being able to run, even when you do so a fairly decent percentage of the time.

Mr. Horse
Mr. Horse

  Packers opponents mostly ignored the run between Benson getting hurt and Harris giving up his car salesman job.

G & G
G & G moderator

  Lacy and Franklin will destroy this "Wall" this season!!

bp.
bp.

  I don't think there are a lot of people who argue its great to have to run into a brick wall over and over again because your quarterback is shitty.

bp.
bp.

   Yeah!  *dances terrible white guy dance*

Rourke
Rourke

 IP address is more useful. It's easy to have multiple accounts open at the same time.

Rourke
Rourke

 Well, it's easy to see: the admin panel shows you the IP addresses.

Rourke
Rourke

 I have never said don't ban him when he misbehaves. I have only said that people who go out of their way to provoke him should also be banned at the same time. I bozoed him and other people in the past when things got inflammatory. Then they cooled down for a while.

Rourke
Rourke

 I know I am a bit of a control freak, so that is one of the major reasons why I went out of my way to unplug from the site completely while I was on vacation and let the guys who took over run it as they saw fit. I never peeked in even once.

Doctor ϟ Professor ☧
Doctor ϟ Professor ☧ moderator

  

Haven't seen people that mad on here since the Gabes Collusion saga lol.

Rourke
Rourke

 Now I am really confused. Editing an old article when a new one was going to be published anyway is senseless. Ah, well, I will restore it to its old form. Hopefully this doesn't happen again. I have always been proud that our authors were trusted to edit their articles as necessary after being published. Most sites don't allow that.

Pat Fenis, Esq.
Pat Fenis, Esq. moderator

  We all felt it was a bigger deal because of what it says about the site.

Rourke
Rourke

 Looks to me like the identical article with 2012 replaced by 2013. Not a huge deal, but still stupid. Should have been published as a new article. I don't like screwing with the chronology of the site.

Pat Fenis, Esq.
Pat Fenis, Esq. moderator

  I think he just wrote over his 2012 predictions so as not to make himself look bad.

Rourke
Rourke

 So there is a new predictions article? I am going to presume that got published while I was gone. Do you have a link to that?

Doctor ϟ Professor ☧
Doctor ϟ Professor ☧ moderator

  

All you gotta do is ask MaC or look at the comments on the article before his predictions one that came out.

Rourke
Rourke

 The two revisions in the history are identical, so I don't know what, if anything, was changed.

Rourke
Rourke

 No one told me about this. Link?