Lindy's Five Essential Websites (Non-Major Media) for 2013
[+] Team Summaries

Tuesday, November 20, 2012

The Network Rankings

by J. Patrick Rhamey Jr., PhD

            There are myriad college football rankings, and, as Scott has noted, the statistical choices that go into each are inherently subjective.  I am going to propose one more ranking to the many that already exist, but as the below comparison demonstrates, most existing rankings are currently converging toward what the ranking I describe below already listed last week (Notre Dame 1, Alabama 2), providing a compelling case in favor of my method.  The subjective choice about the relevant data to include is simple: none of it.  To create my rankings I use only wins and losses, ignoring statistics such as margin of victory, total yards, turnovers, etc. for two reasons. First, we are asking an enormous amount from our data if we are generating predictions of probable team performance based upon a limited number of observations (at this point, 10 or 11 games).  Second, statistics used in existing rankings are incomparable across games given the dramatically different contexts.  If two teams both have 300 total yards of offense, but one team played in the snow in Ann Arbor while the other played on a sunny day in the Rose Bowl, we are essentially equating completely incomparable numbers to generate relative rank.  Margin of victory is perhaps one of the more egregious variables, with large numbers frequently a better indicator of poor sportsmanship, personal grudges, or heated historical rivalries than a decisive indicator of excellence.
            Criticism of existing rankings, however, accomplishes little without a proposed replacement that solves these problems. I’ll begin with the same conceptual principle as Scott Albrecht, Wesley Colley, and others and say that any ranking begins first with wins and losses.  However, unlike alternatives, that is also where I’ll end.  The only information we need to rank FBS teams is their win-loss records and all the other proposed metrics – margin of victory, total yards, turnovers – amount to little more than noise.  Conceptually, if every team in the FBS played every other team in the FBS, there would be no question as to who was number one.  However, that’s not the case, so other rankings resort to collecting additional statistical information to fill in the gaps.  This is essentially what both Albrecht’s BPR ranking and the Colley Matrix do in their “second steps”.  They generate a relative weighting of wins and losses by the predicted probability of victory, or strength of schedule.   However, this second step is asking too much from limited data, and is more importantly, unnecessary.  Using a technique called Network Analysis, we can limit ourselves to analyzing the pattern of wins and losses without having to predict the outcomes of probable matchups or include various other statistics. 
By mid-season (week 7 this year) if we draw lines between teams that have played one another, every team is connected by some degrees of separation to every other team.  The below figure is the network of games played between FBS teams following week 12.  An arrow pointing at a team signifies that team is the winner of the game played.



As we expect, teams in the same conference cluster together tightly given most games are intra-conference.  You can interpret team’s proximity to one another as the degree to which the teams share a similar schedule.  Because teams are now linked in some way to every other team, we can generate an ordering of the quality of team wins based on how central they are in the web of wins in the network. To determine how badly their losses are, we can do the same thing for the web of losses.  Think of it like this: we’re playing a big game of six degrees of separation and trying to figure out which team is Kevin Bacon, or the common denominator to which all other teams in the FBS are connected.  The football team that reaches the most teams through their wins in the fewest degrees of separation (and likewise the fewest teams through their losses) is the highest rank team.  As an example, if Alabama plays 10 teams, but then those 10 teams lose all their other games, Alabama is only connected to 10 teams through their network of wins.  That’s not very good.  If Alabama plays 10 teams, but those ten teams defeat all their other opponents, Alabama is now connected by two degrees of separation to 100 teams.  That’s a lot better.
We can add up all these links between teams using a measure called “average reciprocal distance” (ARD), a centrality measure in network analysis (the program Ucinet 6 by Borgatti, Everett, and Freeman was used to generate the above illustration and the following rankings).  ARD measures how far on average each team is from every other team in the network, which I calculate separately for the paths of wins and losses.  The higher the value, the more central or connected a team is in the network, or the more teams it is connected to by fewer links.  For the network of wins, this will translate to a higher rank, with a higher ARD value signifying greater centrality.  In the network of losses, centrality to the network will result in a lower ranking.  Because we can assume that the direct inverse of a win is a loss, we simply subtract the centrality of a team to the FBS network of wins by the centrality of a team to the FBS network of losses.  For a full discussion of the underlying concept, see Steve Borgatti’s research on the key player problem (https://sites.google.com/site/steveborgatti/research/publications).
Since we now know the outcomes of the games played from this weekend, we’ll use last week’s rankings as a comparison.  The below table lists (1) the ARD in the network of FBS wins, (2) the ARD in the network of FBS losses, (3) the win ARD minus the loss ARD to generate (4) the Network Ranking.  The following columns compare the Network Ranking with the BCS, the AP Poll and Scott Albrecht’s (Hybrid) ranking for all teams in the top 10 in at least one of the rankings

Rankings preceding the Week of November 11-17.
Team
ARD Wins
ARD Losses
Wins – Losses
Network Rank
BCS
AP
Albrecht
Notre Dame
48.27
0
48.27
1
3
3
4
Alabama
45.56
2.58
42.98
2
4
4
3
Florida
44.81
2.28
42.53
3
6
7
5
Ohio State
41.96
0
41.96
4
-
6
7
Oregon
41.53
0
41.53
5
2
1
1
LSU
43.29
3.33
39.96
6
7
8
13
Georgia
42.26
2.58
39.67
7
5
5
6
Kansas State
39.36
0
39.36
8
1
2
2
Texas A&M
41.36
3.33
38.02
9
8
9
8
South Carolina
40.42
3.33
37.09
10
9
12
11
Oklahoma
34.19
2
32.19
12
12
13
9
Florida State
34.75
23.74
11.01
32
10
10
10

            First, we see how the Network Ranking operates.  LSU, for example, is more central to the network of FBS wins than Ohio State or Oregon going into the week, but its two losses, while not at all strongly central to the network of FBS losses, drag it down to number 6 (by comparison the ARD loss score for New Mexico State, the bottom ranked team, is 46.23).
            Second, while no one could have predicted the outcome of the Kansas State and Oregon games, the Network Ranking is alone in suggesting that both teams were over-ranked.  Meaning, they had not yet provided sufficient evidence demonstrating their centrality to the network of FBS wins to merit being ranked over Notre Dame, Alabama, etc.  We see a similar dissonance present with Florida State further down the list.  Prediction (and predictive modeling inherent in most computer rankings) is exactly the problem exposed by the Oregon and Kansas State collapses this weekend.  Predicting based on such a limited number of observations (or in practice as Kirk Herbstreit refers to it, the “look test”) will result in correspondingly limited success.
            Third, as we would expect, the rankings are now converging.  Every ranking this week has Notre Dame #1 and Alabama #2, and the Network Ranking is no different.  But, that’s what the network ranking had last week!  In other words convergence is happening, but all other rankings are converging toward the Network Rankings.
            The Network Rankings conceptually aren’t doing anything new – they are built on the same goal of ranking teams based on wins and losses.  However, unlike alternatives, the method underlying the Network Rankings best corresponds with that goal, with the result being a ranking that actually ranks based on wins and losses rather than predicted probabilities from incomparable metrics.   If we’re going to rank on wins and losses, all we need is wins, losses, and some careful counting of links between teams.

J. Patrick Rhamey Jr., PhD
Assistant Professor
International Studies and Political Science
Virginia Military Institute
rhameyjp@vmi.edu

No comments:

Post a Comment