Player Rating Systems in the Invasion-Territorial Team Sports – What are the Issues?

Executive Summary

  • Player rating systems are important as a means of summarising the overall match performance of individual players.
  • Player performance in the invasion-territorial team sports is multi-dimensional so that a player rating system needs to be able to combine metrics for a number of skill-activities.
  • There are two broad approaches to the construction of player rating systems – win-attribution (or top-down/holistic) approaches and win-contribution (or bottom-up or atomistic) approaches.
  • The plus-minus approach is the most widely used win-attribution approach based on the points margin when a player is playing.
  • A player’s plus-minus score is sensitive to context especially the quality of team mates and opponents. This can be controlled using regression analysis to estimate adjusted plus-minus scores.
  • The plus-minus approach offers a relatively simple way of measuring player performance without the need for detailed player performance data. But the approach works best in high scoring sports with frequent player switches such as basketball and ice hockey.

 

A central issue in sports analytics is the construction of player rating systems particularly in the invasion-territorial team sports. Player rating systems are important as a means of summarising the overall match performance of individual players. Teams can use player rating systems to review performances of their own players as well as tracking the performance levels of potential acquisitions. Moneyball highlighted the possibilities of using performance metrics to inform player recruitment decisions. But the relatively simple game structure of baseball, in essence a series of one-to-one contests between hitters and pitchers, means that the analytical problem is reduced to finding the best metrics to capture hitting and pitching performances.

 

Once we move into invasion-territorial team sports, we are dealing with sports which involve the tactical coordination of players and player performance becomes multi-dimensional. The analytical problem is no longer restricted to identifying the best metric for a single skill-activity per player (i.e. pitching or hitting in baseball) but now involves identifying the full set of relevant skill-activities and creating appropriate metrics for each identified skill-activity.

 

There are essentially two broad approaches to constructing player rating systems when player performances are multi-dimensional. One approach is the win-contribution (or bottom-up or atomistic) approach which involves identifying all of the relevant skill-activities that contribute to the team’s win ratio, developing appropriate metrics for each of these skill-activities, and then combining the set of skill-activity metrics into a single composite measure of performance. Over the years many technical and practical problems have emerged in constructing win-contribution player rating systems. I plan to discuss these in more detail in a future blog. Suffice to say, the most general criticism of the win-contribution approach is the difficulty of identifying all of the relevant skill-activities particularly those that are not directly and/or easily observable such as teamwork and resilience.

 

The alternative approach is a more holistic or top-down approach that uses the match outcome as the ultimate summary metric for measuring team performance and then attributes the match outcome to those involved in its production. I call this the win-attribution approach to player rating systems. The analytical problem is now the choice of an attribution rule.

 

Plus-Minus Player Ratings

The best-known win-attribution approach is plus-minus which has been used for many years in both basketball and ice hockey. It is a very simple method. Just total up the points scored and the points conceded whenever a specific player is on court (or on the ice), and then subtract points conceded from points scored to give the points margin. This represents the player’s plus-minus score.

 

For those of you not familiar with the plus-minus approach, here’s a simple example. Consider the following fictitious data for the first three games of a basketball team with a roster of 10 players.

The results of the three games are:

Game 1: Won, 96 – 73

Game 2: Lost, 68 – 102

Game 3: Won, 109 – 57

The minutes played (Mins) for each player, and points scored (PS) and points conceded (PC) while each player is on court, are as follows:

 

Player Game 1 Game 2 Game 3
Mins PS PC Mins PS PC Mins PS PC
P1 32 54 58 28 35 64 12 27 18
P2 29 63 45 25 33 56 13 30 21
P3 27 48 43 20 36 47 13 29 23
P4 33 58 52 27 32 63 15 33 22
P5 35 63 54 36 37 82 25 54 33
P6 22 49 24 28 44 43 33 72 30
P7 20 45 20 22 35 37 35 76 32
P8 16 37 27 24 38 51 33 77 36
P9 15 35 23 23 36 50 35 82 38
P10 11 28 19 7 14 17 26 65 32

 

A player’s plus-minus score is just the points margin (= PS – PC). So in the case of player P1 in Game 1, he was on court for 32 minutes during which time 54 points were scored and 58 points were conceded. Hence his plus-minus score is -4 (= 54 – 58). Given that the team won the game with a points margin of 23, the plus-minus score indicates a well below average performance. The full set of plus-minus scores are as follows:

 

Player Plus-Minus Scores Average Benchmark Benchmark Deviation
Game 1 Game 2 Game 3 Total
P1 -4 -29 9 -24 8.50 -32.50
P2 18 -23 9 4 10.27 -6.27
P3 5 -11 6 0 12.85 -12.85
P4 6 -31 11 -14 12.94 -26.94
P5 9 -45 21 -15 18.35 -33.35
P6 25 1 42 68 26.46 41.54
P7 25 -2 44 67 31.92 35.08
P8 10 -13 41 38 26.42 11.58
P9 12 -14 44 42 28.81 13.19
P10 9 -3 33 39 28.48 10.52

 

As well as the plus-minus scores for each player in each game, I have also reported the total plus-minus score for each player over the three games. I have also calculated an average benchmark for each player by allocating the final points margin for each game pro rata based on minutes played. So, for example, player P1 played 32 out of 48 minutes in Game 1 which ended with a 23 winning margin. An average performance would have implied a plus-minus score of 15.33 (= 23 x 32/48). His average benchmarks in Games 2 and 3 were -19.83 (= -34 x 28/48) and 13.00 (= 52 x 12/48), respectively. Summing the average benchmarks for each game gives an overall average benchmark of 8.50 for player P1. The final column reports the deviation from benchmark of the player’s actual plus-minus score.

 

In this example players P1 – P5 were given the most game time in Games 1 and 2 but all five players have negative benchmark deviations. The allocation of game time in Game 3 better reflects the benchmark deviations with players P6 – P10 given much more game time.

 

Limitations and Extensions to Plus-Minus Player Ratings

The advantage of the plus-minus approach is its simplicity. It is not dependent on detailed player performance data but only requires information on the starting line-ups, the timing of player switches, and the timing of points scored and conceded. The very first piece of work that I did for Saracens in March 2010 was to rate their players using a plus-minus approach. I focused on positional combinations – front row, locks, back row, half backs, centres, and backs – and calculated the plus-minus scores for each combination. Brendan Venter, the Director of Rugby, was very positive on the results and commented that “your numbers correspond to our intuitions”. It was on the basis of this report that I was engaged to work as their data analyst for five years. The plus-minus approach was used for player ratings in the early stages of the 2010/11 season but was eventually discarded in favour of a win-contribution approach.

 

One of the problems with the simple plus-minus approach is that it will give high scores to players who regularly play with very good players. So, if a particular player was fortunate enough to be playing regularly alongside Michael Jordan, they would have had a high plus-minus score but this reflects the exceptional ability of their team mate more than their own performance. My dear friend, the late Trevor Slack, one of the top people in sport management and a prof at the University of Alberta in Edmonton, used to call it the Wayne Gretzky effect. Those of you who know their ice hockey history will know exactly what Trevor meant. Gretzky was one of the true greats of the NHL and brought the best out of his team mates whenever he was on the ice. The Edmonton Oilers won four Stanley Cups with Gretzky in the 1980s.

 

Similarly it can be argued that the basic plus-minus approach does not make any allowance for the quality of the opposing players. Rookie players given more game time against weaker opponents will have their plus-minus scores inflated just as those players who get proportionately more game time against stronger opponents will see their plus-minus scores reduced. One way around the problems of controlling for the quality of team mates and opponents is to use Adjusted Plus-Minus which involves using regression analysis to model the points margin during a “stint” (i.e. a time interval when no player switches are made) as the function of own and opposing players. The estimated coefficients represent the adjusted plus-minus scores. There have also been various attempts to include other performance data to create real adjusted plus-minus scores which represent a hybrid of the win-attribution and win-contribution approaches.

 

Overall the plus-minus approach offers a relatively simple way of measuring player performance without the need for detailed player performance data. But the approach works best in high scoring sports with frequent player switches such as basketball and ice hockey. The plus-minus approach is not well suited to football (soccer) which is low scoring and teams are restricted to only three substitutions.

 

15th September 2016

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s