Player Rating Systems in the Invasion-Territorial Team Sports – What are the Issues?

Originally Written: September 2016

A central issue in sports analytics is the construction of player rating systems particularly in the invasion-territorial team sports. Player rating systems are important as a means of summarising the overall match performance of individual players. Teams can use player rating systems to review performances of their own players as well as tracking the performance levels of potential acquisitions. Moneyball highlighted the possibilities of using performance metrics to inform player recruitment decisions. But the relatively simple game structure of baseball, in essence a series of one-to-one contests between hitters and pitchers, means that the analytical problem is reduced to finding the best metrics to capture hitting and pitching performances.

Once we move into invasion-territorial team sports, we are dealing with sports which involve the tactical coordination of players and player performance becomes multi-dimensional. The analytical problem is no longer restricted to identifying the best metric for a single skill-activity per player (i.e. pitching or hitting in baseball) but now involves identifying the full set of relevant skill-activities and creating appropriate metrics for each identified skill-activity.

There are essentially two broad approaches to constructing player rating systems when player performances are multi-dimensional. One approach is the win-contribution (or bottom-up or atomistic) approach which involves identifying all of the relevant skill-activities that contribute to the team’s win ratio, developing appropriate metrics for each of these skill-activities, and then combining the set of skill-activity metrics into a single composite measure of performance. Over the years many technical and practical problems have emerged in constructing win-contribution player rating systems. I plan to discuss these in more detail in a future blog. Suffice to say, the most general criticism of the win-contribution approach is the difficulty of identifying all of the relevant skill-activities particularly those that are not directly and/or easily observable such as teamwork and resilience.

The alternative approach is a more holistic or top-down approach that uses the match outcome as the ultimate summary metric for measuring team performance and then attributes the match outcome to those involved in its production. I call this the win-attribution approach to player rating systems. The analytical problem is now the choice of an attribution rule.

Plus-Minus Player Ratings

The best-known win-attribution approach is plus-minus which has been used for many years in both basketball and ice hockey. It is a very simple method. Just total up the points scored and the points conceded whenever a specific player is on court (or on the ice), and then subtract points conceded from points scored to give the points margin. This represents the player’s plus-minus score.

For those of you not familiar with the plus-minus approach, here’s a simple example. Consider the following fictitious data for the first three games of a basketball team with a roster of 10 players.

The results of the three games are:

Game 1: Won, 96 – 73

Game 2: Lost, 68 – 102

Game 3: Won, 109 – 57

The minutes played (Mins) for each player, and points scored (PS) and points conceded (PC) while each player is on court, are as follows:

Player	Game 1			Game 2			Game 3
	Mins	PS	PC	Mins	PS	PC	Mins	PS	PC
P1	32	54	58	28	35	64	12	27	18
P2	29	63	45	25	33	56	13	30	21
P3	27	48	43	20	36	47	13	29	23
P4	33	58	52	27	32	63	15	33	22
P5	35	63	54	36	37	82	25	54	33
P6	22	49	24	28	44	43	33	72	30
P7	20	45	20	22	35	37	35	76	32
P8	16	37	27	24	38	51	33	77	36
P9	15	35	23	23	36	50	35	82	38
P10	11	28	19	7	14	17	26	65	32

A player’s plus-minus score is just the points margin (= PS – PC). So in the case of player P1 in Game 1, he was on court for 32 minutes during which time 54 points were scored and 58 points were conceded. Hence his plus-minus score is -4 (= 54 – 58). Given that the team won the game with a points margin of 23, the plus-minus score indicates a well below average performance. The full set of plus-minus scores are as follows:

Player	Plus-Minus Scores				Average Benchmark	Benchmark Deviation
	Game 1	Game 2	Game 3	Total
P1	-4	-29	9	-24	8.50	-32.50
P2	18	-23	9	4	10.27	-6.27
P3	5	-11	6	0	12.85	-12.85
P4	6	-31	11	-14	12.94	-26.94
P5	9	-45	21	-15	18.35	-33.35
P6	25	1	42	68	26.46	41.54
P7	25	-2	44	67	31.92	35.08
P8	10	-13	41	38	26.42	11.58
P9	12	-14	44	42	28.81	13.19
P10	9	-3	33	39	28.48	10.52

As well as the plus-minus scores for each player in each game, I have also reported the total plus-minus score for each player over the three games. I have also calculated an average benchmark for each player by allocating the final points margin for each game pro rata based on minutes played. So, for example, player P1 played 32 out of 48 minutes in Game 1 which ended with a 23 winning margin. An average performance would have implied a plus-minus score of 15.33 (= 23 x 32/48). His average benchmarks in Games 2 and 3 were -19.83 (= -34 x 28/48) and 13.00 (= 52 x 12/48), respectively. Summing the average benchmarks for each game gives an overall average benchmark of 8.50 for player P1. The final column reports the deviation from benchmark of the player’s actual plus-minus score.

In this example players P1 – P5 were given the most game time in Games 1 and 2 but all five players have negative benchmark deviations. The allocation of game time in Game 3 better reflects the benchmark deviations with players P6 – P10 given much more game time.

Limitations and Extensions to Plus-Minus Player Ratings

The advantage of the plus-minus approach is its simplicity. It is not dependent on detailed player performance data but only requires information on the starting line-ups, the timing of player switches, and the timing of points scored and conceded. The very first piece of work that I did for Saracens in March 2010 was to rate their players using a plus-minus approach. I focused on positional combinations – front row, locks, back row, half backs, centres, and backs – and calculated the plus-minus scores for each combination. Brendan Venter, the Director of Rugby, was very positive on the results and commented that “your numbers correspond to our intuitions”. It was on the basis of this report that I was engaged to work as their data analyst for five years. The plus-minus approach was used for player ratings in the early stages of the 2010/11 season but was eventually discarded in favour of a win-contribution approach.

One of the problems with the simple plus-minus approach is that it will give high scores to players who regularly play with very good players. So, if a particular player was fortunate enough to be playing regularly alongside Michael Jordan, they would have had a high plus-minus score but this reflects the exceptional ability of their team mate more than their own performance. My dear friend, the late Trevor Slack, one of the top people in sport management and a prof at the University of Alberta in Edmonton, used to call it the Wayne Gretzky effect. Those of you who know their ice hockey history will know exactly what Trevor meant. Gretzky was one of the true greats of the NHL and brought the best out of his team mates whenever he was on the ice. The Edmonton Oilers won four Stanley Cups with Gretzky in the 1980s.

Similarly it can be argued that the basic plus-minus approach does not make any allowance for the quality of the opposing players. Rookie players given more game time against weaker opponents will have their plus-minus scores inflated just as those players who get proportionately more game time against stronger opponents will see their plus-minus scores reduced. One way around the problems of controlling for the quality of team mates and opponents is to use Adjusted Plus-Minus which involves using regression analysis to model the points margin during a “stint” (i.e. a time interval when no player switches are made) as the function of own and opposing players. The estimated coefficients represent the adjusted plus-minus scores. There have also been various attempts to include other performance data to create real adjusted plus-minus scores which represent a hybrid of the win-attribution and win-contribution approaches.

Overall the plus-minus approach offers a relatively simple way of measuring player performance without the need for detailed player performance data. But the approach works best in high scoring sports with frequent player switches such as basketball and ice hockey. The plus-minus approach is not well suited to football (soccer) which is low scoring and teams are restricted to only three substitutions.