Developing a Player Report Card
Developing a Player Report Card
Creating my own evaluation method is a marathon, not a sprint
I am no data scientist; just a humble data analyst. So what’s the difference? In this case, I don’t develop new and exciting mathematical equations. Instead, I crunch numbers using already written formulas and interpret the results. I look for patterns, identify them, and point them out. This is what makes my job both subjective and objective in nature. I also used to work in education, where evaluating students is also a blend of subjective and objective. So in that vein, I am developing my own report card system. Congratulations, you are on the ground level as I tweak and play around with this idea.
The Gen-Ed Classes
If you read my article on Blackngoldhockey.com, you’ve already seen a sneak peek of this. I wanted to explore some more advanced metrics that might shed some light on a player’s general abilities, as well as position-specific abilities. The four metrics applied to players of all positions are:
Game Score (GS)
Originally developed by Dom Luszczyszyn in 2016 and updated in 2019, this metric is designed to measure single-game productivity. It takes into account goals, assists, points, expected points, blocks, PIMs, and usage, each weighted to the player’s position on the ice. This number is represented as an average over the course of a season. For context, in 2022-2023 Connor McDavid’s average GS was 1.97, and David Pastrňák’s was 1.55.
Wins Above Replacement (WAR)
Sometimes also known as Goals Above Replacement, this is another metric designed to assign a single number to quantify the value of a player’s contributions to the team. For context, in 2022-2023, McDavid also had the highest WAR score of 5.4.
Expected X
Individual Expected Goals Differential (ixG diff): for skaters, ixG is a quantitative measure of the quality of the scoring chances and goal-scoring abilities specific to that player. Comparing this number to the actual number of goals scored demonstrates if the player was able to meet or exceed expectations – that is what I am calling ixG diff. I am rating players as “meets expectations” when their ixG diff falls between -1 and 1.
Goals Saved Above Expected (GSAx): for goalies, GSAx compares a goalie’s performance to the expected goals based on the quality and quantity of shots faced. It quantifies the number of goals a goalie has prevented or allowed compared to what was anticipated. Positive GSAx values indicate above-average goaltending.
High Danger X
High Danger Goals For Percent (HDGF%): for skaters, HDGF% measures the percent of High Danger Goals scored by the player’s team, while that player is on the ice. High Danger Goals are goals scored from the area immediately surrounding the crease. The higher the percentage, the higher the value of the player.
High Danger Save Percentage (HDSV%): For goalies, HDSV% measures a goalie’s ability to stop shots from High Danger scoring areas (i.e., around the crease). The higher the value, the more skilled the goalie.
The Electives
The metrics that I am calling “Electives” are position-specific. While the “Gen-Eds” allow us to evaluate a player’s value to his club, the electives are individual metrics designed to evaluate the player as a single entity, as well as among other players in the same position.
These metrics are as follows:
For the skaters:
Individual Points Percentage (IPP)
IPP measures the percentage of goals scored by a team in which a skater earns a point. It reflects a skater’s involvement and impact on the team’s scoring. Higher IPP indicates a greater contribution to the team’s overall success. Not surprisingly, in the 2022-2023 season, Connor McDavid’s IPP of 80.53 was the highest in the league.
Defensive Point Shares (DPS)
DPS estimates the number of points a player contributes to their team’s success through defensive play. It considers ice time, plus/minus, penalty-killing performance, and defensive zone starts. Higher DPS values indicate strong defensive contributions. Hampus Lindholm lead the league in DPS in 2022-2023 with 7.2.
Player Usage (PU)
This is another statistic developed by Dom Luszczyszyn in which he calculated players’ Time on Ice (TOI) as a percentile, comparing the player to his teammates and opponents. It’s one way to demonstrate a team’s reliance on a particular player. I’ve put together a relatively simple, unweighted percentile rank to demonstrate player usage.
The percentiles are designed to rank the player in four ways:
Quality of Teammate D: How this player ranks among the defense on his team
Quality of Teammate F: How this player ranks among the forwards on his team
Quality of Competition D: How this player ranks among the defense on all opposing teams
Quality of Competition F: how this player ranks among the forwards on all opposing teams
Remember: a percentile is different from a percentage. Percentiles divide a dataset into 100 equal parts, with each part representing 1% of the data. In the above table, for example, Hampus Lindholm’s TOI is in the 100th percentile for QoTD. That means his TOI is greater than all of his other defensive teammates. Charlie Coyle’s QoTF TOI is in the 92nd percentile, meaning his TOI is greater than 92% of the remaining forwards on his team.
For the goalie:
Goals Saved Above Average (GSAA)
Quality Start Percentage (QS%)
Developed by Ron Vollman, QS% evaluates a goalie’s ability to deliver consistently solid performances. A Quality Start (QS) is defined as a game where the goalie achieves a save percentage equal to or higher than the league average save percentage. QS% is the percent of a goalie’s starts deemed a QS. Of all the goaltenders to play more than half the 2022-2023 season, Linus Ullmark led the way with a QS% of 0.896 – almost 90% of his starts were QS!
Delta (Fenwick) Save Percentage (ΔSV%)
This measures a goalie’s performance relative to the average save percentage of all goalies facing the same opponents. Fenwick shot attempts are all unblocked attempts. This metric indicates how well a goalie performs relative to the average. Of all goalies to start more than half of the season in 2022-2023, Ilya Sorokin holds the top stat with 1.97%.
The Grades
I evaluate each position-specific stat a little differently depending on the players’ position. For example, while any forward-skating player can have defensive capabilities (hello, Patrice Bergeron), I expect a defenseman, such as Connor Clifton, to have excellent DPS stats.
Each report card has the metric named, the player’s score, and notes. Notes indicate how that metric has changed from the 2021-2022 season to the 2022-2023 season. The notes are divided into three categories:
Production (comparison to the previous season):
Significantly Improved (SI)
Improved (I)
Maintained (M)
Declined (D)
Significantly Declined (SD)
Expectations (difference between expected result and actual result):
Exceeded Expectations (EE)
Met Expectations (ME)
Failed Expectations (FE)
Relative to the Average (relative to the league):
Above Average (AA)
Average (A)
Below Average (BA)
The Rubric
As you can see, the rubric is a work in progress. I predict that the majority of players who are getting regular minutes on a team, even if they are a fourth-liner, are scoring at least a C (whatever that might look like, soon), since that is broadly accepted as “average”. Any players who are “below average” are probably not playing much in the NHL to begin with.
The final grades I hand out are also qualitative in nature at this point. I’d like to develop a more quantitative method for assigning final grades in the future. I am evaluating these players as individuals first and as players in the National Hockey League second. This means I am taking into account their individual progress first and then how they performed relative to the league.
If it turns out that too many players are landing in the A-level, I will likely have to re-adjust the rubric. I will also need to consider how to assign a final grade when not all players are created equal. For example, “meets expectations” for Leon Draisaitl probably warrants an A/A+ but a “meets expectations” for Mike Reilly probably earns him something in the C- or B-level. This might look like an adjustment that is based on the percentile that the player’s expected metric falls relative to the rest of the league.
Things to Note:
Over the last decade, several different hockey stats pages have developed different ways to crunch the numbers. For example, Evolving-Hockey.com calculated Swayman’s GSAx as 20.99, but MoneyPuck.com has his GSAx as 24. One isn’t necessarily “right” and the other “wrong”; rather, it is indicative that different data scientists may adjust their calculations differently based on the aspects of the game they perceive to be more important. As a result, I’ve pulled these metrics from as few sources as possible to maintain consistent results. Those sources are:
For skaters:
GS: HockeyStatCards.com
WAR: Evolving-Hockey.com
ixG Diff: Hockey-Reference.com
HDGF%: NaturalStatTrick.com
IPP: NaturalStatTrick.com
DPS: Hockey-Reference.com
PU: MoneyPuck.com
For goalies:
GSAx: Evolving-Hockey.com
HDSV%: Hockey-Reference.com
GSAA: Evolving-Hockey.com
QS%: Hockey-Reference.com
ΔSV%: Evolving-Hockey.com
So stay tuned as the season gets started to watch as this model is tested and adjusted. I’m excited and I hope you are, too.
Cool stuff! As well as the sample players you have that are really good, it would be interesting to know what players the rubrics consider league average.