Last week I posted an article visualising the competition average Goal-Kicking Accuracy (GKA) for shots from every position on the ground. If you haven't read that article yet, I suggest having at least a quick squiz at it before continuing here.
In it I introduced the idea of valuing a team by the quality and number of goal-scoring chances they create1 rather than focusing purely on the final scores they kick. That is, looking at their ExpScore.
By counting ExpScore we are implicitly assuming that every team will convert the same shot with broadly the same consistency. A team with a gun forward line like Adelaide is just as likely to kick a goal as a struggling team like Brisbane given they both have set-shots 30 out, right in front. What differentiates these clubs is not their ability to kick these goals but instead their ability to get the ball into these positions consistently in order to kick these goals.
This may seem unrealistic at first, we all know that there are some dead-eye shooters and some others that are beyond hopeless, don't we? Why would we rate a chance in the hands of two differently skilled players with the same probability of going through? Well that's what this article is all about. Hopefully by the end of it I can convince you that this assumption is not so unreasonable and that ExpScore is a good measurement of team quality.
Team Conversion Accuracy from Game to Game is Essentially Random
If we watch a side kick well one week, we would naturally expect them to be innately more accurate kicks and for this to show up in a better than average accuracy the following week.
Similarly, if we watch a side butcher lots of chances we might think that they are poor converters and have less confidence in them converting any future chances.
As it turns out, this does not seem to be the case. If we compare the extent to which a team's actual score in a single game is either above or below their ExpScore, to how far above/below it is in the following game, we can see that GKA among clubs follows no real pattern at all.2
In the interactive plot above, if there were clear differences in GKA between the clubs, we would expect to see a pattern in the data. We would see a diagonal line going from the bottom left corner up to the top right. This would indicate that teams that kick poorly one week, are more likely to kick poorly again the next week and teams that kick better, more likely to continue to show their class.
Instead, what we see is essentially a random mess. If you kick 30 pts below your ExpScore (awful accuracy) one week, you are just as likely to kick above your ExpScore next week as you are to kick below it. The same goes for sides that kick 30pts above their ExpScore one week. GKA at a club level does not seem to be a repeatable skill.
A similar random pattern turns up if we look at average GKA (diff in actual score to ExpScore) over a whole season. Teams that outperform their ExpScore, tend to come back to earth the next season. As do teams that under-perform their ExpScore. 3
A Clearer Example
I admit, unless you're a bit of a stats geek, that previous plot might make very little sense to you. I'll try and clear things up a bit and explain what I mean when I say GKA is random. Here's how Geelong have kicked relative to their ExpScore so far this season.
A difference between actual score and ExpScore of anything under a goal is about what we would expect on average.4 This means that the team kicked at about the competition average in that match.
Of Geelong's 14 games this year, 9 of them fall into this "comp average" band. In two games they kicked with an accuracy above the competition average, and in their three other games they kicked with an accuracy below the competition average given the difficulty of the shots they created. This is a little below average when taken over the whole season, but certainly not cause to think they actually are below average. They kick above average almost as often as they kick below.
There also doesn't seem to be much of a pattern as to when the good games and the bad games occur. After one particularly poor goal-kicking match against Essendon in Rd 4, they followed up next week by kicking at an above average rate against Port. Whereas after their inaccurate Rd 8 against Adelaide, they followed up with another poor showing against Collingwood in Rd 9.
I'm sure in Rd 5, some were claiming that Geelong had "worked on their goal-kicking" in training and corrected their mistakes from the week before. The point is, this narrative is written after the fact. I'm sure the club went through fairly similar training drills in Rd 9 as well, but nobody is telling that feel-good story then.
Okay okay. But if we expect Geelong to kick within 6 points of their ExpScore, then how do you explain them kicking 31 points less in one game? Their must be some reason that they kicked this poorly. Maybe they had just received some bad news and their heads weren't in the game.
Maybe. But not necessarily. ExpScore is an expectation of a random variable. If we flip a coin 10 times we "expect" to get 5 heads and 5 tails. But if we instead got 3 heads and 7 tails we wouldn't immediately think that coin must be broken somehow. We wouldn't look for psychological reasons for why the coin didn't want to come up heads more often.
Actually, dropping a ball from your hands onto a swiftly moving boot directed towards goal is "random" in a similar way to the way flipping a coin is "random". Theoretically, you should be able to do it with precise, "every time" results. You control the coin and you control the boot, there are no hidden forces in play. But in practice, this is near impossible, no matter how good you are, you will miss sometimes. And sometimes those misses will come bunched together for no other reason then bad luck.
How Could Every Team's GKA Be The Same? Is There No Skill To Goal Kicking?
No. I'm not trying to sell you on that. There is obviously a load of skill in goal kicking. Chuck me the ball and I wouldn't go close to meeting the conversion probabilities shown in my Accuracy Maps.
We observe each team to be kicking with broadly the same accuracy for a few reasons:
- Firstly, something that is dubbed by financial director Michael Mausboussin as The Paradox of Skill.5 As the skill level of any feild increases, the differences between the skill of participants at the top level is too small to have any real impact. In lower levels of footy we will see real differences between skill levels of the best and the worst in a team. By the time we reach the top level, the AFL, every forward is so skilled and so well practiced at goal-kicking that there is very little difference between the bunch.Even if there are real differences in goal kicking skill between players, they are most likely too small to pick up with the relatively limited amount of shots that each player takes. Let's say two players take 4 shots each in a match6, let's also say that these chances each have about a 75% chance of being a goal if taken by an average player. Now imagine one player is very skilled, above average, and actually has about an 86% chance of kicking each chance of kicking each goal. The second player is instead a bit below average skill, he only holds a chance of about 64% of kicking each goal.
From their 4 shots taken, we can expect these two players to each kick 3 goals more often than they kick more or less, despite the gulf in skill between them.Now imagine these two players are much closer in skill level then this. Even over a season of roughly 100 shots, it would still be very hard to say for sure which player is a better kick.
- Assuming that some players do kick with greater accuracy than others does not necessarily mean we expect some teams to kick better than others. Teams are made up a bunch of players with different abilities. With a salary cap system in place it is unlikely that one team would have all of the best kickers. Rather, it is more likely that the best kicker at a club is offset by players of (still high) but slightly lesser ability.
- GKA based of the empirical accuracy maps I posted last week does not show a perfect, "true" probability of kicking a goal from each spot on the ground. Rather it shows the proportion of shots from within 5m of that spot on the ground that did end up going through the big sticks. This unfortunately leads to some "self-selection" issues.7It's not true that the "average" player can kick a goal from 55m out about 50% of the time. But rather that the "average" shot from 55m out goes through about 50% of the time. This is because the only players that take shots from that far out are ones that know they can kick them.8 A similar selection happens for shots deep in the pocket. Not to mention those few players that try from places like this.
This phenomenon does not effect the validity of ExpScore too much as we are rarely overvaluing these shots from tough positions for the simple fact that not many players that aren't part of this top-echelon are attempting them. But it is an element of "player skill" that is being hidden by the system, making all teams look more similar in conversion ability then they may be.
I feel this explanation fits a bit better with our personal experience with football then just saying, every forward line has the same accuracy from everywhere.
Where Does This Leave Us?
There are many other examples and interesting theories I'd like to go through regarding Goal-Kicking Accuracy, but this article is coming up on 2000 words, which is probably pushing the bounds of most people's commitment to an amateur analytics blog as it is. Why would I go to all of this effort in order to convince you that GKA is an essentially random process in the AFL?
If GKA is random, then all that differentiates the teams is their ability to create high quality chances and prevent their opponents from creating them. This allows us to rate teams irrespective of how well they have been converting and whether they have been winning or losing recently.
A team that has been under-performing their ExpScore can be expected to improve their accuracy in the future. This may result in more wins than we would have otherwise expected.
Similarly, if a team's opponent's have been significantly outperforming their ExpScore, we can expect this to come down in the following weeks and for the team to be broadly more competitive on the scoreboard in the future.
I will spend a lot more time on ExpScore in the future, both in research and in the analysis of individual games. I really do think it's a leap forward in the way we think about footy. If you have any questions or further ideas please send them through. The best place to find me is either the comments section of this article or on Twitter.
- compared to those their opposition creates
- Remember that if a team kicks a score above their ExpScore it means they converted at a higher rate than the average team would have given the chances they had. They had above average conversion accuracy. If a team kicks a score below their ExpScore, it means they converted at a lower rate than average. They had below average conversion accuracy.
- Hawthorn seem to be a slight exception to this, they have consistently outperformed their season avg GKA over the last 4 years. Upon closer inspection you can see that the bulk of their "strong" GKA comes from open-play shots, and they are much closer to average in set-shots. This is likely due to them actually creating higher quality open-play shots than average. Remember, as it stands now, a snap over your head while being tackled is rated with the same difficulty as a shot from the same spot on the ground that comes uncontested and with time to settle. Hawthorn likely create more of the latter, simple shots due due to well worked plays. I'm currently working on a system which will differentiate between open-play shots and hopefully provide more precise results. Keep your eyes peeled for this in the next couple of weeks.
- It's important to note that it's unlikely that score and ExpScore match up completely. This is because each shot has an ExpScore somewhere between 1 and 6, but when we take the shot we score the discrete value of either 1 or 6. A set shot 40 out on a slight angle may have an ExpScore of 3.5, but we can never actually kick 3.5 points with a shot. Over many shots, the difference between score and ExpScore will average to zero. But over a game it will usually be at least few points off.
- Thanks to Darren O'Shaugnessy (@RankingSw) for getting me onto this.
- This is probably a pretty reasonable number of chances for a forward to have these days
- I'm currently exploring a few parametric methods of accuracy mapping that will help account for this, but they are still a work in progress for the time being
- Think of Buddy or Tex wheeling around and letting loose, it looks spectacular purely because few others can do it.