For the long term Fat Stats fans (all three of you), you might remember that in 2017 we built a predictive model and sent out email predictions. Unfortunately, AutoTippa died in a nasty github accident, and it has taken me a full 18 months to recover from the pain enough to rebuild it. AutoTippa II, or ATII for short, is now alive and kicking and is quite different to AutoTippa the first, which for spiritual reasons we will no longer discuss. RIP.
AutoTippa II (The retippering) was built trying to replicate the process we go through subconsciously when we do our tipping every Friday. In general, we think good tippers break this process down into four main components:
Long term record - Is the team Carlton or Gold Coast and therefore awful?
Recent record between teams - What happened last time?
Venue Advantages - how does this team play at Subiaco or Jiangwan Stadium?
Player Metrics - is Jake Stringer (best forward in the game) playing?
Literally hundreds of variables were created, found to be wrong due to horrible coding, fixed, tested and retested but in end we settled on 22 metrics which are probably not wrong (update: one is partially wrong). We have several metrics we are still working on, but time has got the better of us for now (2020 job). The selected variables are fed into a random forest model, a type of machine learning algorithm renowned for being easy for users to not screw up. The algorithm outputs probabilities which we map to predicted margins, and away we go.
In our opinion, the strength of AutoTippa II is the unique features we are feeding in. A lot of public models are modifications of ELO and while strong, cannot take into account who is actually playing in the game. We feel these models suffer from not reacting to short term fluctuations and it was for this reason we focused on the player aspect most in our feature creation. An example of where this would have benefited us is the great Essendon fiasco of 2017 - our model would have adjusted to having all the players back, whereas ELO models underestimated Essendon for some time while they recalibrated from the 2016 monstrosity of a season. Note: after reviewing the new squiggle teams it seems like everyone has player input now. Missed the boat.
Long term record:
This one was simple - ELO models have been shown to be strong predictors, and the guys at fitZroy made it easy for us by writing the code out for us. Speaking of fitZroy - this entire model is built around the fitZroy data scrape and fixture functions . AutoTippa the first used to manually scrape the data - the amount of time saved by fitZroy is huge and those guys deserve a lot of credit.
All sorts of variations of rolling averages, margins, scoring shot ratios etc were trialed here but in the end two simple metrics were retained - the margin the last time the teams played, and the 3 game rolling average margin between the teams. These consistently outperformed the others and results were often highly correlated. I am a simple Tasmanian and I like to keep it simple.
In all honesty, there has been a lot of work done on this by other bloggers much smarter than us and we did not put anywhere near as much effort in. Some of our funny results are no doubt a product of this. Our variables we settled on to try and capture home ground advantage are each team's record at that venue over the last 5 games (or the mean of every away team at that ground if less than 5 games played) and also the distance between that teams home ground and the venue.
These metrics make a difference; before we included them funny games like Geelong (very strong at home) v Richmond (who at the time, were breaking records at the MCG) played at the MCG as a Geelong home game in R13 last year. By adding in these particular metrics, the algorithm can tease out that even though Geelong is the home team, they are still travelling, and even though Richmond is the away team, they have won by an average of 25 points at the G over the last 5 games. Unfortunately, we do not include free kicks for in the model, which drastically reduces certain Geelong player's influence on the model result.
These metrics are the driving power behind AutoTippa II, and took by far the most time to generate. 13 metrics are calculated, 4 offensive, 4 midfield and 4 defensive and one overall statistic we call “Fatscore”. Our goal with our player metrics was to capture not only players short term fluctuations in form, but also the variability caused by players being in or out of the team.
For all of the below metrics, “expected” statistics were used. These were simply built using 5 game averages for the players playing in that game. So if Nat Fyfe has averaged 5 inside 50s a game over his previous 5 games, we can assume he will get that somewhere around that again. Metrics were normalised to the 10 game rolling league average to get a percentage value and relative numbers are used in the model. For example, if Essendon have an Forward Pressure rating of 1.2, this means they are 20% better than the league average for the past 10 games. They are playing Geelong, who in this mythical scenario still have Daniel Menzel, and they have a forward pressure of 0.7. Subtracting Geelong's pitiful rating from Essendon's rating gives us a relative value of 0.5 or +50% to Essendon, which is the value the model uses. Formulas and descriptions of the player metrics will be added to the ATII Glossary, time permitting.
Fat Score is our equivalent of Supercoach or dream team scores - an amalgamation of weighted statistics to try and get an idea of a players expected influence on the game. We will do a whole blog piece on it at some point, but in general to calculate our weightings we used simple linear regressions to try and get an idea of how "important" variables were for when predicting margin. We calculate separate weights for each position, and then did some (pretty fuzzy) manipulation/normalisation of the variables to try and ensure that all positions had relatively equal impacts on a game. See the glossary for the actual formulas (coming soon). Fat Score is our most likely metric to alienate us and anger people - we know this, but rightly or wrongly, the number is entirely mathematically derived. Its not our fault Jake "the package" Stringer is criminally under-recognised in his ability to win a game off his own boot.
You will notice that some of the above formulas were position dependent. Whilst some website allocate positions to players, it tends to be on a season by season basis. We wanted to capture any higher frequency positional changes, so we built a SOM using the raw statistics for all players since 2003. We call this FATMAP and we will introduce it its own blog post at some point. Essentially by plugging in a players statistical profile (5 game rolling averages), we can assign a broad position (Forward, Midfield, Ruck, Defender) or a refined position (28 in total, not shown yet). This position updates as a player's role changes over time. It also is a handy way of tracking a player heading towards being elite but more on that later.
We have measured ourselves against the bookies, which is a reasonable benchmark to aim for as in theory represents the tipping of the masses. Our model averages 71.3% accuracy over the past 7 years which is getting hauntingly close to our target of 72% - on average the success rate you get if you choose the averaged favourite with a bookmaker. We can't find a good database of their historical performance but we are almost certain we are outperforming Jon Ralph and Caroline Wilson as well.
All of the historical results and future predictions are updated in an app here:
The app is very basic at the moment, but we have generated a lot of interesting statistics in the process of building ATII, so we have a laundry list of things we would like to add including but not limited to:
Player comparison tool
Player trajectory tool
Time permitting, we hopefully will have these up and going over the next few weeks. We are also hoping to have chaRlie running again with a few small tweaks by round 5 or 6.
Watch this space!
In exciting news, we have achieved a life long (well at least three years) dream because as of today we are now on squiggle! check it out here: