Progress update (Diagnostics)

[nick257]

Hi All,

We’ve been making several changes to the design of the system over the last few weeks.

Most importantly, we’ve built a new diagnostics system, that helps us understand why our “Live” betting system sometimes makes different decisions to our prediction models, which are run in our “Simulation” system.

There is a lot of data to crunch. We operate over 100 individual variables (predictors) for each runner in each race. In a typical day, with 30 races, averaging 10 runners, that’s 30,000 data elements that need to be cross-checked across our two systems.

The good news is that the diagnostics system now lets us check all elements of the system (bet blocking rules, predictor values, and much more). For example, the diagnostic system tells us which predictors are diverging, how often (count by runner), and by how much (STDEV).

Using the results from the diagnostic system, over the last 3 weeks, we’ve found/fixed a number of issues. At this stage, it looks like the changes we’ve made are having the right impact. The system is now operating to within 70% of the profits predicted by our models. Profits, Live System versus Simulation System

So, it’s better, but not quite there yet.

There is one last cluster of 22 predictors that refuse to align… These predictors all relate to a single issue – which is pre-race Jockey name changes.

Jockey changes affects, on average, 3% of runners, but appears to have a disproportionate affect (30%) on predicted profits. As an example, before each race, we often see a jockey name of “reserve”, or a simply a complete substitution of name, at the time of the race start. We have tried to check to see if Jockey name changes were more frequent (pre COVID) in the past – but unfortunately our data doesn’t go back far enough to tell..

To overcome Jockey changes, we’re working on ways to obtain accurate Jockey names just before each race (probably via website scraping). If any of you know a reliable data source for jockey names – please ping me via the messenger service on our website.

Once we’ve solved the Jockey problem, we expect to return to live betting.

All the best

Nick

[Author]

Posts