Decision Making: why you need to know about the Bayes Theorem

In this section of the blog we will take you into the process of decision making, and we will start with introducing you the Bayes Theorem and why it is so important in the decision making process.

But let’s start with the beginning: how do you make a decision?

Decision Making and Probabilities

If you decide to go out without an umbrella, it could be because you just forgot it, but a more likely reason is that you think it will not rain. The weather is quite impossible to forecast with a 100% certainty (especially in my hometown in Ireland), but somehow you have evaluated that the probability of rain was low (let’s say <10%) and that carrying an umbrella with you was not worth the trouble compared to the benefit of having it in the unlikely event of rain.

We are used to make such decisions unconsciously and this is the basis of Risk Management and Decision Making.

Let’s take another example: you are a Retailer and need to decide how many pieces of Item X you need to carry in your store to avoid running out of stock and losing sales. If you decide to carry 5 pcs of Item X (for which you receive replenishment every day) in Stock, it is (or at least it should be) because you have evaluated that you almost never sell more than 5 pcs a day, and that if you do this happens so rarely (let’s say less than 1% of the time) that you are willing to accept the risk of running out of Stock 1% of the time vs. the cost of carrying additional Inventory to prevent against all the possible odds.

This probabilistic decision making process requires a deep understanding about the events of this world (such as rain or making more than 5 sales per day), now how do we evaluate them?

Rethinking Reality: nothing is certain!

Evaluate or Estimate the probabilities about the events of the world are carefully chosen words. This probabilistic approach invites us to rethink Reality and what we hold for certain.

A prediction like The Sun will rise tomorrow sounds so obvious that most of us would hold it as a universal truth. The probabilistic decision maker would instead say The Sun will rise tomorrow with a 99% chance. Then, every day, as the Sun rises, the probabilistic decision maker refines his estimate which eventually becomes The Sun will rise tomorrow with 99.9% chance, then 99.99% chance, then 99.99999% chance. However the probabilistic decision maker will never give into the certainty of holding The Sun will rise tomorrow statement as an absolute truth. For him, nothing is certain in this world and 100% probability does not exist! (as a matter of fact we now know that in about 5 billion years from now the Sun will begin to die, so eventually one day The Sun will NOT rise tomorrow!)

Therefore we will never know for sure the real probabilities necessary for evaluating risks and making decisions like carrying an umbrella, selling more than 5 pcs per day, or seeing the Sun rise tomorrow. However what we can do is Estimate them through Observations and Tests.

It is very important to make the distinction between Tests and Absolute Reality as they are not the same thing and Tests incorporate a risk of error:

  • Tests and Reality are not the same thing: for example being tested positive for Cancer and having Cancer are not the same thing
  • Tests are flawed: Tests can be wrong. For example you can be tested positive for Cancer and not have Cancer at all (this is called a false positive) or being tested negative for Cancer and have it (this is called a false negative)

The Bayes Theorem and its applications

Instead of holding Universal Truths, we are now invited to think the world (even the most certain things like the Sun rising every day) in terms of probabilities, and to evaluate these probabilities through Objective Tests and Observations, and continuously refine these estimates as new evidence comes up.

In a probabilistic world, this translates into the Bayes Theorem:

Bayes Theorem:

P(A¦X) = P(X¦A) * P(A)  /  P(X)

i.e. probability of A happening knowing X happened = probability of X happened knowing A as true (true positive) * probability of A happening / probability of X happening

or its equivalent form

P(A¦X) = P(X¦A) * P(A) / ( P(X¦A) * P(A) + P(X¦not A) * P(not A) )

Now let’s see how the Bayes Theorem works on a practical example. Let’s try to evaluate P(A¦X) the probability of having Cancer (A), following the result of a positive test X

Prior probability of having Cancer before the test P(A) = 1%

We know that P(not A) = 1 – P(A) = 99%

New Event occurs: tested positive for Cancer

  • P(X¦A) is the true-positive probability of having Cancer knowing that you have been tested positive = 80%
  • P(X¦notA) is the false-positive probability of being tested positive if you do not have cancer have cancer:  = 10%

Posterior probability

P(A¦X) = P(X¦A) * P(A) / ( P(X¦A) * P(A) + P(X¦not A) * P(not A) ) = 7.5%

The Bayes Theorem invites us to start with an initial estimate of 1% chance of having cancer, which will increase to 7.5% after having being tested positive, incorporating the risks of true and false positive.

A second positive test would increase the probability of having cancer further

Prior probability of having Cancer before the test P(A) = 7.5%

We know that P(not A) = 1 – P(A) = 92.5%

New Event occurs: tested positive for Cancer

  • P(X¦A) is the true-positive probability of having Cancer knowing that you have been tested positive = 80%
  • P(X¦notA) is the false-positive probability of being tested positive if you do not have cancer have cancer:  = 10%

Posterior probability

P(A¦X) = P(X¦A) * P(A) / ( P(X¦A) * P(A) + P(X¦not A) * P(not A) ) = 41%

After this second positive test we know have 41% chance of having cancer.

Bottom Line

The Bayes theorem is all about acknowledging that we do not know for sure about the events in the world, that we need to think about them probabilistically and that we need to refine our estimates of these probabilities as new data becomes available

Old Forecast + New & Objective data = New Forecast

This sounds obvious but it is the core of Forecasting, Risk Management and Decision Making

AnalystMaster

Football Predictions for Premier League, La Liga and Serie A

Let’s kick off 2018 with our latest Football Predictions!

Premier League Week 22

On the contrary to Bing, we prefer to bet on Brighton vs. Bournemouth in a game that we expect to be close.

After 3 consecutive games without winning, we believe than Manchester United will continue to have a difficult time at Everton in a game too close to call.

Finally in the Arsenal – Chelsea derby we believe that the Gunners will make good use of their Home Advantage and we give them slightly favourable odds.

Premier League Week 22 AnalystMaster Bing
Home Away H D A H D A
Brighton Bournemouth 45% 28% 27% 22% 28% 50%
Burnley Liverpool 32% 25% 43% 29% 21% 50%
Leicester Huddersfield 52% 24% 24% 58% 28% 14%
Stoke Newcastle 40% 25% 35% 46% 36% 18%
Everton Man United 37% 25% 39% 12% 25% 63%
Southampton Crystal Palace 45% 28% 28% 58% 28% 14%
West Ham West Brom 39% 28% 33% 40% 27% 33%
Swansea Tottenham 21% 24% 54% 17% 16% 67%
Man City Watford 66% 18% 16% 89% 5% 6%
Arsenal Chelsea 42% 25% 33% 32% 25% 43%

La Liga Week 18

Despite benefiting from the Home Advantage, we believe that Valencia will be challenged by Girona, with a win % of 49%, much lower than Bing’s 81%.

We agree with Bing that the Leganes-Socidedad game is too close to call, with each side having equal winning probabilities.

La Liga Week 18 AnalystMaster Bing
Home Away H D A H D A
Ath Madrid Getafe 50% 29% 21% 58% 28% 14%
Valencia Girona 49% 24% 27% 81% 15% 4%
Las Palmas Eibar 43% 24% 34% 45% 23% 32%
Sevilla Betis 64% 21% 14% 89% 5% 6%
Leganes Real Sociedad 36% 27% 36% 25% 39% 36%
Barcelona Levante 72% 18% 10% 89% 5% 6%
Ath Bilbao Alaves 46% 30% 24% 58% 28% 14%
Vilarreal La Coruna 63% 21% 16% 89% 5% 6%
Celta Vigo Real Madrid 25% 23% 51% 12% 25% 63%
Malaga Espanyol 44% 25% 31% 64% 20% 16%

Serie A Week 20

We have some differences here with Bing, as we predict the games Chievo vs. Udinese and Fiorentina vs. Inter Milan too close too call

Genoa-Sassuolo is also expected to be a very close game, with the home side having slightly favourable odds.

As usual our odds are lower than Bing for the Napoli, AC Milan, Roma and Juventus games, although we both agree that a win from these top teams is the most likely outcome.

Serie A Week 20 AnalystMaster Bing
Home Away H D A H D A
Chievo Udinese 36% 25% 39% 56% 28% 16%
Fiorentina Inter 37% 26% 37% 8% 16% 76%
Torino Bologna 42% 26% 32% 56% 28% 16%
Benevento Sampdoria 29% 23% 48% 17% 16% 67%
Genoa Sassuolo 40% 25% 35% 25% 39% 36%
Spal Lazio 27% 23% 50% 17% 16% 67%
Napoli Verona 67% 19% 14% 89% 5% 6%
AC Milan Crotone 59% 23% 18% 89% 5% 6%
Roma Atalanta 52% 24% 24% 89% 5% 6%
Cagliari Juventus 24% 21% 56% 8% 16% 76%

Happy New Year 2018 everyone!!

AnalystMaster

Year End Football Predictions

 

Premier League Week 21

The notable difference with Bing Predicts is on the Watford-Swansea and Huddersfield-Burnley games that we predict too close to call, whereas Bing seems to see a pretty clear victory of the Home side for both games.

Bing Predictions for the last 3 games on Dec 31st are not available at this time.

Premier League Week 21 AnalystMaster Bing
Home Away H D A H D A
Chelsea Stoke 64% 20% 16% 81% 15% 4%
Bournemouth Everton 45% 26% 29% 55% 31% 14%
Liverpool Leicester 55% 24% 22% 45% 25% 30%
Watford Swansea 37% 26% 38% 52% 25% 23%
Huddersfield Burnley 37% 29% 34% 46% 36% 18%
Newcastle Brighton 39% 30% 30% 55% 31% 14%
Man United Southampton 64% 23% 13% 89% 5% 6%
Crystal Palace Man City 18% 21% 61% N/A N/A N/A
West Brom Arsenal 35% 26% 39% N/A N/A N/A
Tottenham West Ham 64% 21% 15% N/A N/A N/A

Serie A Week 19

Overall our predictions are similar to Bing, with the notable exception of the Bologna-Udinese game which we think will be very close whereas Bing foresees an easy Bologna victory. We also believe that the game between Inter and Lazio will be very challenging to predict as well, Inter only having a 43% victory odd vs. 55% for Bing.

As usual we believe the 89% that Bing predicts for Sampdoria, Atalanta and Roma wins is too high and our victory probability is in the 60s %

Serie A Week 19 AnalystMaster Bing
Home Away H D A H D A
Crotone Napoli 17% 22% 61% 17% 16% 67%
Fiorentina Milan 47% 23% 30% 55% 31% 14%
Bologna Udinese 36% 25% 40% 81% 15% 4%
Benevento Chievo 28% 25% 48% 12% 25% 63%
Sampdoria SPAL 63% 21% 16% 89% 5% 6%
Atalanta Cagliari 60% 23% 17% 89% 5% 6%
Roma Sassuolo 60% 22% 18% 89% 5% 6%
Torino Genoa 45% 27% 28% 42% 28% 30%
Inter Lazio 43% 22% 35% 55% 31% 14%
Verona Juventus 17% 20% 64% 17% 16% 67%

We hope you all enjoy these last 2017 games, we will come back strong in 2018 with more Football articles and predictions!

Happy New Year everyone!

AnalystMaster

AnalystMaster 1-0 BingPredicts and Boxing Day Predictions

BingPredicts challenge results

Last week we challenged BingPredicts over their Football Predictions over Premier League, La Liga, Ligue 1 and Serie A predictions.

AnalystMaster wins the challenge with 49% of correct predictions vs. 44% for BingPredicts.

In Particular AnalystMaster correctly predicted Barcelona’s win over Real Madrid in El Clasico, or better anticipated the difficulties of Man United at Leicester

Boxing Day Predictions

You will find below our predictions and the ones from BingPredicts for Boxing Day.

They are very similar, we only disagree on the Watford-Leicester game where we predict a tough game with a Leicester win being more likely, whereas Bing bets on a Watford victory.

We also foresee Arsenal having a tough time at Crystal Palace in the last game too close to call, where Bing sees an easy Arsenal win.

Happy Boxing day everybody!

AnalystMaster

Premier League Week 20 AnalystMaster Bing
Home Away H D A H D A
Tottenham Southampton 60% 24% 16% 56% 28% 16%
Man United Burnley 62% 24% 15% 89% 5% 6%
Watford Leicester 32% 23% 45% 56% 28% 16%
Chelsea Brighton 59% 25% 16% 50% 42% 8%
West Brom Everton 44% 27% 30% 64%
Bournemouth West Ham 46% 24% 30% 48% 24% 28%
Huddersfield Stoke 48% 26% 26% 64%
Liverpool Swansea 62% 24% 14% 52% 28% 30%
Newcastle Man City 16% 22% 62% 17% 16% 67%
Crystal Palace Arsenal 36% 25% 39% 17% 16% 67%

Football Predictions: AnalystMaster vs. BingPredicts

Challenging Bing Predicts on Football Predictions

Bing Predicts prides itself at being great in forecasting the outcome of some events. A quick look at its wikipedia page shows that it predicted the winners from American Idol or The Voice with more than 80% accuracy, US elections with more than 90% accuracy and that they did a perfect 15 out of 15 predictions in the 2014 World Cup! Well done Bing Predicts!

At AnalystMaster we also have our Predicting algorithms. We decided to challenge Bing Predicts on the next European Football games by comparing our Predictions with theirs. Although our predictions match on most games, there are few significant differences including “El Clasico” between Real Madrid and Barcelona, Leicester vs. Manchester United or the next Bayern Munich and Juventus games.

See you next week for analysing the results!

AnalystMaster

Premier League Week 19

The first thing that strikes when looking at Bing Predicts forecasts is that it sometimes gives very important winning probabilities.

The Leicester vs. Man United game is a good example. Bing Predicts foresees an easy Man United victory (75%) whereas we think Jose Mouriho’s team will face a much tougher opposition from a team who just took Pep Guardiola’s Man City to penalty kicks in League Cup this week and will be benefiting from the Home Advantage.

Premier League Week 19 AnalystMaster Bing
Home Away H D A H D A
Arsenal Liverpool 48% 21% 31% 58% 28% 14%
Everton Chelsea 32% 24% 43% 17% 16% 67%
Stoke West Brom 33% 28% 39% 48%
Southampton Huddersfield 50% 25% 25% 42%
Swansea Crystal Palace 33% 30% 37% 34% 31% 35%
Man City Bournemouth 66% 19% 14% 89% 5% 6%
West Ham Newcastle 45% 27% 28% 55% 31% 14%
Brighton Watford 33% 26% 41% 29% 28% 43%
Burnley Tottenham 35% 28% 37% 8% 16% 76%
Leicester Man United 36% 26% 39% 6% 19% 75%

La Liga Week 17

Our algorithm disagrees with Bing Predicts regarding “El Clasico”. Bing Predicts bets on an clear victory from Real Madrid (56%), probably giving a lot of weight to the Home Advantage.

This could be surprising to who followed the two teams this year: Real Madrid has been struggling this year and is currently 4th of La Liga so far with 31 pts and losing to Tottenham in Champions League, whereas Barcelona has been outrageously dominating the championship with 42 pts and no defeat so far.

Our algorithm on the other side predicts that this game will be too close to call, giving Barcelona a narrow edge at 39% vs. 37% for a Read Madrid win.

La Liga Week 17 AnalystMaster Bing
Home Away H D A H D A
Levante Leganes 35% 28% 37%
Getafe Las Palmas 58% 24% 18% 81% 15% 4%
Real Sociedad Sevilla 50% 23% 27% 29% 28% 43%
Eibar Girona 38% 25% 37% 32% 35% 33%
Alaves Malaga 46% 26% 28% 33% 38% 29%
Betis Ath Bilbao 48% 23% 29% 34% 30% 36%
Espanyol Ath Madrid 28% 28% 45% 29% 28% 43%
Real Madrid Barcelona 37% 24% 39% 56% 28% 16%
Valencia Villarreal 62% 22% 17% 81%
La Coruna Celta Vigo 32% 24% 44% 29% 28% 43%

Serie A Week 18

Again in Serie A Bing Predicts gives very high probabilities to wins of Udinese (89%), Genoa (81%) or Napoli (89%) which seems quite bold for a game which is in the end quite random. Although we also believe these teams will win, we would not be ready to bet on victories with these odds.

We also predicts that Juventus and AC Milan will have a tougher opposition than Bing Predicts thinks with a winning probability of 42% and 45% vs. a bigger margin of 64% for Bing

Serie A Week 18 AnalystMaster Bing
Home Away H D A H D A
Chievo Bologna 41% 26% 33% 31% 36% 33%
Cagliari Fiorentina 34% 25% 41% 6% 19% 75%
Lazio Crotone 65% 20% 16% 89% 5% 6%
SPAL 2013 Torino 36% 25% 39% 25% 29% 46%
Genoa Benevento 54% 24% 23% 81% 15% 4%
Udinese Verona 46% 24% 30% 89% 5% 6%
Sassuolo Inter Milan 23% 27% 50% 17% 16% 67%
Napoli Sampdoria 59% 22% 19% 89% 5% 6%
AC Milan Atalanta 45% 27% 28% 64%
Juventus Roma 42% 24% 34% 64% 20% 16%

Ligue 1 Week 19

Here too Bing Predicts very high odds on some games, especially the 89% for Monaco vs. Rennes. Our algorithm also largely favours a Monaco win, but gives Rennes (who has been doing quite well in the last few games) more chances to draw or to even win that game.

We also think Amiens will exploit their Home Advantage to challenge Ranieri’s FC Nantes at home, and that Montpellier will bring a much tougher opposition to Bordeaux that Bing Predicts seems to believe.

Ligue 1 Week 19 AnalystMaster Bing
Home Away H D A H D A
Angers Dijon 42% 25% 33% 27% 33% 40%
Lille Nice 43% 28% 29% 56%
Toulouse Lyon 25% 24% 51% 21% 33% 46%
Amiens Nantes 48% 27% 25% 35% 23% 42%
Guingamp St Etienne 55% 27% 18% 42% 25% 33%
Marseille Troyes 61% 21% 18% 55% 31% 14%
Monaco Rennes 68% 18% 14% 89% 5% 6%
Paris SG Caen 75% 16% 9% 89% 5% 6%
Bordeaux Montpellier 39% 29% 32% 55% 31% 14%
Metz Strasbourg 30% 24% 46% 29% 21% 50%

Bundesliga Week 18

Our algorithm predicts a tough time for Bayern Munich at Leverkusen, with a game too close too call. This prediction sounds quite bold when we look at the rankings today: Bayern Munich is at the top of the Bundesliga with 42 pts and Leverkusen at a distance in 4th place with 28 pts. Betting on a Munich win or at least a draw would sound like the reasonable prediction, however we have chosen to leave the algorithm’s results as is as we think that the fact that it gives this game almost equal odds is interesting in itself.

Bing has yet to provide its predictions for Bundesliga matches at this time.

Bundesliga Week 18 AnalystMaster Bing
Home Away H D A H D A
Leverkusen Bayern Munich 39% 24% 37%
Werder Bremen Hoffenheim 39% 24% 37%
Frankfurt Freiburg 52% 22% 26%
Stuttgart Herta Berlin 42% 29% 28%
Augsburg Hamburg 53% 23% 24%
Hannover Mainz 58% 22% 20%
Leipzig Schalke 04 49% 23% 27%
Koln M’gladbach 39% 24% 37%
Dortmund Wolfsburg 56% 22% 23%