Decision Making: why you need to know about the Bayes Theorem

In this section of the blog we will take you into the process of decision making, and we will start with introducing you the Bayes Theorem and why it is so important in the decision making process.

But let’s start with the beginning: how do you make a decision?

Decision Making and Probabilities

If you decide to go out without an umbrella, it could be because you just forgot it, but a more likely reason is that you think it will not rain. The weather is quite impossible to forecast with a 100% certainty (especially in my hometown in Ireland), but somehow you have evaluated that the probability of rain was low (let’s say <10%) and that carrying an umbrella with you was not worth the trouble compared to the benefit of having it in the unlikely event of rain.

We are used to make such decisions unconsciously and this is the basis of Risk Management and Decision Making.

Let’s take another example: you are a Retailer and need to decide how many pieces of Item X you need to carry in your store to avoid running out of stock and losing sales. If you decide to carry 5 pcs of Item X (for which you receive replenishment every day) in Stock, it is (or at least it should be) because you have evaluated that you almost never sell more than 5 pcs a day, and that if you do this happens so rarely (let’s say less than 1% of the time) that you are willing to accept the risk of running out of Stock 1% of the time vs. the cost of carrying additional Inventory to prevent against all the possible odds.

This probabilistic decision making process requires a deep understanding about the events of this world (such as rain or making more than 5 sales per day), now how do we evaluate them?

Rethinking Reality: nothing is certain!

Evaluate or Estimate the probabilities about the events of the world are carefully chosen words. This probabilistic approach invites us to rethink Reality and what we hold for certain.

A prediction like The Sun will rise tomorrow sounds so obvious that most of us would hold it as a universal truth. The probabilistic decision maker would instead say The Sun will rise tomorrow with a 99% chance. Then, every day, as the Sun rises, the probabilistic decision maker refines his estimate which eventually becomes The Sun will rise tomorrow with 99.9% chance, then 99.99% chance, then 99.99999% chance. However the probabilistic decision maker will never give into the certainty of holding The Sun will rise tomorrow statement as an absolute truth. For him, nothing is certain in this world and 100% probability does not exist! (as a matter of fact we now know that in about 5 billion years from now the Sun will begin to die, so eventually one day The Sun will NOT rise tomorrow!)

Therefore we will never know for sure the real probabilities necessary for evaluating risks and making decisions like carrying an umbrella, selling more than 5 pcs per day, or seeing the Sun rise tomorrow. However what we can do is Estimate them through Observations and Tests.

It is very important to make the distinction between Tests and Absolute Reality as they are not the same thing and Tests incorporate a risk of error:

  • Tests and Reality are not the same thing: for example being tested positive for Cancer and having Cancer are not the same thing
  • Tests are flawed: Tests can be wrong. For example you can be tested positive for Cancer and not have Cancer at all (this is called a false positive) or being tested negative for Cancer and have it (this is called a false negative)

The Bayes Theorem and its applications

Instead of holding Universal Truths, we are now invited to think the world (even the most certain things like the Sun rising every day) in terms of probabilities, and to evaluate these probabilities through Objective Tests and Observations, and continuously refine these estimates as new evidence comes up.

In a probabilistic world, this translates into the Bayes Theorem:

Bayes Theorem:

P(A¦X) = P(X¦A) * P(A)  /  P(X)

i.e. probability of A happening knowing X happened = probability of X happened knowing A as true (true positive) * probability of A happening / probability of X happening

or its equivalent form

P(A¦X) = P(X¦A) * P(A) / ( P(X¦A) * P(A) + P(X¦not A) * P(not A) )

Now let’s see how the Bayes Theorem works on a practical example. Let’s try to evaluate P(A¦X) the probability of having Cancer (A), following the result of a positive test X

Prior probability of having Cancer before the test P(A) = 1%

We know that P(not A) = 1 – P(A) = 99%

New Event occurs: tested positive for Cancer

  • P(X¦A) is the true-positive probability of having Cancer knowing that you have been tested positive = 80%
  • P(X¦notA) is the false-positive probability of being tested positive if you do not have cancer have cancer:  = 10%

Posterior probability

P(A¦X) = P(X¦A) * P(A) / ( P(X¦A) * P(A) + P(X¦not A) * P(not A) ) = 7.5%

The Bayes Theorem invites us to start with an initial estimate of 1% chance of having cancer, which will increase to 7.5% after having being tested positive, incorporating the risks of true and false positive.

A second positive test would increase the probability of having cancer further

Prior probability of having Cancer before the test P(A) = 7.5%

We know that P(not A) = 1 – P(A) = 92.5%

New Event occurs: tested positive for Cancer

  • P(X¦A) is the true-positive probability of having Cancer knowing that you have been tested positive = 80%
  • P(X¦notA) is the false-positive probability of being tested positive if you do not have cancer have cancer:  = 10%

Posterior probability

P(A¦X) = P(X¦A) * P(A) / ( P(X¦A) * P(A) + P(X¦not A) * P(not A) ) = 41%

After this second positive test we know have 41% chance of having cancer.

Bottom Line

The Bayes theorem is all about acknowledging that we do not know for sure about the events in the world, that we need to think about them probabilistically and that we need to refine our estimates of these probabilities as new data becomes available

Old Forecast + New & Objective data = New Forecast

This sounds obvious but it is the core of Forecasting, Risk Management and Decision Making

AnalystMaster

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s