# Statistics for Decision Makers - 05.05 - Probability - Base Rate Fallacy

Title

05.05 - Probability - Base Rate Falacy
Author
Bernard Szlachta (NobleProg Ltd) bs@nobleprog.co.uk
Footer
www.NobleProg.co.uk
Subfooter

Training Courses Worldwide

## Drunk Driving。

• Breathalysers display a false result in 5% of the cases tested
• They never fail to detect a truly drunk person
• 1/1000 of drivers are driving drunk
• Policemen then stop a driver at random, and test them
• The breathalyser indicates that the driver is drunk
How high is the probability the driver is really drunk?
Result\Reality Drunk Sober
Test Positive 1 0.05
Test Negative 0 0.95
Historic Data 0.001 0.999

### Drunk Driving。

Let us assume we tested 1,000,000 people.

True Condition
Drunk Not Drunk
1,000 999,000
Positive Negative Positive Negative
1,000 0 49,950 949,050
• How many of those people tested positive?
• How many of those who tested positive were really drunk?
• What is a "favourable" outcome?
• How many "favourable" outcomes are there?
• How many possible outcomes are there?
• What is the probability of a person who is tested being really drunk?

### Drunk Driving。

Let us assume we tested 1,000,000 people.

True Condition
Drunk Not Drunk
1,000 999,000
Positive Negative Positive Negative
1,000 0 49,950 949,050
How many of those people tested positive?
1,000 + 49,950 = 50,950

How many of those who tested positive were really drunk?
1000

What is the probability of a person who is tested being really drunk?
1000/50,950=0.01962


## Base Rate Fallacy。

• The Base Rate in our case is 0.001 and 0.999 probabilities.
• An overwhelming proportion of people are sober, therefore the probability of a false positive (5%) is much more prominent than the 100% probability of a true positive.
• People tend to simply ignore the base rates, hence it is called (base rate neglect).
• In other words, no matter what the base rates, people tend to look at only the "test accuracy rate".

## Base Rate Fallacy Examples。

• Detecting terrorists
• Detecting a rare disease
• Detecting prospective customers (provided that most people will not buy our product)
• Some DNA tests

## Bayes theorem。

What is the probability that a driver is drunk given that the breathalyser indicates that he/she is drunk?
${\displaystyle P(Drunk|Positive)}$

Bayes' Theorem tells us that:

${\displaystyle P(Drunk|Positive)={\frac {P(Positive|Drunk)\,P(Drunk)}{P(Positive|Drunk)\,P(Drunk)+P(Positive|Sober)\,P(Sober)}}}$

We were told the following in the first paragraph:

${\displaystyle P(Drunk)=0.001}$
${\displaystyle P(Sober)=0.999}$
${\displaystyle P(Positive|Drunk)=1.00}$
${\displaystyle P(Positive|Sober)=0.05}$

After using Bayes' Theorem:

${\displaystyle P(Drunk|Positive)=0.019627\cdot }$

## Is a promotion really working?。

An online advertising company knows (based on its historical record) that 10% of the people who try the trial version of their services will convert into paying customers.

• You propose to introduce a promotion: each new customer will be granted a free $100 for advertising • You know that some people will just register to use the$100 even if they do not intend to convert into paying customers
• You want to test the effectiveness of the free $100 promotion • After running the promotion, 40% of customers who converted used the$100 promotion
• Also, 10% of prospects who did not convert used the promotion

### Is the promotion really working?。

Does the promotion increase the probability of conversion?
Events
Prom: Customer uses the promotion
NotProm: Customer does not use the promotion
Con: Customer converts
NotCon: Customer does not convert

P(Con) = 0.1
An online advertising company knows, based on its historical record, that 10% of the people who try the trial version of their services will convert into customers
P(Prom|Con) = 0.4
After running the promotion, 40% of customers who converted used the \$100 promotion
P(Prom|NotCon) = 0.1
Also, 10% of prospects who used the promotion did not convert
Compute complementary probabilities
P(NotCon) = 0.9
P(NotProm|Con) = 0.6
P(NotProm|NotCon) = 0.9


Does the promotion increase probability of conversion?
P(Con|Prom) > P(Con) = 0.1
P(Con|Prom) from BaysTheorm = 0.308


# Quiz

1

The prospective customer conversion rate is 1%

• You want to prioritize prospects who are more likely to convert.
• You create a test, which has an 80% probability of correctly detecting that a prospect will convert into a customer.
• But for 9.6% of prospects, the test misdetects customers, i.e. they will not convert.
A prospect is positively tested, what is the probability they will convert into a customer?

 below 33% between 33% and 66% above 66%

below 33%

7.8%

2

The prospective customer conversion rate is 50%

• You want to prioritize prospects who are more likely to convert.
• You create a test, which has an 80% probability of correctly detecting that the prospect will convert into a customer.
• But for 9.6% of prospects, the test misdetects customers, i.e. they will not convert.
A prospect is positively tested, what is the probability they will convert into a customer?

 below 33% between 33% and 66% above 66%

between 33% and 66%

3

The prospective customer conversion rate is 99%.

• You want to prioritize prospects who are more likely to convert.
• You create a test, which has an 80% probability of correctly detecting that the prospect will convert into a customer.
• But for 9.6% of prospects, the test misdetects customers, i.e. they will not convert.
A prospect is positively tested, what is the probability they will convert into a customer?

 below 33% between 33% and 66% above 66%

above 66%

4

The prospective customer conversion rate is 99%

• You want prioritize prospects who are more likely to convert
• You create a test, which has an 80% probability of correctly detecting that the prospect will convert into a customer
• But for 9.6% of prospects, the test misdetects customers, i.e. they will not convert
According to the text, which of the below is a base rate (select two)?

 1% 99% 9.6% 80% 20%