Statistics for Decision Makers - 05.05 - Probability - Base Rate Fallacy

From Training Material
Jump to: navigation, search
Title

05.05 - Probability - Base Rate Falacy
Author
Bernard Szlachta (NobleProg Ltd) bs@nobleprog.co.uk
Footer
www.NobleProg.co.uk
Subfooter

Training Courses Worldwide

Drunk Driving。[edit]

ClipCapIt-140603-014312.PNG
  • Breathalysers display a false result in 5% of the cases tested
  • They never fail to detect a truly drunk person
  • 1/1000 of drivers are driving drunk
  • Policemen then stop a driver at random, and test them
  • The breathalyser indicates that the driver is drunk
How high is the probability the driver is really drunk?
Result\Reality Drunk Sober
Test Positive 1 0.05
Test Negative 0 0.95
Historic Data 0.001 0.999

Drunk Driving。[edit]

Let us assume we tested 1,000,000 people.

True Condition
Drunk Not Drunk
1,000 999,000
Positive Negative Positive Negative
1,000 0 49,950 949,050
  • How many of those people tested positive?
  • How many of those who tested positive were really drunk?
  • What is a "favourable" outcome?
  • How many "favourable" outcomes are there?
  • How many possible outcomes are there?
  • What is the probability of a person who is tested being really drunk?

Drunk Driving。[edit]

Let us assume we tested 1,000,000 people.

True Condition
Drunk Not Drunk
1,000 999,000
Positive Negative Positive Negative
1,000 0 49,950 949,050
How many of those people tested positive?
1,000 + 49,950 = 50,950
How many of those who tested positive were really drunk?
1000
What is the probability of a person who is tested being really drunk?
1000/50,950=0.01962

Drunk Driving。[edit]

Detect-drunk-driving.jpg

Base Rate Fallacy。[edit]

  • The Base Rate in our case is 0.001 and 0.999 probabilities.
  • An overwhelming proportion of people are sober, therefore the probability of a false positive (5%) is much more prominent than the 100% probability of a true positive.
  • People tend to simply ignore the base rates, hence it is called (base rate neglect).
  • In other words, no matter what the base rates, people tend to look at only the "test accuracy rate".

Base Rate Fallacy Examples。[edit]

  • Detecting terrorists
  • Detecting a rare disease
  • Detecting prospective customers (provided that most people will not buy our product)
  • Some DNA tests

Bayes theorem。[edit]

What is the probability that a driver is drunk given that the breathalyser indicates that he/she is drunk?
P(Drunk|Positive)


Bayes' Theorem tells us that:

P(Drunk|Positive) = \frac{P(Positive|Drunk)\, P(Drunk)}{P(Positive|Drunk)\,P(Drunk)+P(Positive|Sober)\,P(Sober)}


We were told the following in the first paragraph:

P(Drunk) = 0.001
P(Sober) = 0.999
P(Positive|Drunk) = 1.00
P(Positive|Sober) = 0.05


After using Bayes' Theorem:

P(Drunk|Positive) = 0.019627\cdot

Is a promotion really working?。[edit]

An online advertising company knows (based on its historical record) that 10% of the people who try the trial version of their services will convert into paying customers.

  • You propose to introduce a promotion: each new customer will be granted a free $100 for advertising
  • You know that some people will just register to use the $100 even if they do not intend to convert into paying customers
  • You want to test the effectiveness of the free $100 promotion
  • After running the promotion, 40% of customers who converted used the $100 promotion
  • Also, 10% of prospects who did not convert used the promotion

Is the promotion really working?。[edit]

Does the promotion increase the probability of conversion?
Events
Prom: Customer uses the promotion
NotProm: Customer does not use the promotion
Con: Customer converts
NotCon: Customer does not convert 
P(Con) = 0.1
An online advertising company knows, based on its historical record, that 10% of the people who try the trial version of their services will convert into customers
P(Prom|Con) = 0.4
After running the promotion, 40% of customers who converted used the $100 promotion
P(Prom|NotCon) = 0.1
Also, 10% of prospects who used the promotion did not convert
Compute complementary probabilities
P(NotCon) = 0.9
P(NotProm|Con) = 0.6
P(NotProm|NotCon) = 0.9


Does the promotion increase probability of conversion?
P(Con|Prom) > P(Con) = 0.1
P(Con|Prom) from BaysTheorm = 0.308

Quiz。[edit]

Please find the Quiz here

Quiz[edit]

1. The prospective customer conversion rate is 1%

  • You want to prioritize prospects who are more likely to convert.
  • You create a test, which has an 80% probability of correctly detecting that a prospect will convert into a customer.
  • But for 9.6% of prospects, the test misdetects customers, i.e. they will not convert.
A prospect is positively tested, what is the probability they will convert into a customer?
below 33%
between 33% and 66%
above 66%

Answer >>

below 33%

7.8%


2. The prospective customer conversion rate is 50%

  • You want to prioritize prospects who are more likely to convert.
  • You create a test, which has an 80% probability of correctly detecting that the prospect will convert into a customer.
  • But for 9.6% of prospects, the test misdetects customers, i.e. they will not convert.
A prospect is positively tested, what is the probability they will convert into a customer?
below 33%
between 33% and 66%
above 66%

Answer >>

between 33% and 66%


3. The prospective customer conversion rate is 99%.

  • You want to prioritize prospects who are more likely to convert.
  • You create a test, which has an 80% probability of correctly detecting that the prospect will convert into a customer.
  • But for 9.6% of prospects, the test misdetects customers, i.e. they will not convert.
A prospect is positively tested, what is the probability they will convert into a customer?
below 33%
between 33% and 66%
above 66%

Answer >>

above 66%


4. The prospective customer conversion rate is 99%

  • You want prioritize prospects who are more likely to convert
  • You create a test, which has an 80% probability of correctly detecting that the prospect will convert into a customer
  • But for 9.6% of prospects, the test misdetects customers, i.e. they will not convert
According to the text, which of the below is a base rate (select two)?
1%
99%
9.6%
80%
20%

Answer >>

1%, 99%


Your score is 0 / 0