Introduction to Bayes’ Theorem

Objective: By the end of this lesson, students will be able to understand the concept of Bayes’ Theorem, its historical context, and its applications in probability and statistics.

Duration: About  60 minutes

Materials:

• Whiteboard and markers

• Handouts with exercises and problems related to Bayes’ Theorem

• Projector and screen

Introduction:

• Begin the lesson with a question to engage students: “Have you ever wondered how we make decisions based on uncertain information?” Discuss a few examples, like medical diagnoses, weather forecasts, or financial decisions.

• Introduce the lesson’s topic: Bayes’ Theorem, a fundamental concept in probability theory that helps us update our beliefs based on new evidence.

Historical Context:

• Provide a brief overview of the historical background of Bayes’ Theorem:

  • Bayes’ Theorem is named after Reverend Thomas Bayes, an 18th-century English statistician, philosopher, and theologian.

  • Thomas Bayes never published his work on the theorem during his lifetime. It was discovered in his notes after his death and published posthumously.

  Story of Rev. Thomas Bayes:

  • Share an anecdote about Thomas Bayes to humanize his story and make history relatable:
  • Thomas Bayes was born in 1701 and is believed to have been largely self-educated in mathematics and logic.
  • He became a Presbyterian minister, and his interests in theology and mathematics intersected in intriguing ways.
  • While not widely known during his lifetime, his work laid the foundation for modern probability theory.

  Main Content: Bayes’ Theorem

  • Basic Probability Concepts:
  • Define key terms: prior probability, posterior probability, likelihood, evidence, and conditional probability.
  • Provide examples to illustrate these concepts, such as rolling dice or drawing cards from a deck.
  • Understanding Bayes’ Theorem:
  • Present the formula for Bayes’ Theorem and explain each component:

  \[ P(A|B_i) =\frac{P(A \cap B_i)}{P(A)} =\frac{P(B_i |A)}{P(A)} =\frac{P(B_i |A)}{P(B_1 |A)P(A)+P(B_2 |A)P(A) + \cdots + P(B_k |A)P(A)} \]

\[ Posterior =\frac{Prior \times Likelihood}{Normalizing \ Constant} \]

•  Discuss how Bayes’ Theorem allows us to update our beliefs about an event A given new evidence B.

  Applications of Bayes’ Theorem:

• Provide real-world examples where Bayes’ Theorem is applicable:

• Medical diagnosis (e.g., disease testing)

• Criminal investigations (e.g., DNA evidence)

• Spam email filtering

• Weather forecasting

• Calculation and Practice:

• Guide students through solving example problems using Bayes’ Theorem.

• Distribute handouts with exercises and problems for individual or group work.

Discussion and Wrap-Up:

• Facilitate a class discussion to review the example problems and reinforce key concepts.

• Ask students about their thoughts on how Bayes’ Theorem can be applied in their daily lives or in other fields of study.

Homework:

• Assign additional problems from the textbook or handouts to reinforce the understanding of Bayes’ Theorem.

Assessment:

• Evaluate students based on their participation in discussions, their ability to solve example problems, and their performance on the assigned homework.

Conclusion: 

Let students summarize the importance of Bayes’ Theorem in making informed decisions under uncertainty. Encourage students to explore further applications and study advanced topics in probability and statistics.

---

Example: Breast Cancer Screening

Bayes’ Theorem is widely used in medical diagnosis, particularly in the context of disease testing. It helps healthcare professionals assess the likelihood of a person having a certain disease based on the results of a diagnostic test and other relevant information. Here is a real-world examples:

Suppose that out of all women aged 40 who participate in screening, about 1% of them will have breast cancer at the time of testing. Suppose for those women with breast cancer the  tests indicate a positive result 80% of the time and that there is a risk of false positives, with 10% of women without breast cancer testing positive. Suppose that a 40-year-old woman has been screened, and tested positive for cancer. What is the probability that she has the disease?

Solution:

To answer this question, we must first calculate the conditional probability \(P(cancer | +)\).

\[ P(cancer)=0.01; P(+ | no \ cancer)=0.1 \]

\[ P(cancer | +)=\frac{P(+|cancer)P(cancer)}{P(+)} \]

Now,

\[ P(+)=\frac{P(+ \cap cancer)P(cancer)}{true \ positive} + \frac{P(+ \cap no \ cancer)P(cancer)}{false \ positive} = P(+|cancer)P(cancer) + P(+|no \ cancer)P(no \ cancer)= (0.8)(0.01) + (0.1)(0.99) = 0.11 \]

Exercise for Students (From Bayes’ Rules Book)

1. Lactose intolerance is an inability to digest milk, often resulting in an upset stomach. Fatima wants to learn more about the proportion of adults who are lactose intolerant, . Her prior model for is:

\(\pi\)	0.4	0.5	0.6	0.7	Total
\(f(\pi)\)	0.1	0.2	0.44	0.26	1

1. Fatima surveys a random sample of 80 adults and 47 are lactose intolerant. Without doing  any math, make a guess at the posterior model of , and explain your reasoning.

2. Calculate the posterior model. How does this compare to your guess in part a?

3. If Fatima had instead collected a sample of 800 adults and 470 (keeping the sample proportion the same as above) are lactose intolerant, how does that change the posterior model?

2. Repeat problem #1 utilizing simulation to approximate the posterior model of corresponding to Fatima’s survey data. Specifically, simulate data for 10,000 people and remember to set your random number seed.

3. A medical test is designed to detect a disease that about 3% of the population has. For 93% of those who have the disease, the test yields a positive result. In addition, the test falsely yields a positive result for 7% of those without the disease. What is the probability that a person has the disease given that they have tested positive? Answer this question by simulating data for 10,000 people.