Index > Course > 2021-03-16: Bayesian and Frequentist approaches to Confidence
The test is taking the place of next class. It’s available from 10AM to 2PM, and should take about an hour.
Buy the textbook! Signal and the Noise.
Regression is coming in soon. You need software:
I feel like I’m gonna need mathjax for today. My mathjax is rusty but we’ll get through it.
In the next 10 minutes, 30 busses will arrive at the bus station. One of them will be the RIT bus.
50% of RIT people wear RIT gear/swag, while the background rate is 2%.
A bus arrives. The first person off is wearing an RIT hat. Is this the RIT bus?
The second person off the bus is not wearing RIT swag. Is this the RIT bus?
The third person is wearing an RIT shirt. Is this the RIT bus?
A Bayesian starts with a “prior probability”, and then uses each new piece of data to adjust their expected probability that something is true.
The probability of A given B is A intersection B over B
\[P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}\]MATHJAX ENGAGED!
Let’s define some sets:
\[A = \{\text{RIT Bus}, \text{Regular Bus}\} \\ B = \{\text{RIT Swagged}, \text{Regular swag}\}\]Remember that $P(A=\text{Swagged})$ is the probability of set $A$ evaluating to Swagged.
\[P(A = \text{RIT Bus} | B = \text{RIT Swagged}) = \frac{P(B|A) \cdot P(A = \text{RIT Bus})}{P(B=\text{RIT Swagged})} = \frac{0.5 \cdot \frac{1}{30}}{P(B=\text{RIT Swagged})}\]But, how often do we expect to see someone in RIT gear, from any bus?
\[P(B) = P(A = \text{RIT Bus}) \cdot P(B = \text{RIT Swagged} | A = \text{RIT Bus}) + P(A = \text{Regular Bus}) \cdot P(B = \text{Regular swag} | A = \text{Regular Bus}) = \frac{1}{30} \cdot 0.5 + \frac{29}{30} \cdot 0.02 = 0.036\]Therefore:
\[P(A = \text{RIT Bus} | B = \text{RIT Swagged}) = \frac{0.5 \cdot \frac{1}{30}}{0.036} = 0.4630\]After seing the first person, there is a 46% chance that this is the RIT bus.
To calculate the second person, we now have to use 46% as our prior probability ($P(A=\text{RIT Bus})$).
A frequentist looks at the data, and finds the probability of this outcome from a random sample under a “null hypothesis”.
Null Hypothesis: This is a regular bus. The probability of the first person off having RIT gear is 2%. “You are 98% confident that this is not a normal bus” (that’s technically incorrect). If you use a threshold of 95%, then we can safely reject the null hypothesis.
The second person is not wearing gear: $P=(0.2)(0.98)(2)=0.392$
The game: There are 2 black cards and 1 red card. They are shuffled, and if you pick the red one, then you win.
We’re a frequentist, and we suspect that the game is rigged.
Null Hypothesis: Everything is fine.
The first loss: This is expected. We would see this 66% of the time. Cool.
The second loss: $(2/3)^2 = 0.444$ , not unheard of
We would see 7 losses 5.9% of the time. We would see 8 losses 3.9% of the time.
The probability of this outcome (8 losses, 0 wins), given our null hypothesis, is exceedingly rare (3.9%). Therefore, we can reject the null hypothesis. The game is rigged!
we’ll enter with 40% confidence that the game is rigged.
\[A = \{\text{Rigged}, \text{Chill}\} \\ B = \{\text{Loss}, \text{Win}\} \\ P(A=\text{Rigged}|B=\text{Loss}) = \frac{P(B=\text{Loss}|A=\text{Rigged})\cdot P(A=\text{Rigged})}{P(B=\text{Loss})}\]But what is the probability that we lose?
\[P(B=\text{Loss}) = P(B=\text{Loss}|A=\text{Rigged})\cdot P(A=\text{Rigged}) + P(B=\text{Loss}|A=\text{Chill})\cdot P(A=\text{Chill}) = 1 * 0.4 + 0.33 * 0.6 = 0.598\]If $P(A=\text{Rigged})=0.4$, then $P(B=\text{Loss})=0.598$ for the first iteration.
\[P(A=\text{Rigged}|B=\text{Loss}) = (1*0.4)/0.598 = 0.6689\]Recap:
We now need to repeat to get an answer.
I want to write a program for this. I should also learn R, huh?
Time to try uploading to github and debugging my mathjax lmao
Index > Course > 2021-03-16: Bayesian and Frequentist approaches to Confidence