Worked examples ¶

This section contains a number of examples, with their solutions, and commentary about the problem. The problems are listed in alphabetical order below. There are multiple themes worth highlighting to focus your exploration of Bayes theorem for practical real life problems.

1. Art contest
2. An atheist and a priest
3. Bookcase aliens
4. Defective part from two production lines
5. A failing new employee
6. Smoke from a fire or BBQ?
7. Getting married in the rain
8. Getting robbed by a plumber
9. Heuristic for evaluating medical news
10. The jealous coworker
11. Librarian or farmer?
12. The Mony Hall problem
13. The more generous slot machine
14. The mystic seer
15. Product adoption
16. Spam email filter
17. Sparking widgets
18. Strange underwear
19. Who killed Mr. Boddy

Themes ¶

• Conditional probabilities: This set of examples is the most basic and is good for practicing the mechanics of using Bayes theorem. While the examples are simplistic, there are none of the usual coin/card/dice examples you find in most textbooks. You just have to identify the relevant numbers and then plug them into Bayes theorem.
  • Sparking widgets
  • The more generous slot machine
  • Defective part from two production lines
  • Product adoption
  • The Mony Hall problem
  • Art contest
  • Librarian or farmer?
  • Getting robbed by a plumber
  • Smoke from a Fire or BBQ?
  • Getting married in the rain
  • Alien bookcases

  Multiple competing hypotheses: This theme highlights that the methods described in this manual can apply to situations with multiple - more than two - competing hypotheses. If you can only generate a single hypothesis for a real life problem then you might be looking at the problem through an overly simple lens.

  • Art contest
  • Who killed Mr. Boddy
  • The Mony Hall problem
  • The jealous coworker
  • Alien bookcases
  • An atheist and a priest
  • Alien bookcases
  • A failing new employee
  • The mystic seer
  • Spam email filter
  • The Mony Hall problem
  • Heuristic for Evaluating Medical News

  Heuristic for medical news: This theme relates to a technique of thinking that can be applied without any understanding of Bayes theorem. Of course it is powered by Bayes theorem, but the user doesn’t have to know that to benefit from the method. The example is medical related, but the technique is broadly applicable to news media of any kind. Again, first introduced in the Worcestershire cola example , but useful enough that it is explicitly given its own theme.

  Relative likelihoods: Easy introductory problems give you all the information that you need to simply plug numbers into Bayes theorem. Practical problems however will not be so convenient. As described in step 6 of the standard solution process there is a helpful internal prompt that you can use to estimate relative likelihoods. This theme highlights example problems that explicitly use the internal prompt:

  • The Mony Hall problem
  • The more generous slot machine
  • The mystic seer
  • Sparking widgets

  Art contest ¶

  • Pam submitted 15 paintings to the competition, 4% of her works have won first prize in past competitions.
  • Pia put in 5 paintings, 6% of her works have won first prize.
  • Pablo put in 10 paintings, 3% of his works have won first prize.

  This example is a simple illustration of how to accommodate more than two hypothesis - in this case three possible winners. Keep in mind that the result of the competition is still binary. One person will win, the rest will lose.

  The question being asked is: will Pam win the competition? Establish the hypotheses as:

  • \(H_1\) : Pam will win the contest
  • \(H_2\) : Pia will win the contest
  • \(H_3\) : Pablo will win the contest

  For accounting purposes, set the odds as \(H_1:H_2:H_3\) for Pam:Pia:Pablo. Based on the number of submissions each painter made, set the prior odds as 15:5:10. The relative skill of the painters is incorporated when establishing the relative likelihood. Based on the painters historical win rates in other past competitions, the likelihood is set at 4%:6%:3% or 4:6:3. Table Table 5 summarizes the results of the given information.

  Table 5 Data from the art contest problem statement converted into the form for use with Bayes theorem ¶

  Simplified posterior odds

  This assumes to calculate the probabilities that Pam/Pia/Pablo are the only contestants and that one of them has to win (see mutually exclusive and exhaustive propositions for more on this assumption).

  It would have been equally valid to assume just two competing hypotheses for who would win the contest, for example, Pam:other. Because all the needed information was available to us, we could reason about more than two hypotheses. For another more involved example of reasoning about more than two hypotheses at a time, see the Monty Hall problem .

  An atheist and a priest ¶

  An atheist and a priest strike up a conversation. After some debate it is agreed upon that the priest applies a perfect 100% probability to the existence of god, and the atheist applies a perfect 0% probability to the existence of god. How much evidence will it take to convince either one to change their belief, even slightly.

  Bayes’ theorem suggests that no amount of evidence can change their mind if the probabilities are truly 0 and 100 (as opposed to say 0.00001% and 99.99999%).

  — Original source: The Signal and the Noise, The Bayesian Path to Less Wrongness, pg 210

  Table 6 puts the information from the problem statement into the standard solution form. We arbitrarily take the position of the atheist, but the math works the same in reverse for the priest as well. For accounting purposes, denote the odds as God to no God. For argument’s sake, assume that some test demonstrates to the atheist very strong evidence (million to one odds!) that God exists. No matter how strong the evidence might be, because the atheist and priest’s belief is absolute , nothing can can persuade them to change their mind.

  Table 6 The example of the atheist and priest’s absolute belief, and why no evidence can persuade them to believe otherwise. ¶

  If beliefs are very extreme instead of absolute you can work backwards like the alien bookcase example to determine how convincing you would need to be to change someones beliefs. This example shows that sometimes it is truly a waste of time to argue with someone who either starts with an absolute belief, or won’t ever change their mind, no matter how strong the evidence.

  Bookcases aliens ¶

  This is the quantitative argument behind the qualitative statement that “extraordinary claims require extraordinary evidence” (a claim popularized by Carl Sagan , which dates back to at least Pierre-Simon Laplace ).

  Consider the following hypothesis: What if there are Bookcase Aliens who teleport into our houses at night and drop off bookcases?

  Bob offers the following evidence for this claim: “Last week, I visited my friend’s house, and there was a new bookcase there. If there were no bookcase aliens, I wouldn’t have expected that my friend would get a new bookcase. But if there are Bookcase Aliens, then the probability of my finding a new bookcase there was much higher. Therefore, my observation, ‘There is a new bookcase in my friend’s house,’ is strong evidence supporting the existence of Bookcase Aliens.”

  In an intuitive sense, we have a notion that Bob’s evidence “There is a new bookcase in my friend’s house” is not as extraordinary as the claim “There are bookcase aliens” - that the evidence fails to lift the claim. Bayes’s Rule makes this statement precise.

  — Original source: arbital.com

  Bayes theorem certainly applies here. However, given the nature of this problem, any probabilities will be quite subjective. Yet it is still helpful to run the numbers and get a feel for your beliefs.

  The question being asked in the problem is: Who placed the bookcase in Bob’s friends house? Establish the hypotheses as:

  • \(H_1\) : Bookcase aliens placed a bookcase in Bob’s friends house
  • \(H_2\) : There is some other human reason for the appearance of the bookcase

  For accounting purposes, set the odds as \(H_1:H_2\) for ‘bookcase aliens’ to ‘other reason’. In my life experience so far the existence of bookcase aliens seem very unlikely. Any statement where the odds are heavily against bookcase aliens will work as a prior. Take odds of 1:10^9 (one to a billion) against bookcase aliens being the reason why the bookcase appeared at Bob’s friends house.

  Next, the relative likelihood needs to be established. Irregardless of the prior, lets say that bookcase aliens do exist. It is mentally challenging after setting such an extreme prior to objectively put on the “true believer” hat, but you need to work hard to ignore your preconceived notions during this step. What would the probability be if bookcase aliens existed, that they would place a bookcase in Bob’s friends house? Subjectively, I set the probability at 0.5 because I’m trying to be as generous as possible. On the other hand, if there is another reason for the appearance of a bookcase I would prescribe a 0.01 probability of someone getting a new bookcase in any given year. Bookcases are big and heavy, so getting a new one is probably pretty rare. The likelihood is 0.5:0.01 or 50:1. The arguments above are summarized in Table 7 .

  Table 7 A possible solution to the bookcase alien problem using Bayes theorem ¶

  Simplified posterior odds

  Based on these results the evidence is not strong enough to raise the posterior to the ‘pragmatically noticeable’ odds of say 1:100. This begs the question what likelihood ratio would be need to make the posterior 1:100?

  Working backwards, a relative likelihood of \(10^7:1\) would be needed to produce posterior odds that indicate that the existence of bookcase aliens are even pragmatically noticeable. Table 8 summarizes what might need to happen to even consider bookcase aliens as a reason for a bookcase appearing in a house.

  Table 8 Working backwards to identify the likelihood ratio needed for posterior odds of 1:100. ¶

  Simplified posterior odds

  For a likelihood ratio of \(10^7:1\) , you would have to have very, very, compelling evidence that the appearance of a bookshelf was the result of aliens AND there would have to be very, very, low probability of some other possible explanation. Bookcase aliens may be a silly example, but if you can think of a test that clearly supports your hypothesis while eliminating other hypothesis at the same time you have struck gold. In practice, extreme likelihoods usually require lots and lots of independent observations under widely varying circumstances.

  The only other argument that bookcase alien believers can make to refute your results is that you prescribed an unreasonably strong prior. If for the sake of argument your prior was changed to 1:10^6 (one to a million), the relative likelihood would still have to be extremely convincing at odds of 10,000:1.

  In summary, if the prior odds are really extreme, it will take extraordinary evidence to change your mind. Keep in mind that it may not be possible to generate such convincing evidence. See the example about the atheist and priest to further explore even more extreme beliefs.

  Defective part from two production lines ¶

  Two production lines produce the same part. Line 1 produces 1,000 parts per week of which 100 are defective. Line 2 produces 2,000 parts per week of which 150 are defective. If you choose a part randomly from the stock, what is the probability it is defective? If it is defective, what is the probability it was produced by line 1?

  — Original source: Github user gtribello, problem 2

  This is a two part problem. The first question, the probability of a defective part, is not easily handled with the standard solution process shown in this manual. The second question, what is the conditional probability of a part being from line 1 given, that it is defective is a classic Bayes theorem problem.

  The probability of a defective part is simply \(p(\mbox) = \frac = 0.083\) .

  The second question, which asks you to find \(p(\mbox_1|\mbox)\) , can be solved with Bayes theorem. There are two competing hypotheses:

  • \(H_1\) : The part came from line 1
  • \(H_2\) : The part came from line 2

  For accounting purposes, take the odds as \(H_1:H_2\) or \(\mbox_1 : \mbox_2\) . The prior is obtained from the problem statement based on the number of parts produced by each line. Let the prior be 1000:2000 or 1:2. The lines have different historical occurrences of defective parts. If a part came from line 1, its probability of being defective is 100/1000 = 0.1. If a part came off line 2, then its probability of being defective is 150/2000 = 0.075. Line 2 produces defective parts at a lower rate than line 1, but contributes twice as many parts in a week to the total inventory. Take the relative likelihood of a line producing a defective part as 0.1:0.075, or 100:75 = 4:3. Table 9 summarizes the information provided in the problem statement and shows the application of Bayes theorem.

  Table 9 Bayes theorem applied to the two production line problem ¶

  Simplified posterior odds

  If a part is defective, there is a 0.4 probability that it came from line 1.

  For the truly rigorous reader who wants to go beyond practical real world problems, see the theory section on the probability of evidence for a discussion of how \(p(\mbox)\) is mathematically relevant to Bayes theorem. However, skipping this diversion won’t hurt your ability to use Bayes theorem.

  A failing new employee ¶

  A few years back, a senior person at my workplace told me that a new employee wasn’t getting his work done on time, and that she’d had to micromanage him to get any work out of him at all. This was an unpleasant fact for a number of reasons; I’d liked the guy, and I’d advocated for hiring him to our Board of Directors just a few weeks earlier (which is why the senior manager was talking to me). I could have demanded more evidence, I could have demanded that we give him more time to work out, I could have demanded a videotape and signed affidavits… but a new employee not working out, just isn’t that improbable. Could I have named the exact prior odds of an employee not working out, could I have said how much more likely I was to hear that exact report of a long-term-bad employee than a long-term-good employee? No, but ‘somebody hires the wrong person’ happens all the time, and I’d seen it happen before. It wasn’t an extraordinary claim, and I wasn’t licensed to ask for extraordinary evidence. To put numbers on it, I thought the proportion of bad to good employees was on the order of 1:4 at the time, and the likelihood ratio for the manager’s report seemed more like 10:1.”

  — Original source: arbital.com

  In this example the question is: Do I believe the that the employee is underperforming in his role? Cast the hypothesis as a true/false statement:

  • \(H_1\) : True - the employee is under performing
  • \(H_2\) : False - there must me another reason for the bad performance

  For accounting purposes, write the odds as \(H_1:H_2\) for true:false. All of the pertinent numbers were explicitly given to us, we just have to confirm that we agree with them. I have limited experience with hiring and firing coworkers, but immediately after the emotionally draining experience of laying off a direct report following months of bad performance, my manager said this to me: ‘Don’t feel bad, sometimes these things just don’t work out. I estimate that about 20% of my hires over the years did not work out for some reason or another.’ This is exactly the same as odds of bad to good employees being 1:4 in the example, so I think they are reasonable odds. The odds that the manager is telling the truth also seem reasonable to me. I find that it would be unlikely for the manager to lie in general, especially since the author of the example personally recommended the employee. Table 10 summarizes the relevant odds for the sake of being thorough. As described above, this is probably one of those instances when you don’t run the numbers, the intuition of knowing how Bayes theorem works probably gives you enough insight into the situation because ‘extraordinary claims require extraordinary evidence, while ordinary claims just require ordinary evidence’.

  Table 10 Bayes theorem applied to the failing employee problem ¶

  Simplified posterior odds

  There are multiple examples ( bookcase aliens , atheist and the priest ) highlighting examples with extreme claims or evidence. This example contrasts with those examples because extreme odds were intentionally not used. To successfully solve real world problems, it is helpful to be able to identify when extraordinary evidence is required as well as when just ordinary evidence is needed to change your beliefs.

  Smoke from a fire or BBQ? ¶

  From your suburban home you see smoke in the distance, what is the probability that there is a structure fire?

  — Original source: mathsisfun.com

  In the suburbs, there are not many reasons to have a fire. There could be a structural fire, or there could could be smoke coming from some kind of bar-b-que. Due to restrictions from the home owners association and the city, it is unlikely that someone is burning leaves or trash.

  The question being asked is: Do you believe that there is a structure fire? Cast the hypotheses as:

  • \(H_1\) : There is a structure fire
  • \(H_2\) : Some other source, maybe a BBQ, is causing the smoke

  For accounting purposes, write the odds as \(H_1:H_2\) for ‘structure fire’ : BBQ. I have only seen one burned out single family home in a neighborhood near me in my entire life. So set the prior odds at 1:99 against a fire.

  To determine the relative likelihood, I ask myself what would the probability be of seeing smoke if each hypothesis was true:

  • If there was a structure fire, I would almost certainly see smoke. Set the probability at 0.9.
  • If there was a BBQ the probability of seeing smoke would be much less. Charcoal grills only smoke for a few minutes when you first start them up. Gas grills may not smoke at all when started. Set the probability to 0.1.

  The probabilities just happen by coincidence to sum to 1 in this example, in general this is not a requirement. The relative likelihood is therefore the ratio of the probabilities, 0.9:0.1 = 9:1. Table 11 summarizes the reasoning used in this example.

  Table 11 A summary of the reasoning for the structure fire problem and the use of Bayes theorem. ¶

  Simplified posterior odds

  The result shows that even if you see smoke, it is unlikely that there is a structure fire.

  The odds of a structure fire are so low to begin with that you need better evidence than just smoke to believe that there is a structure fire. I worry that my prior was not skeptical enough. The odds in favor of a structural fire might be even smaller. Either way, this example teaches that the base rate of an event occurring is important to consider. If the base rate is very small, the probability of the event occurring given normal evidence is still very low.

  Getting married in the rain ¶

  Marie is getting married tomorrow, at an outdoor ceremony in the desert. In recent years, it has rained only 5 days each year. Unfortunately, the weatherman has predicted rain for tomorrow. When it actually rains, the weatherman correctly forecasts rain 90% of the time. When it doesn’t rain, he incorrectly forecasts that it will rain 10% of the time. What is the probability that it will rain on the day of Marie’s wedding? Assume that there are no leap years.

  — Original source: Github user gtribello, problem 3

  The question being asked in this example is: Do you believe that it will rain tomorrow? Cast the hypotheses as:

  • \(H_1\) : Rain at the wedding ceremony
  • \(H_2\) : Clear or simply overcast at the wedding ceremony

  For accounting purposes, write the odds as \(H_1:H_2\) for rain:clear. The problem statement provides us with historical data to set our prior at 5:360 = 1:72.

  The book The Signal and the Noise dedicates a chapter to how weather forecasts are made. At least one forecaster admits that per company policy when a low probability of rain is predicted, they intentionally forecast the probability of rain to be higher than their models indicate. This is an example of where presentation takes precedent over accuracy, because people interpret a 0% ‘chance’ of rain as an iron clad promise that it will not rain. Adding a little bit of wiggle room to the forecast helps protect the forecasters integrity in the eyes of a public who are not fully versed in probability theory. In the problem statement, when the forecaster incorrectly predicts rain when it is clear instead 10% of the time, this is probably an intentional error.

  To determine the relative likelihood we can take the ratio of the provided probabilities for correctly predicting rain, 0.9:0.1 = 9:1. The probabilities just happen to sum to 1. This is not strictly required for a relative likelihood. Table Table 12 summarizes the information provided in the problem statement.

  Table 12 A summary of Bayes theorem applied to the rain on the wedding day problem. ¶

  Simplified posterior odds

  The good news for the wedding party is that the odds of rain are still very low given the forecast for rain. This is because the base rate of rain in the desert is so low to begin with that it would take stronger evidence to give significant credence to the forecast for rain.

  If, however, we could trust the forecaster to present the true probability, then the probability of a clear day occurring when the forecast is for rain is only 0.01, then the posterior odds are closer to 1:1 for rain on the wedding day.

  Getting robbed by a plumber ¶

  A group of girl friends rented a house on the beach. While they were at the house they had to call a repair man to fix the stove. The repair man made the group nervous because he was seen peering (snooping) into the bedrooms of the house as he was walked from the front door to the kitchen to work on the stove. Someone in the group of friends had recently heard a story in the news of a repair man coming back to later rob a house they had recently worked on - presumably after seeing valuables in vulnerable locations and memorizing the layout of the house. What is the probability they will get robbed by the repairman?

  — Original source: Julia Galef (video at 4:30)

  The question being asked in the problem is: Do you believe that the repairman will later rob the house? Cast the hypotheses as:

  • \(H_1\) : The repairman is a burglar
  • \(H_2\) : The repair man is honest

  For accounting purposes, write the odds as \(H_1:H_2\) for burglar:honest. While I am sure that there are some criminals posing as repairmen, I would find this to be the exception to the rule. In my experience most repairmen are hard working professionals, so I put the odds at 1:100 for burglar:honest.

  This particular repairman however was acting strangely while in the house. To determine the relative likelihood I ask myself what would the probability be of the repairman snooping if each hypothesis was true:

  • If the repairman was a burglar, I would almost certainly expect him to look around the house trying to spot the location of valuables. Set the probability at 0.8 because a professional burglar might know how to look around discretely.
  • If the repairman was honest I would expect there to be a low probability of snooping. The repairman relies on his reputation to get future work, and acting strangely could negatively impact his reputation. I place a probability of an honest repairman snooping at 0.1.

  Therefore, take the relative likelihood as 0.8:0.1 = 8:1. Table 13 summarizes the reasoning described so far in this problem and the application of Bayes theorem.

  Table 13 A summary of using Bayes theorem for the repairman problem. ¶

  Simplified posterior odds

  Based on this reasoning, it is unlikely that the house will later get robbed. The initial idea for the hypothesis was generated from a news story. A single news story is usually a bad way to generate a hypothesis because the newspaper focuses on negative and unlikely events to sell more papers. For more on interpreting the news with Bayes theorem, see the heuristic for evaluating the news . Either way, the base rate of burglar repairmen is low, so it will take much stronger evidence than snooping to give credence to the idea that the house will be burglarized.

  Heuristic for evaluating medical news ¶

  A heuristic technique is any approach to problem solving or self-discovery that employs a practical method that is not guaranteed to be optimal, perfect, or rational, but is nevertheless sufficient for reaching an immediate, short-term goal or approximation…Heuristics can be mental shortcuts that ease the cognitive load of making a decision. — Wikipedia

  I’m a big fan of heuristics for solving practical problems. When you are comfortable with an approximate solution, they can be the proverbial Swiss army knife of problem solving. The following example is really a method unto itself. As I discuss elsewhere , this method very nearly became the basis for this book. Other considerations ultimately made the standard solution process look a bit different, but it is still a very useful way to apply Bayes theorem to practical problems. Strictly speaking, this is a method for generating the relative likelihood, everything else from the standard method still applies.

  This heuristic, which I originally found suggested by Jeff Leek, deals with bits of evidence. Each bit of evidence either doubles the likelihood that your hypothesis is true, or halves the likelihood. The original description was tailored to medical news, but you could easily modify the process for any news source. Remember that a news headline is basically advertising, so you should rely on your critical thinking skills to evaluate its merit independently.

  1. Was the study a clinical study in humans?
  2. Was the outcome of the study something directly related to human health like longer life or less disease?
  3. Was the outcome something you care about, such as living longer or feeling better?
  4. Was the study a randomized, controlled trial (RCT)?
  5. Was it a large study — at least hundreds of patients?
  6. Did the treatment have a major impact on the outcome?
  7. Did predictions hold up in at least two separate groups of people?

  The original source of this example uses the proposed heuristic to evaluate a news headline that said “Hospital checklist cut infections, saved lives.”

  The question being implied here is: Do you believe the hospital checklists cut infections and save lives? Cast the hypotheses as:

  • \(H_1\) : True, hospital checklists meaningfully reduce infections
  • \(H_2\) : False, checklists are not effective

  For accounting purposes, write the odds as \(H_1:H_2\) for true:false. The original author put the odds at 1:4 against checklists being effective. I have created and used safety checklists before, so I would set my prior in favor of checklists being effective. This is the beauty of establishing your prior odds, the original author and I could have a productive discussion over why we chose the prior odds that we did. I see the heuristic as a kind of checklist itself, but because this example is all about establishing the relative likelihood, we will just go with prior odds of 1:4.

  The research article cited in the news story described above was used to answer the heuristic questions:

  1. Was the study a clinical study in humans? yes
  2. Was the outcome of the study something directly related to human health like longer life or less disease? yes
  3. Was the outcome something you care about, such as living longer or feeling better? yes
  4. Was the study a randomized, controlled trial (RCT)? no
  5. Was it a large study — at least hundreds of patients? yes
  6. Did the treatment have a major impact on the outcome? yes
  7. Did predictions hold up in at least two separate groups of people? yes

  The heuristic treats each of the seven questions as equally important. If a question is answered yes , then the likelihood is increased by a factor of 2. If the answer is no , the likelihood is decreased by a factor of 1/2. Based on these results, 6 yes answers and 1 no answer the likelihood would be calculated as:

  \[likelihood = 2 \times 2 \times 2 \times \frac<1> \times 2 \times 2 \times 2 = 32\]

  The relative likelihood is implied from this result. If the heuristic indicates that the headline is 32 times more likely to be true than false then the relative likelihood will be 32:1. Table 14 summarizes the reasoning and the results of the heuristic for the checklist headline.

  Table 14 A summary of the reasoning with Bayes theorem and the heuristic on the checklist headline. ¶

  Simplified posterior odds

  Based on the additional research into the underlying study, the original author revised his beliefs to indicate that the news headline was likely true.

  As a counter example, a second headline was considered in the original example. The headline “How using Facebook could increase your risk of cancer” did not hold up under the scrutiny of the heuristic. Based on the underlying research article, it was found that only 1 of the heuristic questions could be answered yes, and the remaining six were no’s. Therefore, the likelihood ratio was calculated as:

  \[likelihood = 2 \times \frac<1> \times \frac<1> \times \frac<1> \times \frac<1> \times \frac<1> \times \frac<1> = \frac<1>\]

  This implies a relative likelihood of 1:32. In conjunction with a skeptical prior, the likelihood further reinforces that the headline should not be trusted.

  This heuristic, or any other like it, seems to be a very practical method for systematically evaluating lots of similar events. Specifically I could see investments being evaluated with a similar heuristic. Given enough attempts, the bias of the heuristic could be identified and the heuristic revised to possibly reduce the bias in future versions.

  The original headline for the article that described this heuristic was “Finally, a Formula For Decoding Health News” . In this case, I agree whole heartedly that the headline is true.

  The Jealous coworker ¶

  You believe that Bob is jealous/competitive with Alice at work. This matters because you are both of their supervisors. At a lunch with just you and Bob he is complaining about Alice not completing a project on time. This seems to support the hypothesis of jealousy. But if you asked yourself what is the likelihood that a coworker complains about someone not finishing a task on time, it could be argued that it is very fair to complain about the situation.

  — Original source: Julia Galef (video at 6:30)

  The question being asked in the problem is: Do you believe that Bob is jealous of Alice? Cast the hypotheses as:

  • \(H_1\) : Bob is jealous of Alice
  • \(H_2\) : Bob is simply annoyed with Alice

  For accounting purposes, write the odds as \(H_1:H_2\) for jealous:annoyed. The problem statement didn’t say anything about your past experience with Bob/Alice, so just assume that you believe the probability that Bob is jealous of Alice is at 0.75, or equivalently odds of 3:1.

  Next, the relative likelihood of observing Bob complain about Alice needs to be evaluated. The point of this example is to highlight that an observation is not always strong evidence for or against a hypothesis. Sometimes the evidence is only somewhat influential in changing your beliefs. Complaining about a co-worker does suggest jealousy. But complaining about a co-worker who does not finish work on time as promised is also not that unusual. Ultimately, the observation of Bob complaining about Alice in this situation is very weak evidence that Bob is jealous. Maybe cast the odds at just slightly better than even odds - so 9:8 in favor of Bob being jealous. You could probably stop at this point by realizing that the likelihood is very near even odds will not change your beliefs appreciably. For the sake of thoroughness, Table 15 summarizes the discussion so far.

  Table 15 A summary of the reasoning with Bayes theorem for the Bob vs Alice jealousy problem. ¶

  The evidence was not very compelling for either hypothesis, so you revised your beliefs that Bob was jealous from 0.75 to 0.77. If you really need to test if Bob is jealous of Alice, you will need a better test where the relative likelihood ratio is more extreme before you can noticeably modify your beliefs.

  Librarian or farmer? ¶

  • Steve is a librarian
  • Steve is a farmer

  This example comes from a famous cognitive science journal paper by Daniel Kahneman and Amos Traversky. The only reason I mention their names is that Daniel Kahneman was awarded the 2002 Nobel Prize in Economic Sciences for his work related to the paper, so this example is well known.

  The question being asked in the problem is: Do you believe that Steve is a librarian, or a farmer? Cast the hypotheses as:

  • \(H_1\) : Steve is a librarian
  • \(H_2\) : Steve is a farmer

  In the paper, they put the prior of farmers:librarians as 20:1 and the likelihood of Steve being a farmer based on the observation of his personality at 1:4 for a posterior of 20:4 = 5:1 or an 83% chance of being a farmer. Of course, the paper showed that people uninitiated to Bayes theorem fell victim to the base rate fallacy and estimate that Steve is a librarian a majority of the time 2 . Labor statistics show that there are simply many more farmers in the US population that librarians. The standard solution process for this problem is shown in Table 16 .

  Table 16 Solution proposed by Kahneman and Traversky using Bayes theorem to reason about the librarian or farmer problem ¶

  Simplified posterior odds

  The Monty Hall problem ¶

  There are 3 doors, behind which are two goats and a car. You are given a chance to pick one of the three doors (call it door A). You’re hoping to pick the door with the car behind it of course.

  You don’t get to see what is behind your door yet. Monty Hall, the game show host, examines the other doors (B & C) and opens one of the doors with a goat behind it. (If both doors have goats, he picks randomly.)

  Here’s the game: Do you stick with door A (original guess) after Monty opens a door with a goat, or do you switch to the unopened door? Does your strategy matter in this game?

  — Original source: betterexplained.com , plus some additional discussion on assigning likelihoods from towardsdatascience.com

  I will admit that I am stretching the definition of real life problems with this example. This problem is loosely based on the 1963 television show ‘Let’s Make a Deal’ which was co-created and hosted by Monty Hall 3 . Contestants are selected from the audience to participate. To increase their likelihood of being picked from the crowd they wear funny and outrageous costumes. The television announcer would then instruct the selected contestant to “Come on down!” to the stage. The optimal strategy for the game described here is not intuitive in my opinion, and it is a commonly used example of applying Bayes theorem.

  The question being asked in the problem is: Which door do you believe has a car behind it? Cast the hypotheses as:

  • \(H_A\) : Door A
  • \(H_B\) : Door B
  • \(H_C\) : Door C

  For accounting purposes, write the odds as \(H_A:H_B:H_C\) for the odds of finding a car behind doors A:B:C respectively. The car and goats are randomly placed behind the doors, so there is no reason to initially favor one door over another. Set your prior belief at 1:1:1.

  Let’s assume that you initially select door A, but are not shown its contents. Monty is then tasked with opening one of the unselected doors (either door B or C). Monty doesn’t arbitrarily open one of the remaining doors because that could potentially reveal the location of the car. Instead he will always show you a goat. Here is a summary of how he will pick a door to open:

  • If the car is behind door A (your initial selection), then Monty has two choices. He can show you either of the goats which are behind doors B and C. He will choose between B and C randomly.
  • If the car is behind door B then Monty has exactly one door that he can open, door C.
  • If the car is behind door C, then Monty has exactly one door that he can open, door B.

  You initially select door A, in our example Monty opens door B to reveal a goat.

  To determine the relative likelihood, ask yourself what would the probability be of Monty opening door B if each hypothesis was true:

  • If the car is behind door A (the door you chose), and you know that there is a goat behind door B, then you know that Monty will choose randomly which door to show you. If this hypothesis is true you have a 0.5 probability of seeing door B opened.
  • Monty revealed a goat behind door B, so the probability of a car being behind this door is 0.0.
  • If the car is behind door C, then Monty can only open door B. Door A is not allowed because that was your initial choice, and door C is not allowed because it would reveal the car. If this hypothesis is true, then there is a 1.0 probability of seeing door B opened.

  Table 17 summarizes the reasoning used in the problem.

  Table 17 A summary of the reasoning for the Monty Hall problem using Bayes theorem. This assumes that initially door A was chosen and Monty opens door B to show a goat. ¶

  Simplified posterior odds

  Working through the numbers shows that the odds of picking the car are not spread equally across the different doors. Because of the constraints on which door Monty can open, door C is more likely to have the car than door A.

  The strategy certainly does matter in this game. If a contestant chooses to switch doors after viewing a goat then they increase their probability of winning a car from \(\frac\) to \(\frac\) . I myself am not below dressing up in a funny costume to have better than even odds of winning a car!

  Note that our analysis applies in general, there is no stipulation that door A be the left most door. The label A simply represents the door that you initially choose. Doors B and C are assigned to the other doors regardless of how they are arranged physically.

  The more generous slot machine ¶

  • A machine that pays out 10% of the time
  • A machine that pays out 20% of the time

  The question being asked in the problem is: Which slot machine are you playing? Cast the hypotheses as:

  • \(H_1\) : Generous
  • \(H_2\) : Stingy

  For accounting purposes, write the odds as \(H_1:H_2\) for the machine you chose being generous:stingy. There is no reason initially to suspect that one machine is better than the other, so set your prior as 1:1.

  To determine the relative likelihood, ask yourself what would the probability be of the slot machine producing a loss if each hypothesis was true:

  • If the slot machine is generous, then the probability of loosing on any single play is \(1-0.2=0.8\) .
  • If the slot machine is stingy, then the probability of loosing on any single play is \(1-0.1=0.9\)

  Take the relative likelihood as 0.8:0.9 = 8:9. The discussion above is summarized in Table 18 .

  Table 18 A summary of the reasoning using Bayes theorem for the more generous slot machine problem. ¶

  After collecting one data point, you can say that the probability that you choose the generous slot machine is 0.47. This example is a great jumping off point for a number of statistical diversions. In the theory section, there is a discussion for how to use a different forms of Bayes theorem to describe your beliefs about the payout rate of the chosen slot machine as a distribution instead of a point estimate. I would also suggest Conned Again, Watson! Cautionary Tales Of Logic, Math, And Probability , which is a fun discussion of many of the logical fallacies that gamblers and laymen may have.

  A likelihood ratio of 8:9 implies that your observation will do little to modify your prior beliefs. In this example, your observation are an inefficient method of changing your beliefs. It will take the collection of multiple data points before you can say with any certainty which machine you are playing on. Slot machines are an artificially random process that is a poor analogy for real life problems. In real life, you can often devise a better test to increase the likelihood ratio and, therefore, update your beliefs more efficiently.

  The mystic seer ¶

  “The hand belongs to Mr. Don S. Carter, male member of a honeymoon team on route across the Ohio countryside to New York City. In one moment, they will be subjected to a gift most humans never receive in a lifetime. For one penny, they will be able to look into the future. The time is now, the place is a little diner in Ridgeview, Ohio, and what this young couple doesn’t realize is that this town happens to lie on the outskirts of the Twilight Zone.”

  Rod Serling spoke those words during the opening of the Second season’s seventh episode of the legendary TV show, The Twilight Zone. (Ep. 43 - Nick of Time, originally aired November 18, 1960)

  — Source: mysticseer.com

  This example pays homage to my father-in-law who is a big fan of the twilight zone. This example is derived from a similar example found in Bayesian Statistics the Fun Way .

  A young William Shatner plays the highly superstitious Don Carter in this episode. His new bride Pat, played by Patricia Breslin, is more rational. When car trouble forces the honeymooning couple to wait in a rural diner while their car is repaired, they find a fortune telling machine. For a penny, you can ask the mystic seer a yes or no question. Answers are given on a printed card that comes out of the machine after pulling a lever . Don asks a series of 7 questions that are shown to all be accurate predictions. Don becomes convinced that the mystic seer is an accurate fortune teller, Pat remains skeptical.

  Estimate Don and Pat’s prior odds that a fortune telling machine existed, before seeing 7 correct predictions. This is an example of working backwards from perceived posterior odds to someones prior odds as a way to understand what they might be thinking.

  The use of convinced and skeptical to describe someones beliefs are far from quantitative descriptions, so we have to make some brave assumptions to solve this problem.

  The question being asked in the problem is: Do you believe that the mystic seer is a fortune teller? Cast the hypotheses as:

  • \(H_1\) : True
  • \(H_2\) : False

  For accounting purposes, write the odds as \(H_1:H_2\) for the machine being a fortune teller as true:false.

  For the sake of argument, assume that convinced translates into odds of 3:2 in favor of the machine being a fortune teller, and skeptical can be taken to be odds of 1:5. Obviously, if you watched the episode you might generate different quantitative interpretations of convinced and skeptical which is fine. The beauty of Bayes theorem is not what odds we pick, but how we must justify those odds to others.

  The relative likelihood of the mystic seer being a fortune teller requires us to look at each hypothesis and ask what the probability of observing the data would be given that the hypothesis is true:

  • If \(H_1\) is true, and the mystic seer is a clairvoyant fortune teller, then the probability of getting all seven predictions correct would be a perfect 1.0. Any number of correct predictions would have a probability of 1.0 because the hypothesis is that the machine is a fortune teller. A single wrong prediction at any point however would eliminate this hypothesis and make the probability exactly 0.0.
  • If \(H_2\) is true, then the mystic seer simply got lucky and guessed the correct answer 7 times in a row. Because only yes or no questions are allowed, the probability of answering any one question correctly is 0.5 under this hypothesis. The probability of randomly guessing the correct answer 7 times in a row would be \(0.5 \times 0.5 \times 0.5 \times 0.5 \times 0.5 \times 0.5 \times 0.5 = (0.5)^7 = 0.0078125 \approx 0.008\)

  The evidence in favor of the mystic seer being a fortune teller is strong in this case. The relative likelihood of the seer being a fortune teller is \(1:0.0078125 \Rightarrow 1/0.0078125 = 128\) times more likely than the hypothesis that the machine is randomly guessing.

  The discussion above is summarized in Table 19 for Don and Table 20 for Pat.

  Table 19 A summary of the known and assumed quantities for Don in the mystic seer example ¶ Table 20 A summary of the known and assumed quantities for Pat in the mystic seer example ¶

  Transforming the usual definition of Bayes theorem first shown in equation (1) to solve for the prior odds gives the result of:

  \[prior \space odds = \frac \]

  Therefore, Don’s prior odds for believing the seer was a fortune teller were \(3/128:2/1 = 0.0234375:2 \approx 0.01\) which is roughly \(1:85\) against the seer being a fortune teller initially.

  Pat’s prior odds were \(1/128:5/1 = 0.0078125:5 \approx 0.002\) which is approximately \(1:640\) against the seer being a fortune teller initially.

  I will leave the detailed analysis of what this might mean for the dynamic between Don and Pat to a licensed marriage counselor, but you can see that they each approached the belief of the seer being a fortune teller very differently.

  Product adoption ¶

  You are selling a product in an area where 30% of the people live in the city and the rest live in the suburbs. Currently, 20% of the city dwellers use your product and 10% of the suburbanites use your product. You are presented with two new sales strategies the first will increase your market share in the suburbs to 15%. The second will increase your market share in the city to 25%. Which strategy should you adopt? What percentage of the people who own your product are city dwellers before your new sales drive?

  — Original source: Github user gtribello, problem 4

  This problem has multiple parts, but because everything you need to solve the problems was provided in the problem statement you can plug the relevant numbers in to find the answers.

  The problem asks: what percent of current users are city dwellers before the sales drive? Table 21 outlines the relevant quantities and calculates that about 46% of current users live in the city.

  Table 21 A summary of the reasoning for calculating the percent of current customers who are city dwellers. ¶

  Simplified posterior odds

  The second part of this questions asks you which strategy you should choose to increase your profits. Table 22 outlines the expected outcome for option A, increasing your market share in the suburbs to 15%.

  Table 22 A summary of the reasoning for calculating the outcome of option A with Bayes theorem. ¶

  Simplified posterior odds

  Table 23 outlines the expected outcome for option B, increasing your market share in the city to 25%.

  Table 23 A summary of the reasoning for calculating the outcome of option B with Bayes theorem. ¶

  Simplified posterior odds

  Before implementing the sales drive, you could expect for every 100 people of population that 30 people would live in the city and 70 would live in the suburbs. For the baseline case (before the new sales strategy), you would expect to sell \(30 \times 0.2 + 70 \times 0.10 = 13\) units of your product per hundred people. Option A would be expected to sell \(30 \times 0.2 + 70 \times 0.15 = 16.5\) units per hundred people. Option B would be expected to sell \(30 \times 0.25 + 70 \times 0.10 = 14.5\) units per hundred people. Option A is your best strategy because the suburban market is much bigger than the city market.

  Spam email filter ¶

  Back in the day, spam emails were a real problem for email users. Today, I rarely, if ever, check my spam folder because the filtering is so precise. Bayes theorem played a major role in removing the burden of spam from email users, and email was then widely adopted as an official form of communication across most organizations. The following example describes the essence of how an email spam filter operates. More importantly for real world problems, this example shows how Bayes theorem can be used when there are multiple pieces of evidence that can be used to update your beliefs.

  Assume that you collect a number of emails and manually categorize them as spam or normal emails. This is an easy task, humans tend to be very fast and accurate with this sort of classification. The problem is that humans quickly tire of such a monotonous activity, which is why teaching a computer to filter spam messages is so helpful. Within each of these collections, you count how often common words appear in the messages, and compile the results below.

  Spam	Normal
  Hello	3	3
  Darling	1	5
  Buy	3	2
  Viagra	3	0

  What are the odds of a message being spam that contains the phrase “hello darling”?

  — Original source: betterexplained.com

  The question being asked in the problem is: what is your belief that a message with the words “hello darling” is spam? Cast the hypotheses as:

  • \(H_1\) : Spam message
  • \(H_2\) : Normal message

  For accounting purposes, write the odds as \(H_1:H_2\) for the odds of spam to normal messages. The statistics on how many emails are actually spam varies, but every statistic that I have seen indicates that spam emails are much more likely than normal email, possibly as high a 98% of all emails are spam. The original author of this example suggested that the odds of spam:normal emails might be 9:1, so we will take that as our prior.

  We have data to help generate the relative likelihood ratios. A message with the word ‘hello’ has even odds, 3:3 = 1:1, of being a spam message. Messages with the word ‘darling’ have odds of 1:5 of being spam. The word ‘hello’ will not cause us to change our beliefs, but the word ‘darling’ will move our beliefs towards the hypothesis that the message is normal. The not so hard trick is that Bayes theorem can allow us to use both pieces of evidence together. Table 24 summarizes the discussion from above.

  Table 24 The odds, calculated with Bayes theorem, of a message that contains the words ‘hello darling’ is spam. ¶

  Simplified posterior odds

  The total likelihood of the evidence ‘hello darling’ was calculated in two steps as \(3:3 \times 1:5 = 3:15 = 1:5\) in favor of a normal message. The odds form of Bayes theorem makes it very convenient to use multiple pieces of evidence when revising our beliefs, just keep multiplying the odds until you have considered all your evidence. Ultimately, the high base rate of spam messages lead us to conclude that the message is still likely to be spam with a probability of 0.64. In this example, the evidence lead us to believe that the message was less spammy than your average message.

  As a second related example, what are the odds of a message being spam that contains the phrase “buy viagra”? This is in my experience the quintessential spam message. Keeping the same accounting process as above for odds of spam to normal messages, start with a prior of 9:1.

  Based on the observed word frequencies, the odds of a spam message containing the work ‘buy’ is 3:2. The word ‘viagra’ only appears in spam messages, never in a normal message. Stating the odds as 3:0 will break our mathematical framework like in the atheist and priest example. As a work around, conservatively assume that somewhere in the history of the internet that there is a normal message with the word ‘viagra’ in it. Tweak the likelihood of ‘viagra’ to be 3:1 just to avoid breaking our mathematical framework. Intuitively, we know this message will be marked as spam, but run the numbers in Table 25 just to be thorough.

  Table 25 The odds of a message that contains the words ‘buy viagra’ is spam. ¶

  After viewing the evidence ‘buy viagra’ in the message, the odds are highly in favor of spam - even after conservatively tweaking the likelihood for the word ‘viagra’.

  At this point, the question becomes at what odds do we move a message into the spam folder. Accidentally classifying a normal message as spam has fairly serious consequences. Users would much rather see a few spam messages, than have a normal messages hidden from them in their spam folder. If you find even a few normal emails in your spam folder, you quickly lose confidence in the filter and start manually checking your spam folder, thereby, eliminating the benefit of the filter in the first place. Moving normal messages into the spam folder is known as a false positive.

  In the ‘buy viagra’ example, odds of 81:2 implies that for every 83 emails with the words ‘buy viagra’, 2 will be falsely labeled spam when they are actually a normal email. A 0.98 probability of spam is probably too low for a typical email user. It was suggested by the original author of this example that a threshold closer to 0.999 might be more appropriate.

  This example described what is known a Naive Bayesian filter. The filter is naive because it ignores the very important concept of independence between observations. The odds of seeing the phrase ‘buy viagra’ together in a spam message is much higher than seeing the words ‘buy’ and ‘viagra’ separately in the message. Similarly, if your spouse addresses you as ‘darling’, then the phrase ‘hello darling’ is much more likely than ‘hello’ and ‘darling’ appearing separately. In most cases, ignoring independence of observations is a grave mistake that can lead to poor reasoning. For example, if you are trying to estimate the number of people with blue eyes, it would be misleading to check the eye color of a few families. Within a family, the odds of multiple family members having blue eyes are much higher than the odds of a random person having blue eyes. If you don’t have a large enough sample size, you could easily over state the odds of having blue eyes because you got unlucky and randomly chose a family or two with blue eyes. Naive Bayesian filters unabashedly ignore the requirement for independent observations, and they still work just fine! This is the exception to the rule of independent observations. Amazingly, Bayes theorem still works well in this special case.

  Sparking widgets ¶

  On an assembly line 10% of widgets are defective and 90% are good. 4% of good widgets emit sparks when operating, and 12% of the defective widgets emit sparks. What percentage of sparking widgets are bad?

  — Original source: arbital.com

  The question being asked in the problem is: if you observe a sparking widget, do you believe that widget is bad? Cast the hypotheses as:

  • \(H_1\) : Defective widget
  • \(H_2\) : Good widget

  For accounting purposes, write the odds as \(H_1:H_2\) for defective:good. The problem statement provides you with the information needed for your prior. 10% of widgets from the assembly line are defective, and 90% are good. Take the prior as 0.1:0.9 = 1:9.

  To determine the relative likelihood, I ask myself what would the probability be of the widget sparking if each hypothesis was true:

  • If the widget is defective, we would expect 12% of these widgets to spark
  • If the widget is good, we would expect 4% of these widgets to spark

  Therefore, set the relative likelihood at 0.12:0.4 = 12:4 = 3:1. Table 26 summarizes the reasoning described above.