Bayes' Theorem | Additional Topics for IIT JAM Mathematics PDF Download

13.5  Bayes' Theorem
Consider that there are two bags I and II. Bag I contains 2 white and 3 red balls and Bag II contains 4 white and 5 red balls. One ball is drawn at random from one of the bags. We can find the probability of selecting any of the bags  or probability of drawing a ball of a particular colour (say white) from a particular bag (say Bag I). In other words, we can find the probability that the ball drawn is of a particular colour, if we are given the bag from which the ball is drawn. But, can we find the probability that the ball drawn is from a particular bag (say Bag II), if the colour of the ball drawn is given? Here, we have to find the reverse probability of Bag II to be selected when an event occurred after it is known. Famous mathematician, John Bayes' solved the problem of finding reverse probability by using conditional probability. The formula developed by him is known as ‘Bayes theorem’ which was published posthumously in 1763.

Before stating and proving the Bayes' theorem, let us first take up a definition and some preliminary results.
13.5.1 Partition of a sample space
A set of events E₁, E₂, ..., E_n is said to represent a partition of the sample space S if

In other words, the events E₁, E₂, ..., En represent a partition of the sample space S if they are pairwise disjoint, exhaustive and have nonzero probabilities.

As an example, we see that any nonempty event E and its complement E′ form a partition of the sample space S since they satisfy E ∩ E′ = φ and E ∪ E′ = S.

From the Venn diagram in Fig 13.3, one can easily observe that if E and F are any two events associated with a sample space S, then the set  {E ∩ F′, E ∩ F, E′ ∩ F, E′ ∩ F′} is a partition of the sample space S. It may be mentioned that the partition of a sample space is not unique. There can be several partitions of the same sample space.

We shall now prove a theorem known as Theorem of total probability.

13.5.2  Theorem of total probability
Let {E₁, E₂,...,E_n}  be a partition of the sample space S,  and suppose that each of the events E₁, E₂,..., En has nonzero probability of occurrence. Let A be any event associated with S, then
P(A) = P(E₁) P(A|E₁) + P(E₂) P(A|E₂) + ... + P(E_n) P(A|E_n)

Proof  Given that E₁, E₂,..., En is a partition of the sample space S (Fig 13.4). Therefore,
S = E₁ ∪ E₂ ∪ ... ∪ E_n ... (1)
and E_i ∩ E_j = φ, i ≠ j, i, j = 1, 2, ..., n

Now, we know that for any event A,
A = A ∩ S
= A ∩ (E₁ ∪ E₂ ∪ ... ∪ E_n)
= (A ∩ E₁) ∪ (A ∩ E₂) ∪ ...∪ (A ∩ E_n)
Also A ∩ E_i and A ∩ E_j are respectively the subsets of E_i and E_j. We know that E_i and E_j are disjoint, for i ≠ j , therefore, A ∩ E_i and A ∩ E_j are also disjoint for all i ≠ j,  i, j = 1, 2, ..., n.
Thus, P(A) = P [(A ∩ E₁) ∪ (A ∩ E₂)∪ .....∪ (A ∩ E_n)]
= P (A ∩ E₁) + P (A ∩ E₂) + ... + P (A ∩ E_n)
Now, by multiplication rule of probability, we have P(A ∩ E_i) = P(E_i) P(A|E_i) as P(E_i) ≠ 0∀i = 1,2,..., n
Therefore, P (A) = P (E₁) P (A|E₁) + P (E₂) P (A|E₂) + ... + P (E_n)P(A|E_n)
or  

Example 15. A person has undertaken a construction job. The probabilities are 0.65 that there will be strike, 0.80 that the construction job will be completed on time if there is no strike, and 0.32 that the construction job will be completed on time if there is a strike. Determine the probability that the construction job will be completed on time.
Solution. Let A be the event that the construction job will be completed on time, and B be the event that there will be a strike. We have to find P(A).
We have
P(B) = 0.65, P(no strike) = P(B′) = 1 − P(B) = 1 − 0.65 = 0.35
P(A|B) = 0.32, P(A|B′) = 0.80
Since events B and B′ form a partition of the sample space S, therefore, by theorem on total probability, we have P(A) = P(B) P(A|B) + P(B′) P(A|B′)
= 0.65 × 0.32 + 0.35 × 0.8
= 0.208 + 0.28 = 0.488
Thus, the probability that the construction job will be completed in time is 0.488.
We shall now state and prove the Bayes' theorem.

Bayes’ Theorem If E₁, E₂ ,..., En are n non empty events which constitute a partition of sample space S, i.e. E₁, E₂ ,..., En are pairwise disjoint and E₁∪ E₂∪ ... ∪ E_n = S and A is any event of nonzero probability, then

Proof By formula of conditional probability, we know that

 (by multiplication rule of probability)
  (by the result of theorem of total probability)

Remark.  The following terminology is generally used when Bayes' theorem is applied.
The events E₁, E₂, ..., En are  called hypotheses.
The probability P(E_i) is called the priori probability of the hypothesis E_i The conditional probability P(E_i |A) is called a posteriori probability of the hypothesis E_i.
Bayes' theorem is also called the formula for the probability of "causes". Since the E_i's are a partition of the sample space S, one and only one of the events E_i occurs (i.e. one of the events E_i must occur and only one can occur). Hence, the above formula gives us the probability of a particular E_i (i.e. a "Cause"), given that the event A has occurred.
The Bayes'  theorem has its applications in variety of situations, few of which are illustrated in following examples.

Example 16. Bag I contains 3 red and 4 black balls while another Bag II contains 5 red and 6 black balls. One ball is drawn at random from one of the bags and it is found to be red. Find the probability that it was drawn from Bag II.
Solution. Let E₁ be the event of choosing the bag I, E₂ the event of choosing the bag II and A be the event of drawing a red ball.
Then P(E₁) = P(E₂) = 1/2
Also P(A|E₁) = P(drawing a red ball from Bag I) = 3/7
and P(A|E₂) = P(drawing a red ball from Bag II) = 5/11
Now, the probability of drawing a ball from Bag II, being given that it is red, is P(E₂|A) By using Bayes' theorem, we have

Example 17. Given three identical boxes I, II and III, each containing two coins. In box I, both coins are gold coins, in box II, both are silver coins and in the box III, there is one gold and one silver coin. A person chooses a box at random and takes out a coin. If the coin is of gold, what is the probability that the other coin in the box is also of gold?
Solution. Let E₁, E₂ and E₃ be the events that boxes I, II and III are chosen, respectively.
Then P(E₁) = P(E₂) = P(E₃) = 1/3
Also, let A be the event that ‘the coin drawn is of gold’
Then P(A|E₁) = P(a gold coin from bag I) = 2/2 = 1
P(A|E₂) = P(a gold coin from bag II) = 0
P(A|E₃) = P(a gold coin from bag III) = 1/2
Now, the probability that the other coin in the box is of gold
= the probability that gold coin is drawn from the box I.
= P(E₁|A)
By Bayes' theorem, we know that

Example 18. Suppose that the reliability of a HIV test is specified as follows:
Of people having HIV, 90% of the test detect the disease but 10% go undetected. Of people free of HIV, 99% of the test are judged HIV–ive but 1% are diagnosed as showing HIV+ive. From a large population of which only 0.1% have HIV, one person is selected at random, given the HIV test, and the pathologist reports him/her as HIV+ive. What is the probability that the person actually has HIV?
Solution. Let E denote the event that the person selected is actually having HIV and A the event that the person's HIV test is diagnosed as +ive. We need to find P(E|A).
Also E′ denotes the event that the person selected is actually not having HIV.
Clearly, {E, E′} is a partition of the sample space of all people in the population.
We are given that

P(E′) = 1 – P(E) = 0.999
P(A|E) = P(Person tested as HIV+ive given that he/she is actually having HIV)

and P(A|E′) = P(Person tested as HIV +ive given that he/she is actually not having HIV)
= 1% = 1/100 = 0.01
Now, by Bayes' theorem


Thus, the probability that a person selected at random is actually having HIV given that he/she is tested HIV+ive is 0.083.

Example 19. In a factory which manufactures bolts, machines A, B and C manufacture respectively 25%, 35% and 40% of the bolts. Of their outputs, 5, 4 and 2 percent are respectively defective bolts. A bolt is drawn at random from the product and is found to be defective. What is the probability that it is manufactured by the machine B?
Solution. Let events B₁, B₂, B₃ be the following :
B₁ : the bolt is manufactured by machine A
B2 : the bolt is manufactured by machine B
B₃ : the bolt is manufactured by machine C
Clearly, B₁, B₂, B₃ are mutually exclusive and exhaustive events and hence, they represent a partition of the sample space.
Let the event E be ‘the bolt is defective’.
The event E occurs with B₁ or with B₂ or with B₃. Given that,
P(B₁) = 25% = 0.25, P (B₂) = 0.35 and P(B₃) = 0.40
Again P(E|B₁) = Probability that the bolt drawn is defective given that it is manufactured by machine A = 5% = 0.05
Similarly, P(E|B₂) = 0.04,  P(E|B₃) = 0.02.
Hence, by Bayes' Theorem, we have

Example 20. A doctor is to visit a patient. From the past experience, it is known that the probabilities that he will come by train, bus, scooter or by other means of transport are respectively  The probabilities that he will be late are  if he comes by train, bus and scooter respectively, but if he comes by other means of transport, then he will not be late. When he arrives, he is late. What is the probability that he comes by train?
Solution. Let E be the event that the doctor visits the patient late and let T₁, T₂, T₃, T₄ be the events  that the doctor comes by train, bus, scooter, and other means of transport respectively.

Then  (given)
P(E|T₁) = Probability that the doctor arriving late comes by train = 1/4
Similarly, P(E|T₂) = 1/3, P(E|T₃) = 1/12 and P(E|T4) = 0, since he is not late if he comes by other means of transport.

Therefore, by Bayes' Theorem, we have
P(T₁|E) = Probability that the doctor arriving late comes by train

Hence, the required probability is 1/2

Example 21. A man is known to speak truth 3 out of 4 times. He throws a die and reports that it is a six. Find the probability that it is actually a six.
Solution. Let E be the event that the man reports that six occurs in the throwing of the die and let S₁ be the event that six occurs and S₂ be the event that six does not occur.
Then P(S₁) = Probability that six occurs = 1/6
P(S₂) = Probability that six does not occur = 5/6
P(E|S₁) = Probability that the man reports that six occurs when six has actually occurred on the die = Probability that the man speaks the truth = 3/4
P(E|S₂) = Probability that the man reports that six occurs  when six has not actually occurred on the die
= Probability that the man does not speak the truth
Thus, by Bayes' theorem, we get
P(S_1|E) = Probability that the report of the man that six has occurred is actually a six

Hence, the required probability is 3/8.