Concepts in Probability | Additional Topics for IIT JAM Mathematics PDF Download

Clear all your Mathematics doubts with EduRev Join for Free

Join for Free

Further Concepts in Probability
The study of probability mostly deals with combining different events and studying these events alongside each other. How these different events relate to each other determines the methods and rules to follow when we're studying their probabilities.
Events can be pided into two major categories dependent or Independent events.

Independent Events
When two events are said to be independent of each other, what this means is that the probability that one event occurs in no way affects the probability of the other event occurring. An example of two independent events is as follows; say you rolled a die and flipped a coin. The probability of getting any number face on the die in no way influences the probability of getting a head or a tail on the coin.

Dependent Events
When two events are said to be dependent, the probability of one event occurring influences the likelihood of the other event.

For example, if you were to draw a two cards from a deck of 52 cards. If on your first draw you had an ace and you put that aside, the probability of drawing an ace on the second draw is greatly changed because you drew an ace the first time. Let's calculate these different probabilities to see what's going on.
There are 4 Aces in a deck of 52 cards

On your first draw, the probability of getting an ace is given by:

If we don't return this card into the deck, the probability of drawing an ace on the second pick is given by


As you can clearly see, the above two probabilities are different, so we say that the two events are dependent. The likelihood of the second event depends on what happens in the first event.

Conditional Probability
We have already defined dependent and independent events and seen how probability of one event relates to the probability of the other event.
Having those concepts in mind, we can now look at conditional probability.
Conditional probability deals with further defining dependence of events by looking at probability of an event given that some other event first occurs.

Conditional probability is denoted by the following:
P(B|A)
The above is read as the probability that B occurs given that A has already occurred.
The above is mathematically defined as:

Outcomes and Events
We consider experiments, which comprise: a collection of distinguishable outcomes, which are termed elementary events, and typically denoted by Ω and a collection of sets of possible outcomes to which we might wish to assign probabilities, A the event.
In order to obtain a sensible theory of probability, we require that our collection of events A is an algebra over, i.e. it must possess the following properties
(i.) Ω ∈ A
(ii.) If A in A, then

(iii.) If A₁ and A₂ ∈ A, then A₁ ∪ A₂ ∈ A.

In the case of finite Ω, we might note that the collection of all subsets of Ω necessarily satisfies the above properties and by using this default choice of algebra, we can assign probabilities to any possible combination of elementary events.

Proposition 1.1: If A is an algebra, then ∅ ∈ A.
Proposition 1.2: If A₁ and A₂ ∈ A, then A₁ ∩ A₂ ∈ A for any algebra A.
Proposition 1.3: If A is an algebra and A₁, A₂,..., A_n ∈ A, then (∩_i=1 A_i) ∈ A.

Probability Functions/Measures
Let Ω denote the sample space and A denote a collection of events assumed to be a σ-algebra.
Definition 1.1: (Probability Function): A probability function P[·] is a set function with domain A (a σ-algebra of events) and range [0,1], i.e., P: A → [0,1], which satisfies the following axioms
(i.) P[A] ≥ 0 for every A ∈ A
(ii.) P[Ω] = 1
(iii.) If A₁, A₂,... is a sequence of mutually exclusive events (i.e. A_i ∩ A_j = ∅ for any i ≠ j), in A and if 

Properties of P[.]
A remarkably rich theory emerges from these three axioms (together, of course, with those of set theory). Indeed, all formal probability follows as a logical consequence of these axioms. Some of the most important simple results are summarised here. Throughout this section, assume that Ω is our collection of possible outcomes A is a σ-algebra over Ω and P[.] is an associated probability distribution.
Many of these results simply demonstrate that things which we would intuitively want to be true of probabilities do, indeed, arise as logical consequences of this simple axiomatic framework.
Proposition 1.4: P(∅) = 0 .
Proposition 1.5: If A₁, A₂,..., A_n are pairwise disjoint elements of A, corresponding to mutually exclusive outcomes in our experiment, then

Proposition 1.6: If A ∈ A then

Proposition 1.7: For any two events A, B ∈ A
P(A ∪ B) = P(A) + P(B) - P(A ∩ B)

Proposition 1.8: If A, B ∈ A and A ⊂ B , then

P(A) ≤ P(B)

Proposition 1.9 (Boole’s Inequality): If A₁,..._n ∈ A, then

P(A₁ ∪ A₂ ∪...∪ A_n) ≤ P(A₁) + P(A₂) +...+ P(A_n).

Definition 1.2: (Probability Space): A probability space is the triple (Ω, A, P[·]), where Ω is a sample space, A is a σ - algebra over Ω, and P[·] is a probability function with domain A.

Conditional Probability and Independence
Sometimes it's possible to observe that one event has occurred. In this situation, we wish to have a model for the behaviour of the probability that other events compatible with B. Conditional probability is the appropriate language.

Definition 1.3: (Conditional Probability): Let A and B be events in A of the given probability space (Ω, A, P[·]). The conditional probability of event A given event B, denoted by P[A | B], is defined as
and is left undefined when P [B]=0.

Exercise 1.3.1: Consider the experiment of tossing two coins, Ω = {(H,H), (H,T), (T,H), (T,T)}, and assume that each point is equally likely. Find
(i) The probability of two heads given a head on the first coin.
(ii) The probability of two heads given at least one head.

i.e. B₁,...,B_n partition Ω and P[B_j] > 0, j = 1,...,n, then for every A ∈ A,
Conditional probability has a number of useful properties. The following elementary result is surprisingly important and has some far-reaching consequences.
Theorem 1.2 (Bayes' Formula): For a given probability space (Ω, A, P[·]), if A, B ∈ A are such that P[A] > 0, P[B] > 0, then:
Theorem 1.3 (Partition Formula): If B₁,...,B_n ∈ A partition Ω, then for any A ∈ A:
Theorem 1.4 (Multiplication Rule): For a given probability space (Ω, A, P[·]), let A₁,...,A_n be events belonging to A for which P[A₁,...,A_n-1] > 0, then
P[A₁, A₂,...,A_n] = P[A₂ | A₁]...P[A_n | A₁,...,A_n-1].

Theorem 1.1: (Law of Total Probability): For a given probability space (Ω, A, P[·]), if B₁, ..., B_n is a collection of mutually disjoint events in A satisfying
i.e. B₁,...,B_n partition Ω and P[B_j] > 0, j = 1,...,n, then for every A ∈ A,
Conditional probability has a number of useful properties. The following elementary result is surprisingly important and has some far-reaching consequences.

Theorem 1.2 (Bayes' Formula): For a given probability space (Ω, A, P[·]), if A, B ∈ A are such that P[A] > 0, P[B] > 0, then:
Theorem 1.3 (Partition Formula): If B₁,...,B_n ∈ A partition Ω, then for any A ∈ A:
Theorem 1.4 (Multiplication Rule): For a given probability space (Ω, A, P[·]), let A₁,...,A_n be events belonging to A for which P[A₁,...,A_n-1] > 0, then
P[A₁, A₂,...,A_n] = P[A₂ | A₁]...P[A_n | A₁,...,A_n-1].

Definition 1.4 (Independent Events): For a given probability space (Ω, A, P[.]), let A and B be two events in A. Events A and B are defined to be independent iff one of the following conditions is satisfied:
(i) P[A ∩ B] = P[A]P[B]
(ii) P[A | B] = P[A] if P[B] > 0
(iii) P[B | A] = P[B] if P[A] > 0.
Exercise 1.3.2: Consider the experiment of rolling two dice. Let A = {total is odd}, B = {6 on the first die}, C = {total is seven}.
(i) Are A and B independent?
(ii) Are A and C independent?
(iii) Are B and C independent?

Definition 1.5 (Independence of Several Events): For a given probability space (Ω, A, P[.]), let A₁, ..., A_n be events in A. Events A₁, ..., A_n are defined to be independent iff

Set Theory in Probability
A sample space is defined as a universal set of all possible outcomes from a given experiment.
Given two events A and B and given that these events are part of a sample space S. This sample space is represented as a set as in the diagram below.

The entire sample space of S is given by:
S = {A, B, C}
Remember the following from set theory:


The different regions of the set S can be explained as using the rules of probability.

Rules of Probability
When dealing with more than one event, there are certain rules that we must follow when studying probability of these events. These rules depend greatly on whether the events we are looking at are Independent or dependent on each other.
First acknowledge that

Multiplication Rule (A∩B)
This region is referred to as 'A intersection B' and in probability; this region refers to the event that both A and B happen. When we use the word and we are referring to multiplication, thus A and B can be thought of as AxB or (using dot notation which is more popular in probability) A•B

If A and B are dependent events, the probability of this event happening can be calculated as shown below:

If A and B are independent events, the probability of this event happening can be calculated as shown below:

Conditional probability for two independent events can be redefined using the relationship above to become:

The above is consistent with the definition of independent events, the occurrence of event A in no way influences the occurrence of event B, and so the probability that event B occurs given that event A has occurred is the same as the probability of event B.

Additive Rule (A∪B)
In probability we refer to the addition operator (+) as or. Thus when we want to we want to define some event such that the event can be A or B, to find the probability of that event:


Thus it follows that:

But remember from set theory that and from the way we defined our sample space above:

and that:

So we can now redefine out event as

The above is sometimes referred to as the subtraction rule.

Mutual Exclusivity
Certain special pairs of events have a unique relationship referred to as mutual exclusivity.
Two events are said to be mutually exclusive if they can't occur at the same time. For a given sample space, its either one or the other but not both. As a consequence, mutually exclusive events have their probability defined as follows:

An example of mutually exclusive events are the outcomes of a fair coin flip. When you flip a fair coin, you either get a head or a tail but not both, we can prove that these events are mutually exclusive by adding their probabilities:

For any given pair of events, if the sum of their probabilities is equal to one, then those two events are mutually exclusive.

Rules of Probability for Mutually Exclusive Events

• Multiplication Rule
  From the definition of mutually exclusive events, we should quickly conclude the following:
  
• Addition Rule
  As we defined above, the addition rule applies to mutually exclusive events as follows:
  P(A + B) = 1
• Subtraction Rule
  From the addition rule above, we can conclude that the subtraction rule for mutually exclusive events takes the form;
  

Conditional Probability for Mutually Exclusive Events
We have defined conditional probability with the following equation:

We can redefine the above using the multiplication rule

hence

Below is a venn diagram of a set containing two mutually exclusive events A and B.