Module 10 Reasoning with Uncertainty - Probabilistic reasoning Notes | EduRev

: Module 10 Reasoning with Uncertainty - Probabilistic reasoning Notes | EduRev

 Page 1


 
 
 
 
 
Module 
10 
 
Reasoning with 
Uncertainty - 
Probabilistic reasoning 
 
 
 
Version 2 CSE IIT, Kharagpur 
 
Page 2


 
 
 
 
 
Module 
10 
 
Reasoning with 
Uncertainty - 
Probabilistic reasoning 
 
 
 
Version 2 CSE IIT, Kharagpur 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Lesson 
 28 
 
Bayes Networks  
Version 2 CSE IIT, Kharagpur 
 
Page 3


 
 
 
 
 
Module 
10 
 
Reasoning with 
Uncertainty - 
Probabilistic reasoning 
 
 
 
Version 2 CSE IIT, Kharagpur 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Lesson 
 28 
 
Bayes Networks  
Version 2 CSE IIT, Kharagpur 
 
 
10.5 Bayesian Networks 
 
10.5.1 Representation and Syntax 
Bayes nets (BN) (also referred to as Probabilistic Graphical Models and Bayesian Belief 
Networks) are directed acyclic graphs (DAGs) where each node represents a random 
variable. The intuitive meaning of an arrow from a parent to a child is that the parent 
directly influences the child. These influences are quantified by conditional probabilities. 
 
BNs are graphical representations of joint distributions. The BN for the medical expert 
system mentioned previously represents a joint distribution over 8 binary random 
variables {A,T,E,L,S,B,D,X}. 
 
 
 
Conditional Probability Tables 
 
Each node in a Bayesian net has an associated conditional probability table or CPT. 
(Assume all random variables have only a finite number of possible values). This gives 
the probability values for the random variable at the node conditional on values for its 
parents. Here is a part of one of the CPTs from the medical expert system network. 
 
 
If a node has no parents, then the CPT reduces to a table giving the marginal distribution 
on that random variable.  
Version 2 CSE IIT, Kharagpur 
 
Page 4


 
 
 
 
 
Module 
10 
 
Reasoning with 
Uncertainty - 
Probabilistic reasoning 
 
 
 
Version 2 CSE IIT, Kharagpur 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Lesson 
 28 
 
Bayes Networks  
Version 2 CSE IIT, Kharagpur 
 
 
10.5 Bayesian Networks 
 
10.5.1 Representation and Syntax 
Bayes nets (BN) (also referred to as Probabilistic Graphical Models and Bayesian Belief 
Networks) are directed acyclic graphs (DAGs) where each node represents a random 
variable. The intuitive meaning of an arrow from a parent to a child is that the parent 
directly influences the child. These influences are quantified by conditional probabilities. 
 
BNs are graphical representations of joint distributions. The BN for the medical expert 
system mentioned previously represents a joint distribution over 8 binary random 
variables {A,T,E,L,S,B,D,X}. 
 
 
 
Conditional Probability Tables 
 
Each node in a Bayesian net has an associated conditional probability table or CPT. 
(Assume all random variables have only a finite number of possible values). This gives 
the probability values for the random variable at the node conditional on values for its 
parents. Here is a part of one of the CPTs from the medical expert system network. 
 
 
If a node has no parents, then the CPT reduces to a table giving the marginal distribution 
on that random variable.  
Version 2 CSE IIT, Kharagpur 
 
 
Consider another example, in which all nodes are binary, i.e., have two possible values, 
which we will denote by T (true) and F (false). 
 
We see that the event "grass is wet" (W=true) has two possible causes: either the water 
sprinker is on (S=true) or it is raining (R=true). The strength of this relationship is shown 
in the table. For example, we see that Pr(W=true | S=true, R=false) = 0.9 (second row), 
and hence, Pr(W=false | S=true, R=false) = 1 - 0.9 = 0.1, since each row must sum to one. 
Since the C node has no parents, its CPT specifies the prior probability that it is cloudy 
(in this case, 0.5). (Think of C as representing the season: if it is a cloudy season, it is less 
likely that the sprinkler is on and more likely that the rain is on.) 
 
10.5.2 Semantics of Bayesian Networks 
The simplest conditional independence relationship encoded in a Bayesian network can 
be stated as follows: a node is independent of its ancestors given its parents, where the 
ancestor/parent relationship is with respect to some fixed topological ordering of the 
nodes.  
 
In the sprinkler example above, by the chain rule of probability, the joint probability of 
all the nodes in the graph above is  
Version 2 CSE IIT, Kharagpur 
 
Page 5


 
 
 
 
 
Module 
10 
 
Reasoning with 
Uncertainty - 
Probabilistic reasoning 
 
 
 
Version 2 CSE IIT, Kharagpur 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Lesson 
 28 
 
Bayes Networks  
Version 2 CSE IIT, Kharagpur 
 
 
10.5 Bayesian Networks 
 
10.5.1 Representation and Syntax 
Bayes nets (BN) (also referred to as Probabilistic Graphical Models and Bayesian Belief 
Networks) are directed acyclic graphs (DAGs) where each node represents a random 
variable. The intuitive meaning of an arrow from a parent to a child is that the parent 
directly influences the child. These influences are quantified by conditional probabilities. 
 
BNs are graphical representations of joint distributions. The BN for the medical expert 
system mentioned previously represents a joint distribution over 8 binary random 
variables {A,T,E,L,S,B,D,X}. 
 
 
 
Conditional Probability Tables 
 
Each node in a Bayesian net has an associated conditional probability table or CPT. 
(Assume all random variables have only a finite number of possible values). This gives 
the probability values for the random variable at the node conditional on values for its 
parents. Here is a part of one of the CPTs from the medical expert system network. 
 
 
If a node has no parents, then the CPT reduces to a table giving the marginal distribution 
on that random variable.  
Version 2 CSE IIT, Kharagpur 
 
 
Consider another example, in which all nodes are binary, i.e., have two possible values, 
which we will denote by T (true) and F (false). 
 
We see that the event "grass is wet" (W=true) has two possible causes: either the water 
sprinker is on (S=true) or it is raining (R=true). The strength of this relationship is shown 
in the table. For example, we see that Pr(W=true | S=true, R=false) = 0.9 (second row), 
and hence, Pr(W=false | S=true, R=false) = 1 - 0.9 = 0.1, since each row must sum to one. 
Since the C node has no parents, its CPT specifies the prior probability that it is cloudy 
(in this case, 0.5). (Think of C as representing the season: if it is a cloudy season, it is less 
likely that the sprinkler is on and more likely that the rain is on.) 
 
10.5.2 Semantics of Bayesian Networks 
The simplest conditional independence relationship encoded in a Bayesian network can 
be stated as follows: a node is independent of its ancestors given its parents, where the 
ancestor/parent relationship is with respect to some fixed topological ordering of the 
nodes.  
 
In the sprinkler example above, by the chain rule of probability, the joint probability of 
all the nodes in the graph above is  
Version 2 CSE IIT, Kharagpur 
 
P(C, S, R, W) = P(C) * P(S|C) * P(R|C,S) * P(W|C,S,R)  
 
By using conditional independence relationships, we can rewrite this as  
ause R is independent of S given its 
arent C, and the last term because W is independent of C given its parents S and R. We 
 from a parent to a child is that the parent directly 
fluences the child. The direction of this influence is often taken to represent casual 
 
n into a product of conditional probabilities 
sing repeated applications of the product rule. 
ditionally independent of all 
 
P(C, S, R, W) = P(C) * P(S|C) * P(R|C) * P(W|S,R)  
 
where we were allowed to simplify the third term bec
p
can see that the conditional independence relationships allow us to represent the joint 
more compactly. Here the savings are minimal, but in general, if we had n binary nodes, 
the full joint would require O(2^n) space to represent, but the factored form would 
require O(n 2^k) space to represent, where k is the maximum fan-in of a node. And fewer 
parameters makes learning easier. 
 
The intuitive meaning of an arrow
in
influence. The conditional probabilities give the strength of causal influence. A 0 or 1 in 
a CPT represents a deterministic influence. 
10.5.2.1 Decomposing Joint Distributions 
 
A joint distribution can always be broken dow
u
 
We can order the variables however we like: 
 
 
10.5.2.2 Conditional Independence in Bayes Net 
A Bayes net represents the assumption that each node is con
its non-descendants given its parents. 
 
So for example, 
 
Version 2 CSE IIT, Kharagpur 
 
Read More
Offer running on EduRev: Apply code STAYHOME200 to get INR 200 off on our premium plan EduRev Infinity!