Courses

# Module 10 Reasoning with Uncertainty - Probabilistic reasoning Notes | EduRev

## : Module 10 Reasoning with Uncertainty - Probabilistic reasoning Notes | EduRev

``` Page 1

Module
10

Reasoning with
Uncertainty -
Probabilistic reasoning

Version 2 CSE IIT, Kharagpur

Page 2

Module
10

Reasoning with
Uncertainty -
Probabilistic reasoning

Version 2 CSE IIT, Kharagpur

Lesson
28

Bayes Networks
Version 2 CSE IIT, Kharagpur

Page 3

Module
10

Reasoning with
Uncertainty -
Probabilistic reasoning

Version 2 CSE IIT, Kharagpur

Lesson
28

Bayes Networks
Version 2 CSE IIT, Kharagpur

10.5 Bayesian Networks

10.5.1 Representation and Syntax
Bayes nets (BN) (also referred to as Probabilistic Graphical Models and Bayesian Belief
Networks) are directed acyclic graphs (DAGs) where each node represents a random
variable. The intuitive meaning of an arrow from a parent to a child is that the parent
directly influences the child. These influences are quantified by conditional probabilities.

BNs are graphical representations of joint distributions. The BN for the medical expert
system mentioned previously represents a joint distribution over 8 binary random
variables {A,T,E,L,S,B,D,X}.

Conditional Probability Tables

Each node in a Bayesian net has an associated conditional probability table or CPT.
(Assume all random variables have only a finite number of possible values). This gives
the probability values for the random variable at the node conditional on values for its
parents. Here is a part of one of the CPTs from the medical expert system network.

If a node has no parents, then the CPT reduces to a table giving the marginal distribution
on that random variable.
Version 2 CSE IIT, Kharagpur

Page 4

Module
10

Reasoning with
Uncertainty -
Probabilistic reasoning

Version 2 CSE IIT, Kharagpur

Lesson
28

Bayes Networks
Version 2 CSE IIT, Kharagpur

10.5 Bayesian Networks

10.5.1 Representation and Syntax
Bayes nets (BN) (also referred to as Probabilistic Graphical Models and Bayesian Belief
Networks) are directed acyclic graphs (DAGs) where each node represents a random
variable. The intuitive meaning of an arrow from a parent to a child is that the parent
directly influences the child. These influences are quantified by conditional probabilities.

BNs are graphical representations of joint distributions. The BN for the medical expert
system mentioned previously represents a joint distribution over 8 binary random
variables {A,T,E,L,S,B,D,X}.

Conditional Probability Tables

Each node in a Bayesian net has an associated conditional probability table or CPT.
(Assume all random variables have only a finite number of possible values). This gives
the probability values for the random variable at the node conditional on values for its
parents. Here is a part of one of the CPTs from the medical expert system network.

If a node has no parents, then the CPT reduces to a table giving the marginal distribution
on that random variable.
Version 2 CSE IIT, Kharagpur

Consider another example, in which all nodes are binary, i.e., have two possible values,
which we will denote by T (true) and F (false).

We see that the event "grass is wet" (W=true) has two possible causes: either the water
sprinker is on (S=true) or it is raining (R=true). The strength of this relationship is shown
in the table. For example, we see that Pr(W=true | S=true, R=false) = 0.9 (second row),
and hence, Pr(W=false | S=true, R=false) = 1 - 0.9 = 0.1, since each row must sum to one.
Since the C node has no parents, its CPT specifies the prior probability that it is cloudy
(in this case, 0.5). (Think of C as representing the season: if it is a cloudy season, it is less
likely that the sprinkler is on and more likely that the rain is on.)

10.5.2 Semantics of Bayesian Networks
The simplest conditional independence relationship encoded in a Bayesian network can
be stated as follows: a node is independent of its ancestors given its parents, where the
ancestor/parent relationship is with respect to some fixed topological ordering of the
nodes.

In the sprinkler example above, by the chain rule of probability, the joint probability of
all the nodes in the graph above is
Version 2 CSE IIT, Kharagpur

Page 5

Module
10

Reasoning with
Uncertainty -
Probabilistic reasoning

Version 2 CSE IIT, Kharagpur

Lesson
28

Bayes Networks
Version 2 CSE IIT, Kharagpur

10.5 Bayesian Networks

10.5.1 Representation and Syntax
Bayes nets (BN) (also referred to as Probabilistic Graphical Models and Bayesian Belief
Networks) are directed acyclic graphs (DAGs) where each node represents a random
variable. The intuitive meaning of an arrow from a parent to a child is that the parent
directly influences the child. These influences are quantified by conditional probabilities.

BNs are graphical representations of joint distributions. The BN for the medical expert
system mentioned previously represents a joint distribution over 8 binary random
variables {A,T,E,L,S,B,D,X}.

Conditional Probability Tables

Each node in a Bayesian net has an associated conditional probability table or CPT.
(Assume all random variables have only a finite number of possible values). This gives
the probability values for the random variable at the node conditional on values for its
parents. Here is a part of one of the CPTs from the medical expert system network.

If a node has no parents, then the CPT reduces to a table giving the marginal distribution
on that random variable.
Version 2 CSE IIT, Kharagpur

Consider another example, in which all nodes are binary, i.e., have two possible values,
which we will denote by T (true) and F (false).

We see that the event "grass is wet" (W=true) has two possible causes: either the water
sprinker is on (S=true) or it is raining (R=true). The strength of this relationship is shown
in the table. For example, we see that Pr(W=true | S=true, R=false) = 0.9 (second row),
and hence, Pr(W=false | S=true, R=false) = 1 - 0.9 = 0.1, since each row must sum to one.
Since the C node has no parents, its CPT specifies the prior probability that it is cloudy
(in this case, 0.5). (Think of C as representing the season: if it is a cloudy season, it is less
likely that the sprinkler is on and more likely that the rain is on.)

10.5.2 Semantics of Bayesian Networks
The simplest conditional independence relationship encoded in a Bayesian network can
be stated as follows: a node is independent of its ancestors given its parents, where the
ancestor/parent relationship is with respect to some fixed topological ordering of the
nodes.

In the sprinkler example above, by the chain rule of probability, the joint probability of
all the nodes in the graph above is
Version 2 CSE IIT, Kharagpur

P(C, S, R, W) = P(C) * P(S|C) * P(R|C,S) * P(W|C,S,R)

By using conditional independence relationships, we can rewrite this as
ause R is independent of S given its
arent C, and the last term because W is independent of C given its parents S and R. We
from a parent to a child is that the parent directly
fluences the child. The direction of this influence is often taken to represent casual

n into a product of conditional probabilities
sing repeated applications of the product rule.
ditionally independent of all

P(C, S, R, W) = P(C) * P(S|C) * P(R|C) * P(W|S,R)

where we were allowed to simplify the third term bec
p
can see that the conditional independence relationships allow us to represent the joint
more compactly. Here the savings are minimal, but in general, if we had n binary nodes,
the full joint would require O(2^n) space to represent, but the factored form would
require O(n 2^k) space to represent, where k is the maximum fan-in of a node. And fewer
parameters makes learning easier.

The intuitive meaning of an arrow
in
influence. The conditional probabilities give the strength of causal influence. A 0 or 1 in
a CPT represents a deterministic influence.
10.5.2.1 Decomposing Joint Distributions

A joint distribution can always be broken dow
u

We can order the variables however we like:

10.5.2.2 Conditional Independence in Bayes Net
A Bayes net represents the assumption that each node is con
its non-descendants given its parents.

So for example,

Version 2 CSE IIT, Kharagpur

```
Offer running on EduRev: Apply code STAYHOME200 to get INR 200 off on our premium plan EduRev Infinity!