Module 10 Reasoning with Uncertainty - Probabilistic reasoning Notes | EduRev

: Module 10 Reasoning with Uncertainty - Probabilistic reasoning Notes | EduRev

 Page 1


 
 
 
 
 
Module 
10 
 
Reasoning with 
Uncertainty - 
Probabilistic reasoning 
 
 
 
Version 2 CSE IIT,Kharagpur 
 
Page 2


 
 
 
 
 
Module 
10 
 
Reasoning with 
Uncertainty - 
Probabilistic reasoning 
 
 
 
Version 2 CSE IIT,Kharagpur 
 
 
10.1 Instructional Objective 
• The students should understand the role of uncertainty in knowledge representation 
• Students should learn the use of probability theory to represent uncertainty 
• Students should understand the basic of probability theory, including 
o Probability distributions 
o Joint probability 
o Marginal probability  
o Conditional probability 
o Independence 
o Conditional independence 
• Should learn inference mechanisms in probability theory including 
o Bayes rule 
o Product rule 
• Should be able to convert natural language statements into probabilistic statements 
and apply inference rules 
• Students should understand Bayesian networks as a data structure to represent 
conditional independence 
• Should understand the syntax and semantics of Bayes net 
• Should understand inferencing mechanisms in Bayes net 
• Should understand efficient inferencing techniques like variable ordering 
• Should understand the concept of d-separation 
• Should understand inference mechanism for the special case of polytrees 
• Students should have idea about approximate inference techniques in Bayesian 
networks 
 
At the end of this lesson the student should be able to do the following: 
• Represent a problem in terms of probabilistic statemenst 
• Apply Bayes rule and product rule for inferencing 
• Represent a problem using Bayes net 
• Perform probabilistic inferencing using Bayes net. 
Version 2 CSE IIT,Kharagpur 
 
Page 3


 
 
 
 
 
Module 
10 
 
Reasoning with 
Uncertainty - 
Probabilistic reasoning 
 
 
 
Version 2 CSE IIT,Kharagpur 
 
 
10.1 Instructional Objective 
• The students should understand the role of uncertainty in knowledge representation 
• Students should learn the use of probability theory to represent uncertainty 
• Students should understand the basic of probability theory, including 
o Probability distributions 
o Joint probability 
o Marginal probability  
o Conditional probability 
o Independence 
o Conditional independence 
• Should learn inference mechanisms in probability theory including 
o Bayes rule 
o Product rule 
• Should be able to convert natural language statements into probabilistic statements 
and apply inference rules 
• Students should understand Bayesian networks as a data structure to represent 
conditional independence 
• Should understand the syntax and semantics of Bayes net 
• Should understand inferencing mechanisms in Bayes net 
• Should understand efficient inferencing techniques like variable ordering 
• Should understand the concept of d-separation 
• Should understand inference mechanism for the special case of polytrees 
• Students should have idea about approximate inference techniques in Bayesian 
networks 
 
At the end of this lesson the student should be able to do the following: 
• Represent a problem in terms of probabilistic statemenst 
• Apply Bayes rule and product rule for inferencing 
• Represent a problem using Bayes net 
• Perform probabilistic inferencing using Bayes net. 
Version 2 CSE IIT,Kharagpur 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Lesson 
 26 
 
Reasoning with 
Uncertain information  
Version 2 CSE IIT,Kharagpur 
 
Page 4


 
 
 
 
 
Module 
10 
 
Reasoning with 
Uncertainty - 
Probabilistic reasoning 
 
 
 
Version 2 CSE IIT,Kharagpur 
 
 
10.1 Instructional Objective 
• The students should understand the role of uncertainty in knowledge representation 
• Students should learn the use of probability theory to represent uncertainty 
• Students should understand the basic of probability theory, including 
o Probability distributions 
o Joint probability 
o Marginal probability  
o Conditional probability 
o Independence 
o Conditional independence 
• Should learn inference mechanisms in probability theory including 
o Bayes rule 
o Product rule 
• Should be able to convert natural language statements into probabilistic statements 
and apply inference rules 
• Students should understand Bayesian networks as a data structure to represent 
conditional independence 
• Should understand the syntax and semantics of Bayes net 
• Should understand inferencing mechanisms in Bayes net 
• Should understand efficient inferencing techniques like variable ordering 
• Should understand the concept of d-separation 
• Should understand inference mechanism for the special case of polytrees 
• Students should have idea about approximate inference techniques in Bayesian 
networks 
 
At the end of this lesson the student should be able to do the following: 
• Represent a problem in terms of probabilistic statemenst 
• Apply Bayes rule and product rule for inferencing 
• Represent a problem using Bayes net 
• Perform probabilistic inferencing using Bayes net. 
Version 2 CSE IIT,Kharagpur 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Lesson 
 26 
 
Reasoning with 
Uncertain information  
Version 2 CSE IIT,Kharagpur 
 
10. 2 Probabilistic Reasoning 
Using logic to represent and reason we can represent knowledge about the world with 
facts and rules, like the following ones: 
 
bird(tweety). 
fly(X) :- bird(X). 
 
We can also use a theorem-prover to reason about the world and deduct new facts about 
the world, for e.g.,  
?- fly(tweety). 
Yes 
 
However, this often does not work outside of toy domains - non-tautologous certain 
rules are hard to find. 
 
A way to handle knowledge representation in real problems is to extend logic by using 
certainty factors. 
 
In other words, replace 
IF condition THEN fact 
with 
IF condition with certainty x THEN fact with certainty f(x) 
 
Unfortunately cannot really adapt logical inference to probabilistic inference, since the 
latter is not context-free. 
 
Replacing rules with conditional probabilities makes inferencing simpler. 
 
Replace 
smoking -> lung cancer 
or 
lotsofconditions, smoking -> lung cancer 
with 
P(lung cancer | smoking) = 0.6 
 
Uncertainty is represented explicitly and quantitatively within probability theory, a 
formalism that has been developed over centuries. 
 
A probabilistic model describes the world in terms of a set S of possible states - the 
sample space. We don’t know the true state of the world, so we (somehow) come up with 
a probability distribution over S which gives the probability of any state being the true 
one. The world usually described by a set of variables or attributes. 
 
Consider the probabilistic model of a fictitious medical expert system. The ‘world’ is 
described by 8 binary valued variables: 
Version 2 CSE IIT,Kharagpur 
 
Page 5


 
 
 
 
 
Module 
10 
 
Reasoning with 
Uncertainty - 
Probabilistic reasoning 
 
 
 
Version 2 CSE IIT,Kharagpur 
 
 
10.1 Instructional Objective 
• The students should understand the role of uncertainty in knowledge representation 
• Students should learn the use of probability theory to represent uncertainty 
• Students should understand the basic of probability theory, including 
o Probability distributions 
o Joint probability 
o Marginal probability  
o Conditional probability 
o Independence 
o Conditional independence 
• Should learn inference mechanisms in probability theory including 
o Bayes rule 
o Product rule 
• Should be able to convert natural language statements into probabilistic statements 
and apply inference rules 
• Students should understand Bayesian networks as a data structure to represent 
conditional independence 
• Should understand the syntax and semantics of Bayes net 
• Should understand inferencing mechanisms in Bayes net 
• Should understand efficient inferencing techniques like variable ordering 
• Should understand the concept of d-separation 
• Should understand inference mechanism for the special case of polytrees 
• Students should have idea about approximate inference techniques in Bayesian 
networks 
 
At the end of this lesson the student should be able to do the following: 
• Represent a problem in terms of probabilistic statemenst 
• Apply Bayes rule and product rule for inferencing 
• Represent a problem using Bayes net 
• Perform probabilistic inferencing using Bayes net. 
Version 2 CSE IIT,Kharagpur 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Lesson 
 26 
 
Reasoning with 
Uncertain information  
Version 2 CSE IIT,Kharagpur 
 
10. 2 Probabilistic Reasoning 
Using logic to represent and reason we can represent knowledge about the world with 
facts and rules, like the following ones: 
 
bird(tweety). 
fly(X) :- bird(X). 
 
We can also use a theorem-prover to reason about the world and deduct new facts about 
the world, for e.g.,  
?- fly(tweety). 
Yes 
 
However, this often does not work outside of toy domains - non-tautologous certain 
rules are hard to find. 
 
A way to handle knowledge representation in real problems is to extend logic by using 
certainty factors. 
 
In other words, replace 
IF condition THEN fact 
with 
IF condition with certainty x THEN fact with certainty f(x) 
 
Unfortunately cannot really adapt logical inference to probabilistic inference, since the 
latter is not context-free. 
 
Replacing rules with conditional probabilities makes inferencing simpler. 
 
Replace 
smoking -> lung cancer 
or 
lotsofconditions, smoking -> lung cancer 
with 
P(lung cancer | smoking) = 0.6 
 
Uncertainty is represented explicitly and quantitatively within probability theory, a 
formalism that has been developed over centuries. 
 
A probabilistic model describes the world in terms of a set S of possible states - the 
sample space. We don’t know the true state of the world, so we (somehow) come up with 
a probability distribution over S which gives the probability of any state being the true 
one. The world usually described by a set of variables or attributes. 
 
Consider the probabilistic model of a fictitious medical expert system. The ‘world’ is 
described by 8 binary valued variables: 
Version 2 CSE IIT,Kharagpur 
 
 
Visit to Asia? A  
Tuberculosis? T 
Either tub. or lung cancer? E  
Lung cancer? L 
Smoking? S  
Bronchitis? B 
Dyspnoea? D  
Positive X-ray? X 
 
We have 2
8
 = 256 possible states or configurations and so 256 probabilities to find. 
 
10.3 Review of Probability Theory 
The primitives in probabilistic reasoning are random variables.  Just like primitives in 
Propositional Logic are propositions. A random variable is not in fact a variable, but a 
function from a sample space S to another space, often the real numbers. 
 
For example, let the random variable Sum (representing outcome of two die throws) be 
defined thus: 
Sum(die1, die2) = die1 +die2 
 
Each random variable has an associated probability distribution determined by the 
underlying distribution on the sample space 
 
Continuing our example : P(Sum = 2) = 1/36, 
P(Sum = 3) = 2/36, . . . , P(Sum = 12) = 1/36 
 
Consdier the probabilistic model of the fictitious medical expert system mentioned 
before. The sample space is described by 8 binary valued variables. 
 
Visit to Asia? A  
Tuberculosis? T 
Either tub. or lung cancer? E  
Lung cancer? L 
Smoking? S  
Bronchitis? B 
Dyspnoea? D  
Positive X-ray? X 
 
There are 2
8
 = 256 events in the sample space. Each event is determined by a joint 
instantiation of all of the variables. 
 
S = {(A = f, T = f,E = f,L = f, S = f,B = f,D = f,X = f), 
(A = f, T = f,E = f,L = f, S = f,B = f,D = f,X = t), . . . 
(A = t, T = t,E = t,L = t, S = t,B = t,D = t,X = t)} 
Version 2 CSE IIT,Kharagpur 
 
Read More
Offer running on EduRev: Apply code STAYHOME200 to get INR 200 off on our premium plan EduRev Infinity!