Evaluation | Artificial Intelligence for Class 10 PDF Download

 Page 1


 
 
Evaluation 
Introduction 
Till now we have learnt about the 4 stages of AI project cycle, viz. Problem scoping, Data acquisition, 
Data exploration and modelling. While in modelling we can make different types of models, how do 
we check if one’s better than the other? That’s where Evaluation comes into play. In the Evaluation 
stage, we will explore different methods of evaluating an AI model. Model Evaluation is an integral 
part of the model development process. It helps to find the best model that represents our data and 
how well the chosen model will work in the future 
What is evaluation? 
Evaluation is the process of understanding the reliability of any AI model, based on outputs by feeding 
test dataset into the model and comparing with actual answers.  There can be different Evaluation 
techniques, depending of the type and purpose of the model. Remember that It’s not recommended 
to use the data we used to build the model to evaluate it. This is because our model will simply 
remember the whole training set, and will therefore always predict the correct label for any point in 
the training set. This is known as overfitting.  
Firstly, let us go through various terms which are very important to the evaluation process.   
Model Evaluation Terminologies 
There are various new terminologies which come into the picture when we work on evaluating our 
model. Let’s explore them with an example of the Forest fire scenario. 
The Scenario 
Imagine that you have come up with an AI based prediction model which has been deployed in a forest 
which is prone to forest fires. Now, the objective of the model is to predict whether a forest fire has 
broken out in the forest or not. Now, to understand the efficiency of this model, we need to check if 
the predictions which it makes are correct or not. Thus, there exist two conditions which we need to 
ponder upon: Prediction and Reality. The prediction is the output which is given by the machine and 
the reality is the real scenario in the forest when the prediction has been made. Now let us look at 
various combinations that we can have with these two conditions. 
  
Page 2


 
 
Evaluation 
Introduction 
Till now we have learnt about the 4 stages of AI project cycle, viz. Problem scoping, Data acquisition, 
Data exploration and modelling. While in modelling we can make different types of models, how do 
we check if one’s better than the other? That’s where Evaluation comes into play. In the Evaluation 
stage, we will explore different methods of evaluating an AI model. Model Evaluation is an integral 
part of the model development process. It helps to find the best model that represents our data and 
how well the chosen model will work in the future 
What is evaluation? 
Evaluation is the process of understanding the reliability of any AI model, based on outputs by feeding 
test dataset into the model and comparing with actual answers.  There can be different Evaluation 
techniques, depending of the type and purpose of the model. Remember that It’s not recommended 
to use the data we used to build the model to evaluate it. This is because our model will simply 
remember the whole training set, and will therefore always predict the correct label for any point in 
the training set. This is known as overfitting.  
Firstly, let us go through various terms which are very important to the evaluation process.   
Model Evaluation Terminologies 
There are various new terminologies which come into the picture when we work on evaluating our 
model. Let’s explore them with an example of the Forest fire scenario. 
The Scenario 
Imagine that you have come up with an AI based prediction model which has been deployed in a forest 
which is prone to forest fires. Now, the objective of the model is to predict whether a forest fire has 
broken out in the forest or not. Now, to understand the efficiency of this model, we need to check if 
the predictions which it makes are correct or not. Thus, there exist two conditions which we need to 
ponder upon: Prediction and Reality. The prediction is the output which is given by the machine and 
the reality is the real scenario in the forest when the prediction has been made. Now let us look at 
various combinations that we can have with these two conditions. 
  
 
* Images shown here are the property of individual organisations and are used here for reference purpose only. 
   
Case 1: Is there a forest fire? 
 
Here, we can see in the picture that a forest fire has broken out in the forest. The model predicts a Yes 
which means there is a forest fire. The Prediction matches with the Reality. Hence, this condition is 
termed as True Positive. 
Case 2: Is there a forest fire? 
 
Here there is no fire in the forest hence the reality is No. In this case, the machine too has predicted 
it correctly as a No. Therefore, this condition is termed as True Negative. 
Page 3


 
 
Evaluation 
Introduction 
Till now we have learnt about the 4 stages of AI project cycle, viz. Problem scoping, Data acquisition, 
Data exploration and modelling. While in modelling we can make different types of models, how do 
we check if one’s better than the other? That’s where Evaluation comes into play. In the Evaluation 
stage, we will explore different methods of evaluating an AI model. Model Evaluation is an integral 
part of the model development process. It helps to find the best model that represents our data and 
how well the chosen model will work in the future 
What is evaluation? 
Evaluation is the process of understanding the reliability of any AI model, based on outputs by feeding 
test dataset into the model and comparing with actual answers.  There can be different Evaluation 
techniques, depending of the type and purpose of the model. Remember that It’s not recommended 
to use the data we used to build the model to evaluate it. This is because our model will simply 
remember the whole training set, and will therefore always predict the correct label for any point in 
the training set. This is known as overfitting.  
Firstly, let us go through various terms which are very important to the evaluation process.   
Model Evaluation Terminologies 
There are various new terminologies which come into the picture when we work on evaluating our 
model. Let’s explore them with an example of the Forest fire scenario. 
The Scenario 
Imagine that you have come up with an AI based prediction model which has been deployed in a forest 
which is prone to forest fires. Now, the objective of the model is to predict whether a forest fire has 
broken out in the forest or not. Now, to understand the efficiency of this model, we need to check if 
the predictions which it makes are correct or not. Thus, there exist two conditions which we need to 
ponder upon: Prediction and Reality. The prediction is the output which is given by the machine and 
the reality is the real scenario in the forest when the prediction has been made. Now let us look at 
various combinations that we can have with these two conditions. 
  
 
* Images shown here are the property of individual organisations and are used here for reference purpose only. 
   
Case 1: Is there a forest fire? 
 
Here, we can see in the picture that a forest fire has broken out in the forest. The model predicts a Yes 
which means there is a forest fire. The Prediction matches with the Reality. Hence, this condition is 
termed as True Positive. 
Case 2: Is there a forest fire? 
 
Here there is no fire in the forest hence the reality is No. In this case, the machine too has predicted 
it correctly as a No. Therefore, this condition is termed as True Negative. 
 
* Images shown here are the property of individual organisations and are used here for reference purpose only. 
   
Case 3: Is there a forest fire? 
 
Here the reality is that there is no forest fire. But the machine has incorrectly predicted that there is 
a forest fire. This case is termed as False Positive. 
Case 4: Is there a forest fire? 
 
Page 4


 
 
Evaluation 
Introduction 
Till now we have learnt about the 4 stages of AI project cycle, viz. Problem scoping, Data acquisition, 
Data exploration and modelling. While in modelling we can make different types of models, how do 
we check if one’s better than the other? That’s where Evaluation comes into play. In the Evaluation 
stage, we will explore different methods of evaluating an AI model. Model Evaluation is an integral 
part of the model development process. It helps to find the best model that represents our data and 
how well the chosen model will work in the future 
What is evaluation? 
Evaluation is the process of understanding the reliability of any AI model, based on outputs by feeding 
test dataset into the model and comparing with actual answers.  There can be different Evaluation 
techniques, depending of the type and purpose of the model. Remember that It’s not recommended 
to use the data we used to build the model to evaluate it. This is because our model will simply 
remember the whole training set, and will therefore always predict the correct label for any point in 
the training set. This is known as overfitting.  
Firstly, let us go through various terms which are very important to the evaluation process.   
Model Evaluation Terminologies 
There are various new terminologies which come into the picture when we work on evaluating our 
model. Let’s explore them with an example of the Forest fire scenario. 
The Scenario 
Imagine that you have come up with an AI based prediction model which has been deployed in a forest 
which is prone to forest fires. Now, the objective of the model is to predict whether a forest fire has 
broken out in the forest or not. Now, to understand the efficiency of this model, we need to check if 
the predictions which it makes are correct or not. Thus, there exist two conditions which we need to 
ponder upon: Prediction and Reality. The prediction is the output which is given by the machine and 
the reality is the real scenario in the forest when the prediction has been made. Now let us look at 
various combinations that we can have with these two conditions. 
  
 
* Images shown here are the property of individual organisations and are used here for reference purpose only. 
   
Case 1: Is there a forest fire? 
 
Here, we can see in the picture that a forest fire has broken out in the forest. The model predicts a Yes 
which means there is a forest fire. The Prediction matches with the Reality. Hence, this condition is 
termed as True Positive. 
Case 2: Is there a forest fire? 
 
Here there is no fire in the forest hence the reality is No. In this case, the machine too has predicted 
it correctly as a No. Therefore, this condition is termed as True Negative. 
 
* Images shown here are the property of individual organisations and are used here for reference purpose only. 
   
Case 3: Is there a forest fire? 
 
Here the reality is that there is no forest fire. But the machine has incorrectly predicted that there is 
a forest fire. This case is termed as False Positive. 
Case 4: Is there a forest fire? 
 
 
* Images shown here are the property of individual organisations and are used here for reference purpose only. 
   
Here, a forest fire has broken out in the forest because of which the Reality is Yes but the machine has 
incorrectly predicted it as a No which means the machine predicts that there is no Forest Fire. 
Therefore, this case becomes False Negative. 
Confusion matrix 
The result of comparison between the prediction and reality can be recorded in what we call the 
confusion matrix. The confusion matrix allows us to understand the prediction results. Note that it is 
not an evaluation metric but a record which can help in evaluation. Let us once again take a look at 
the four conditions that we went through in the Forest Fire example:  
 
Let us now take a look at the confusion matrix: 
 
 Prediction and Reality can be easily mapped together with the help of this confusion matrix. 
Page 5


 
 
Evaluation 
Introduction 
Till now we have learnt about the 4 stages of AI project cycle, viz. Problem scoping, Data acquisition, 
Data exploration and modelling. While in modelling we can make different types of models, how do 
we check if one’s better than the other? That’s where Evaluation comes into play. In the Evaluation 
stage, we will explore different methods of evaluating an AI model. Model Evaluation is an integral 
part of the model development process. It helps to find the best model that represents our data and 
how well the chosen model will work in the future 
What is evaluation? 
Evaluation is the process of understanding the reliability of any AI model, based on outputs by feeding 
test dataset into the model and comparing with actual answers.  There can be different Evaluation 
techniques, depending of the type and purpose of the model. Remember that It’s not recommended 
to use the data we used to build the model to evaluate it. This is because our model will simply 
remember the whole training set, and will therefore always predict the correct label for any point in 
the training set. This is known as overfitting.  
Firstly, let us go through various terms which are very important to the evaluation process.   
Model Evaluation Terminologies 
There are various new terminologies which come into the picture when we work on evaluating our 
model. Let’s explore them with an example of the Forest fire scenario. 
The Scenario 
Imagine that you have come up with an AI based prediction model which has been deployed in a forest 
which is prone to forest fires. Now, the objective of the model is to predict whether a forest fire has 
broken out in the forest or not. Now, to understand the efficiency of this model, we need to check if 
the predictions which it makes are correct or not. Thus, there exist two conditions which we need to 
ponder upon: Prediction and Reality. The prediction is the output which is given by the machine and 
the reality is the real scenario in the forest when the prediction has been made. Now let us look at 
various combinations that we can have with these two conditions. 
  
 
* Images shown here are the property of individual organisations and are used here for reference purpose only. 
   
Case 1: Is there a forest fire? 
 
Here, we can see in the picture that a forest fire has broken out in the forest. The model predicts a Yes 
which means there is a forest fire. The Prediction matches with the Reality. Hence, this condition is 
termed as True Positive. 
Case 2: Is there a forest fire? 
 
Here there is no fire in the forest hence the reality is No. In this case, the machine too has predicted 
it correctly as a No. Therefore, this condition is termed as True Negative. 
 
* Images shown here are the property of individual organisations and are used here for reference purpose only. 
   
Case 3: Is there a forest fire? 
 
Here the reality is that there is no forest fire. But the machine has incorrectly predicted that there is 
a forest fire. This case is termed as False Positive. 
Case 4: Is there a forest fire? 
 
 
* Images shown here are the property of individual organisations and are used here for reference purpose only. 
   
Here, a forest fire has broken out in the forest because of which the Reality is Yes but the machine has 
incorrectly predicted it as a No which means the machine predicts that there is no Forest Fire. 
Therefore, this case becomes False Negative. 
Confusion matrix 
The result of comparison between the prediction and reality can be recorded in what we call the 
confusion matrix. The confusion matrix allows us to understand the prediction results. Note that it is 
not an evaluation metric but a record which can help in evaluation. Let us once again take a look at 
the four conditions that we went through in the Forest Fire example:  
 
Let us now take a look at the confusion matrix: 
 
 Prediction and Reality can be easily mapped together with the help of this confusion matrix. 
 
* Images shown here are the property of individual organisations and are used here for reference purpose only. 
   
Evaluation Methods 
Now as we have gone through all the possible combinations of Prediction and Reality, let us see how 
we can use these conditions to evaluate the model. 
Accuracy 
Accuracy is defined as the percentage of correct predictions out of all the observations. A prediction 
can be said to be correct if it matches the reality. Here, we have two conditions in which the Prediction 
matches with the Reality: True Positive and True Negative. Hence, the formula for Accuracy becomes: 
 
Here, total observations cover all the possible cases of prediction that can be True Positive (TP), True 
Negative (TN), False Positive (FP) and False Negative (FN). 
As we can see, Accuracy talks about how true the predictions are by any model. Let us ponder: 
Is high accuracy equivalent to good performance? 
__________________________________________________________________________________
__________________________________________________________________________________ 
How much percentage of accuracy is reasonable to show good performance? 
__________________________________________________________________________________
__________________________________________________________________________________ 
Let us go back to the Forest Fire example. Assume that the model always predicts that there is no fire. 
But in reality, there is a 2% chance of forest fire breaking out. In this case, for 98 cases, the model will 
be right but for those 2 cases in which there was a forest fire, then too the model predicted no fire. 
Here, 
True Positives = 0 
True Negatives = 98 
Total cases = 100 
Therefore, accuracy becomes: (98 + 0) / 100 = 98%