Subjective Type Questions - Evaluation | Artificial Intelligence for Class 10 PDF Download

Q.1. What do you mean by Evaluation? 

Evaluation means testing a model with testing data for its accuracy of output and predictions. Depending on the Evaluation result i.e. acceptable performance a  mode is selected and deployed.

Q.2. Write the stages of AI project cycle?

AI Project Cycle stages are – 

(i) Problem Scoping
(ii) Data Acquisition
(iii) Data Exploration
(iv) Modelling
(v) Evaluation

Q.3. What is Evaluation Stage?

The evaluation stage follows modeling stage. In Evaluation stage, the desired model is trained with the training data set and ready for testing with the help of testing data set.

Q.4. What is Testing data set?

Testing data set is use to test the model before the complete deployment. Testing data set is prepared by the trained professionals after exploring and cleaning the raw big data. Testing data set is a completely different, separate and new entity for the model to be tested.

Q.5. What is Scenario?

The problem area for which a model has been developed is called scenario. Scenario is the reality in which the real problem exists and the model has to be developed. 

Q.6. Define Scenario with reference to evaluation of AI model?

Scenario is the source of real data which is fed into the model for processing either at regular intervals (hourly, daily, weekly, etc.) or in real time.  Scenario is the reality in which the real problem exists and the model has to be developed. The model has to deal with the scenario the way it has been trained.

Q.7. How many types of Scenario?

There are two types of Scenario.

(i) A regular-interval Scenario, and 

(ii) Real-time Scenario

Q.8. What is Regular-interval Scenario? Give example.

A regular-interval Scenario can be a less critical problem area in which emergent threat is not there. 

For example, pollution monitoring in a region needs weekly or monthly data, studying a cancer patient for research, monitoring the diet and fitness of sports persons, monitoring performance of students and their study habits, etc.

Q.9. Why is the process of evaluation important in any AI model?

When we are at the modelling stage, we take up various modelling types for a project. The selection of the model we take up depends on how we are going to evaluate the project. An efficient evaluation model proves helpful in selecting the most suitable modelling method that would represent our data. Therefore, the model evaluation stage is an essential part of the process of selecting an appropriate model for a project.

Q.10. Is high accuracy equivalent to good performance?

Let us take the example of forest fire. Assuming that the model always predicts that there is no fire. But in reality, there is a 2% chance of forest fire breaking out. In this case, for 98 cases, the model will be right but for those 2 cases in which there was a forest fire, then too the model predicted no fire.

Here,

True Positives = 0

True Negatives = 98

Total cases = 100

Therefore, accuracy becomes: (98 + 0) / 100 = 98%

98% is a fairly high accuracy for an AI model. But this parameter is useless for us as the actual cases where the fire broke out are not taken into account. Hence, there is a need to look at another parameter which takes account of such cases as well.

Q.11. What percentage of accuracy is reasonable for representing a good performance?

No amount of percentage is reasonable for representing a good performance. Even though the accuracy is 98% or 99%, it is useless. Let us take the example of forest fire. Assuming that the model always predicts that there is no fire. But in reality, there is a 2% chance of forest fire breaking out. In this case, for 98 cases, the model will be right but for those 2 cases in which there was a forest fire, then too the model predicted no fire.

Here,

True Positives = 0

True Negatives = 98

Total cases = 100

Therefore, accuracy becomes: (98 + 0) / 100 = 98%98%  is a fairly high accuracy for an AI model. But this parameter is useless for us as the actual cases where the fire broke out are not 

taken into account.

Q.12. Is good precision equivalent to good model performance? Why?

Precision is equivalent to good model performance in few cases but not in all cases. Taking the example of forest fire, in this case, assuming that the model always predict that there is a forest fire irrespective of the reality. In this case, all the Positive conditions would be taken into account that is, True Positive (Prediction = Yes and Reality = Yes) and False Positive (Prediction = Yes and Reality = No). In this case, the firefighters will check for the fire all the time to see if the alarm was True or False. Similarly, here if the Precision is low (which means there are more False alarms than the actual ones) then the firefighters would get complacent and might not go and check every time considering it could be a false alarm. This makes Precision an important evaluation criteria. If Precision is high, this means the True Positive cases are more, giving lesser False alarms.

Q.13. What is the difference between Precision and Recall?

Precision: Precision is defined as the percentage of true positive cases versus all the cases where the prediction is true. 

Precision = TP/(TP + FP) x 100%

Precision takes into account True Positive (TP) and False Positive (FP).

Recall: Recall can be defined as the fraction of positive cases that are correctly identified. 

Recall = TP/(TP + FN)

Recalls considers True Positive (TP) and False Nega