IT & Software  >  Hadoop Tutorials: Brief Introduction  >  Hadoop NameNode Automatic Failover

Hadoop NameNode Automatic Failover Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software

Document Description: Hadoop NameNode Automatic Failover for IT & Software 2022 is part of Hadoop Tutorials: Brief Introduction preparation. The notes and questions for Hadoop NameNode Automatic Failover have been prepared according to the IT & Software exam syllabus. Information about Hadoop NameNode Automatic Failover covers topics like and Hadoop NameNode Automatic Failover Example, for IT & Software 2022 Exam. Find important definitions, questions, notes, meanings, examples, exercises and tests below for Hadoop NameNode Automatic Failover.

Introduction of Hadoop NameNode Automatic Failover in English is available as part of our Hadoop Tutorials: Brief Introduction for IT & Software & Hadoop NameNode Automatic Failover in Hindi for Hadoop Tutorials: Brief Introduction course. Download more important topics related with notes, lectures and mock test series for IT & Software Exam by signing up for free. IT & Software: Hadoop NameNode Automatic Failover Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software
1 Crore+ students have signed up on EduRev. Have you?

1. Objective

This article is all about Hadoop NameNode Automatic failover. Here we will cover Introduction to failover in Hadoop. It also covers types of Failover i.e. Graceful Failover and Automatic Failover. The components add by failover in Hadoop HDFS- ZooKeeper quorum, ZKFailoverController Process (ZKFC) are also covered on this blog in detail.


2. What is Failover?

Failover is the process in which system transfers its control to the secondary system when it detects a fault or failure.

There are two types of Failover:

  • Graceful Failover- Administrator manually initiates Graceful Failover, for example, in the case of routine maintenance. The system will not trigger a failover automatically from the active to standby namenode, even if the active node has failed in Graceful Failover.
  • Automatic Failover- In the case of NameNode failure, Failover will start automatically. This failover is known as Automatic failover.

3. Apache Hadoop NameNode Automatic Failover


Automatic failover adds below components to an HDFS deployment:

  • ZooKeeper quorum.
  • ZKFailoverController Process (ZKFC).

3.1. Zookeeper Quorum


It is a basic service for maintaining small amounts of coordination data, configuration information, and naming. And it provides distributed synchronization and group services. It informs the client of changes in the data, and track clients for failures.

Implementation of automatic HDFS failover relies on Zookeeper for:

  • Failure detection- Zookeeper maintains a session with the namenodes. At the time of failure, the session will expire. And the Zookeeper will inform other namenodes to start the failover process.
  • Active NameNode election- Zookeeper provides a mechanism to only elect a node as active. As a result, when active namenode fails, other namenode can take an exclusive lock in Zookeeper. It states that it wants to become the next active namenode.

3.2. ZKFailoverController process (ZKFC)

ZKFC is a client of Zookeeper that monitors and manages the namenode status. So, Each of the machines which run namenode service also runs a ZKFC.


ZKFC handles:

  • Health monitoring- ZKFS pings its local namenode with a health check command. The ZKFC considers the namenode healthy, as long as the namenode responds at a time. The health check will mark a node as unhealthy, when the node has crashed or entered an unhealthy state.
  • Zookeeper session management- ZKFC holds a session open in Zookeeper when the local namenode is healthy. It also holds a special “lock” znode, if the local namenode is active. If the session expires, then the lock will be automatically deleted.
  • Zookeeper-based election- In HDFS, if the local namenode is healthy, and the ZKFC sees that no other node currently holds the lock znode, it will itself try to get the lock. So, If it succeeds, then it has won the election, and it handles running a failover to make its local namenode active.

4. Conclusion

In conclusion, we can say that it starts automatically in case of NameNode failure. Automatic failover adds ZooKeeper quorum and ZKFC components to an HDFS deployment. Hence, Zookeeper is a service that provides the automatic failover capability in HDFS.

The document Hadoop NameNode Automatic Failover Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software is a part of the IT & Software Course Hadoop Tutorials: Brief Introduction.
All you need of IT & Software at this link: IT & Software
Download as PDF

Download free EduRev App

Track your progress, build streaks, highlight & save important lessons and more!

Related Searches

ppt

,

Hadoop NameNode Automatic Failover Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software

,

Hadoop NameNode Automatic Failover Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software

,

Viva Questions

,

past year papers

,

Hadoop NameNode Automatic Failover Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software

,

shortcuts and tricks

,

Previous Year Questions with Solutions

,

MCQs

,

Free

,

video lectures

,

Summary

,

pdf

,

Extra Questions

,

Objective type Questions

,

mock tests for examination

,

practice quizzes

,

Important questions

,

study material

,

Semester Notes

,

Sample Paper

,

Exam

;