Important architecture points

Following are some important points to remember about the HDFS HA using shared storage architecture:

• In the cluster, there are two separate machines: active state NameNode and standby state NameNode.
• At any given point in time, one-and-only, one of the NameNodes is in the active state, and the other is in the standby state.
• The active NameNode manages the requests from all client DataNodes in the cluster, while the standby remains a slave.
• All the DataNodes are configured in such a way that they send their block reports and heartbeats to both the active and standby NameNodes.
• The standby NameNode keeps its state synchronized with the active NameNode.
• Active and standby nodes both have access to a filesystem on a shared storage device (for example, an NFS mount from a NAS)
• When a client makes any filesystem change, the active NameNode makes the corresponding change (edits) to the edit log file residing on the network shared directory.
• The standby NameNode makes all the corresponding changes to its own namespace. That way, it remains in sync with the active NameNode.
• In the event of the active NameNode being unavailable, the standby NameNode makes sure that it absorbs all the changes (edits) from the shared network directory and promotes itself to an active NameNode.
• The Hadoop administrator should apply the fencing method to the shared storage to avoid a scenario that makes both the NameNodes active at a given time. In the event of failover, the fencing method cuts the access to the previous active NameNode to make any changes to the shared storage to ensure smooth failover to standby NameNode. After that, the standby NameNode becomes the active NameNode.