During Hadoop 2.x and onwards

In order to overcome HDFS high availability (HA) problems and make NameNode a SPOF, the architecture has changed. The new architecture provides a running of two redundant NameNodes in the same cluster in an active/passive configuration with a hot standby. This allows a fast failover to a new NameNode in the event of a machine crashing, or a graceful administrator-initiated failover for the purpose of planned maintenance. The following two architectural options are provided for HDFS HA:

  • Using shared storage
  • Using quorum journal manager