Defining NameNode

The NameNode is a master node of all DataNodes in the Hadoop cluster. It stores only the metadata of files and directories stored in the form of a tree. The important point is NameNode never stores any other data other than metadata. NameNode keeps track of all data written to DataNodes in the form of blocks. The default block size is 256 MB (which is configurable). Without the NameNode, the data on the DataNodes filesystem cannot be read. The metadata is stored locally on the NameNode using two files—filesystem namespace image file, FSImage, and edit logs. FSImage is the snapshot of the filesystem from the start of the NameNode edit logs—all the changes of the filesystem since the NameNode started, when the NameNode starts, it reads FSImage file and edits log files. All the transactions (edits) are merged into the FSImage file. The FSImage file is written to disk and a new, empty edits log file is created to log all the edits. Since NameNode is not restarted very often, the edits log file becomes very large and unmanageable. When NameNode is restarted, it takes a very long time to restart it as all the edits need to be applied to the FSImage file. In the event of NameNode crashing, all the metadata in the edits log file will not be written to the FSImage file and will be lost.