- Modern Big Data Processing with Hadoop
- V. Naresh Kumar Prashant Shindgikar
- 64字
- 2025-04-04 17:12:20
Configuration of YARN
You can perform the following steps for the configuration of YARN:
- Start Hadoop NameNode, secondary NameNode, and DataNode
- Alter yarn-env.sh.
Find corresponding XML files based on your Hadoop installation.
- Add the following under the definition of YARN_CONF_DIR:
export HADOOP_CONF_DIR="${HADOOP_CONF_DIR:-$YARN_HOME/etc/hadoop}" export HADOOP_COMMON_HOME="${HADOOP_COMMON_HOME:-$YARN_HOME}" export HADOOP_HDFS_HOME="${HADOOP_HDFS_HOME:-$YARN_HOME}"
- Alter yarn-site.xml:
<?xml version="1.0"?> <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce.shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> </configuration>
- Alter mapred-site.xml:
<?xml version="1.0"?> <?xml-stylesheet href="configuration.xsl"?> <configuration> <property> <name>mapreduce.framework.name </name> <value>yarn</value> </property> </configuration>
- Start the YARN services:
yarn resourcemanager yarn nodemanager