by Commissioning of nodes means adding new data node in cluster and decommissioning stands for removing node from cluster. You can’t directly add/remove dataNode in large and a real-time cluster as it can cause a lot of disturbance. So if you want to scale your cluster , you need commissioning and steps are below.
Commission:
Pre-requirements:
- Clone existing Node.
- Change IP address and hostname – 192.168.1.155 and DN3
- Update Hosts files on all nodes – add this entry in /etc/hosts file “192.168.1.155 DN3”
- Make it password less
Configuration changes:
We need to update the include file on both the Resource Manager and the Namenode . If it’s not present, then create an include file on both the Nodes.
Go to your NameNode and add include file in hdfs-site.xml file.
|
1 2 3 4 5 6 7 |
[root@NN conf] cd /etc/hadoop/conf [root@NN conf] Vi hdfs-site.xml -- Add this property <property> <name>dfs.hosts</name> <value>/etc/hadoop/conf/includes</value> </property> |
Also update the slaves file on NameNode and add new DataNode IP address.
|
1 2 3 4 |
[root@NN conf] Vi /etc/hadoop/conf/include 192.168.1.152 192.168.1.153 192.168.1.155 |
Edit the “yarn-site.xml” file where ResourceManager is running.
|
1 2 3 4 5 6 |
[root@RM conf]# vi yarn-site.xml # Add this property path of include file. <property> <name>yarn.resourcemanager.nodes.include-path</name> <value>/etc/hadoop/conf/includes</value> </property> |
Now update the include RM.
|
1 2 3 4 |
[root@RM conf]# cat /etc/hadoop/conf/includes 192.168.1.152 192.168.1.153 192.168.1.155 |
New DataNode Set up:
Copy all configuration files from NameNode and then refresh the Nodes.
|
1 2 3 4 5 6 |
[root@NN conf]# scp -r * DN3:/etc/hadoop/conf/ [root@NN conf]# sudo -u hdfs hdfs dfsadmin -refreshNodes Refresh nodes successful [root@RM ~]# sudo -u yarn yarn rmadmin -refreshNodes 18/05/21 23:03:34 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8033 |
Now start the services on DataNode.
|
1 2 |
[root@DN3 ~]# service hadoop-hdfs-datanode start [root@DN3 ~]# service hadoop-yarn-nodemanager start |
Commissioning of new DataNode is complete. Check Hadoop admin report using the command
|
1 |
sudo -u hdfs hadoop dfsadmin -report |
Decommission a Node:
For decommission DataNode, exclude property need at NameNode side. Do decomm activity in non-peak hour. Any process running on this decommissioned node can fail.
Note: It’s very important to note that the include/exclude files should be mutually exclusive. Means can’t have same values in both exclude and include file.
|
1 2 3 4 5 6 7 8 9 |
[root@NN conf] vi hdfs-site.xml <property> <name>dfs.hosts.exclude</name> <value>/etc/hadoop/conf/excludes</value> </property> # create exclude file [root@NN conf]# cat /etc/hadoop/conf/excludes 192.168.1.155 |
Update
|
1 2 3 4 5 6 7 8 |
[root@RM conf]# vi yarn-site.xml <property> <name>yarn.resourcemanager.nodes.exclude-path</name> <value>/etc/hadoop/conf/excludes</value> </property> -- Remove file [root@RM conf]# cat /etc/hadoop/conf/excludes 192.168.1.155 |
Refresh the nodes.
|
1 2 3 4 |
[root@NN ~]# sudo -u hdfs hdfs dfsadmin -refreshNodes Refresh nodes successful [root@RM ~]# sudo -u yarn yarn rmadmin -refreshNodes 18/05/21 00:00:58 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8033 |
Hadoop Balancer:
Hadoop Balancer is a built-in property which makes sure that no datanode will be over utilized. When you run the balancer utility, it checks whether some datanode are under-utilized or over-utilized and will balance the replication factor. But make sure the Balancer should run in only off-peak hours in a real cluster, because if you run this during peak hours, it will cause a heavy load to networking, as it will transfer a large amount of data.
|
1 |
hadoop balancer |
Hope this post was helpful in understanding about the Commissioning and Decommissioning of the datanodes in Hadoop.
Thanks
Mandy
Leave a Reply