From my previous blog, we learnt how to set up a Hadoop Single Node Installation. Now, I will show how to set up a Hadoop Multi Node Cluster. A Multi Node Cluster in Hadoop contains two or more DataNodes in a distributed Hadoop environment. This is practically used in organizations to store and analyse their Petabytes and Exabytes of data.
Here in this blog, we are taking three machine to set up multi-node cluster – MN and DN1/DN2.
- Master node (MN) will run the NameNode and ResourcesManager Daemons.
- Data Nodes (DN1 and DN2) will be our data nodes that stores the actual data and provide processing power to run the jobs. Both hosts will run the DataNode and NodeManager daemons.
- REHL 7 – Set up MN and DN1/DN2 with REHL 7 operating system – Minimal Install.
- JAVA 7
Configure the System
First of all, we have to edit hosts file in /etc/ folder in MasterNode (MN) , specify the IP address of each system followed by their host names.
# vi /etc/hosts
enter the following lines in the /etc/hosts file.
Disable the firewall restrictions. Continue reading → Set up Hadoop Cluster – Multi-Node