RSS

set up hadoop with two nodes

01 Nov

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/

There are some points to pay attention:
1 change all the configuration file for both master and slave. core-site.xml, mapreduce.xml

2 pay attention to firewall, use sudo ufw status, to check whether there is a firewall.

3 For the namenode format, only performance it in master node. Because slave node is pointing to the master node hdfs url.

4 check java version, operating system version. They have the same version of java otherwise master and slave will synchronise very slowly.

5 Pay attention to ssh, the best way is to ssh-keygen in all nodes, and cat the public key to authorized_key and copy this file to all nodes. Make sure all authorized_keys contains all machines information in all nodes.

6 Make sure the /etc/hosts, delete 127.0.1.1 , this will cause the “getProcolversion….” error.

Advertisements
 
Leave a comment

Posted by on November 1, 2011 in Uncategorized

 

Tags:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: