标签：des style class blog code java
| IP | Host Name | Software | Node |
| 192.168.23.128 | ae01 | JDK 1.7 | NameNode, SecondaryNameNode, DataNode, JobTracker, TaskTracker |
| 192.168.23.129 | ae02 | JDK 1.7 | DataNode, TaskTracker |
| 192.168.23.130 | ae03 | JDK 1.7 | DataNode, TaskTracker |
进入每台服务器，分别安装SSH，并生成ssh-key。下面安装步骤只讲解在ae01安装SSH和生成SSH-KEY，ae02、ae03重复此步骤。
user@ae01:~$ sudo apt-get install openssh-server
user@ae01:~# ssh-keygen -t rsa -P "" Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory ‘/root/.ssh‘. Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: 64:3d:a4:70:94:c4:33:64:6b:6b:1c:7c:e9:8f:15:93 user@ae01 The key‘s randomart image is: +--[ RSA 2048]----+ | .=*.. | | === . . | | Oo= E | | = = . o | | S . . | | . + | | . . | | | | | +-----------------+
user@ae01:~/.ssh$ sudo cp id_rsa.pub id_rsa_ae01.pub
复制id_rsa_ae01.pub到服务器ae02
user@ae01:~/.ssh$ scp ./id_rsa_ae01.pub user@192.168.23.129:~/.ssh/
登录到ae02，将id_rsa_ae01.pub 添加到authorized_keys
user@ae02:~/.ssh$ cat id_rsa_ae01.pub >> authorized_keys
重新登录到ae01，并尝试无密码方面ae02
user@ae01:~/.ssh$ ssh ae02 Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.5.0-23-generic x86_64) * Documentation: https://help.ubuntu.com/ System information as of Thu Jun 12 12:21:45 CST 2014 System load: 0.0 Processes: 86 Usage of /: 10.3% of 18.45GB Users logged in: 1 Memory usage: 35% IP address for eth0: 192.168.23.129 Swap usage: 0% Graph this data and manage this system at https://landscape.canonical.com/ 122 packages can be updated. 69 updates are security updates. Last login: Tue Jun 10 20:06:57 2014 from 192.168.23.128
对以上机器都进行如上操作，确保两两之间可以实现无密码ssh.
user@ae01:/usr/local/ae$ sudo vim /etc/hosts
127.0.0.1 localhost 192.168.23.128 ae01 192.168.23.129 ae02 192.168.23.129 ae03
user@ae01:/usr/local/ae$ sudo tar -zxvf hadoop-1.2.1.tar.gz
export HADOOP_HOME=/usr/local/ae/hadoop-1.2.1 export HADOOP_HOME_WARN_SUPPRESS=1 export PATH=$PATH:$HADOOP_HOME/bin
修改$HADOOP_HOME/conf/hadoop-env.sh
添加JAVA_HOME
export JAVA_HOME=/usr/local/ae/jdk1.7.0_51
修改$HADOOP_HOME/conf/core-site.xml
加入以下文件到<configuration>节点
<property> <name>hadoop.tmp.dir</name> <value>/usr/local/ae/storage/hadoop/temp</value> <description>A base for other temporary directories.</description> </property> <property> <name>fs.default.name</name> <value>hdfs://ae01:9000</value> <description> The name of the default file system. A URI whose scheme and authority determine the FileSystem implementation. The uri‘s scheme determines the config property (fs.SCHEME.impl) naming the FileSystem implementation class. The uri‘s authority is used to determine the host, port, etc. for a filesystem. </description> </property> <property> <name>fs.checkpoint.period</name> <value>3600</value> <description> The number of seconds between two periodic checkpoints. </description> </property> <property> <name>fs.checkpoint.size</name> <value>67108864</value> <description> The size of the current edit log (in bytes) that triggers a periodic checkpoint even if the fs.checkpoint.period hasn‘t expired. </description> </property>
修改$HADOOP_HOME/conf/hdfs-site.xml
加入以下文件到<configuration>节点
<property> <name>dfs.name.dir</name> <value>/usr/local/ae/storage/hadoop/name</value> <description> Determines where on the local filesystem the DFS name node should store the name table(fsimage). If this is a comma-delimited list of directories then the name table is replicated in all of the directories, for redundancy. </description> </property> <property> <name>dfs.data.dir</name> <value>/usr/local/ae/storage/hadoop/data</value> <description>] Determines where on the local filesystem the DFS name node should store the name table(fsimage). If this is a comma-delimited list of directories then the name table is replicated in all of the directories, for redundancy. </description> </property> <property> <name>dfs.http.address</name> <value>ae01:50070</value> <description> The address and the base port where the dfs namenode web ui will listen on. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>dfs.permissions</name> <value>false</value> <description> If "true", enable permission checking in HDFS. If "false", permission checking is turned off, but all other behavior is unchanged. Switching from one parameter value to the other does not change the mode, owner or group of files or directories. </description> </property> <property> <name>dfs.replication</name> <value>1</value> <description>Default block replication. The actual number of replications can be specified when the file is created. The default is used if replication is not specified in create time. </description> </property>
修改$HADOOP_HOME/conf/hdfs-site.xml
加入以下文件到<configuration>节点
<property> <name>mapred.job.tracker</name> <value>ae01:9001</value> <description> The host and port that the MapReduce job tracker runs at. If "local", then jobs are run in-process as a single map and reduce task. </description> </property>
修改$HADOOP_HOME/conf/masters
ae01
修改$HADOOP_HOME/conf/slaves
ae01
ae02
ae03在上述配置中：
fs.default.name的值hdfs://ae23:9000 用来决定NameNode
mapred.job.tracker的值ae23:9001 用来决定JobTracker
masters文件的值决定SecondaryNameNode
slaves文件的值决定DataNode和TaskTracker
创建文件目录/usr/local/ae/storage/hadoop, 并赋予hadoop
文件夹足够的权限
user@ae01: ~$ /usr/local/ae$ sudo chmod 777 ./storage/hadoop/
将配置好的Hadoop复制到ae02和ae03，并在ae02和ae03上创建文件目录/usr/local/ae/storage/hadoop
初始化和启动Hadoop
登录ae01，初始化
user@ae01：~$ hadoop namenode -format
启动Hadoop
user@ae01：~$ start-all.sh
使用jps查看java进程
ae01
user@ae01:/usr/local/ae$ jps
26239 JobTracker
26158 SecondaryNameNode
36052 Jps
26468 TaskTracker
25687 NameNode
25926 DataNode
ae02
user@ae02:~$ jps
25021 Jps
18999 TaskTracker
18791 DataNode
ae03
user@ae03:~$ jps
3901 DataNode
9485 Jps
4106 TaskTracker
标签：des style class blog code java
原文地址：http://www.cnblogs.com/tannerBG/p/3783395.html
