Hadoop2.2 伪分布式配置
部署上，很简单，就是分成两部分：修改配置文件盒启动脚本。一：修改配置文件：
hadoop2.2的配置文件在/opt/hadoop-2.2.0/etc/hadoop文件夹下，具体配置文件修改如下：
1、修改/etc/hosts文件（sudo gedit /etc/hosts）
192.168.222.154 hd2-single
2、修改core-site.xml
hadoop.tmp.dir
/home/sujx/hadoop/tmp
fs.defaultFS
hdfs://hd2-single:9000
true
fs.defaultFS：HDFS文件系统的URL
3.修改hdfs-site.xml
dfs.namenode.name.dir
file:/home/sujx/hadoop/dfs/name
true
dfs.datanode.data.dir
file:/home/sujx/hadoop/dfs/data
true
dfs.replication
1
dfs.permissions
false
4.修改mapred-site.xml
mapreduce.framework.name
yarn
mapred.system.dir
file:/home/sujx/hadoop/mapred/system
true
mapred.local.dir
file:/home/sujx/hadoop/mapred/local
true
5.修改yarn-site.xml
yarn.nodemanager.aux-services
mapreduce_shuffle
shuffle service that needsto be set for Map Reduce to run
yarn.resourcemanager.hostname
hd2-single
hostanem of RM
6.修改slave
hd2-single
至此，配置文件修改完毕，比较多，挺麻烦的。
二：启动Hadoop脚本。
启动hadoop脚本，需呀用到一些环境变量，所以需要先修改Ubuntu的profile文件。
使用命令：sudo /etc/profile
export HADOOP_HOME=/opt/hadoop-2.2.0
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
在初次运行Hadoop的时候需要初始化Hadoop文件系统，命令如下：
hdfs namenode -format
1.启动脚本一：
sujx@ubuntu:~$ hadoop-daemon.sh start namenode
starting namenode, logging to /opt/hadoop-2.2.0/logs/hadoop-sujx-namenode-ubuntu.out
sujx@ubuntu:~$ hadoop-daemon.sh start datanode
starting datanode, logging to /opt/hadoop-2.2.0/logs/hadoop-sujx-datanode-ubuntu.out
sujx@ubuntu:~$ hadoop-daemon.sh start secondarynamenode
starting secondarynamenode, logging to /opt/hadoop-2.2.0/logs/hadoop-sujx-secondarynamenode-ubuntu.out
sujx@ubuntu:~$ jps
9310 SecondaryNameNode
9345 Jps
9140 NameNode
9221 DataNode
sujx@ubuntu:~$ yarn-daemon.sh start resourcemanager
starting resourcemanager, logging to /opt/hadoop-2.2.0/logs/yarn-sujx-resourcemanager-ubuntu.out
sujx@ubuntu:~$ yarn-daemon.sh start nodemanager
starting nodemanager, logging to /opt/hadoop-2.2.0/logs/yarn-sujx-nodemanager-ubuntu.out
sujx@ubuntu:~$ jps
9310 SecondaryNameNode
9651 NodeManager
9413 ResourceManager
9140 NameNode
9709 Jps
9221 DataNode
sujx@ubuntu:~$
2.启动脚本二：
sujx@ubuntu:~$ start-dfs.sh
Starting namenodes on
hd2-single: starting namenode, logging to /opt/hadoop-2.2.0/logs/hadoop-sujx-namenode-ubuntu.out
hd2-single: starting datanode, logging to /opt/hadoop-2.2.0/logs/hadoop-sujx-datanode-ubuntu.out
Starting secondary namenodes
0.0.0.0: starting secondarynamenode, logging to /opt/hadoop-2.2.0/logs/hadoop-sujx-secondarynamenode-ubuntu.out
sujx@ubuntu:~$ start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /opt/hadoop-2.2.0/logs/yarn-sujx-resourcemanager-ubuntu.out
hd2-single: starting nodemanager, logging to /opt/hadoop-2.2.0/logs/yarn-sujx-nodemanager-ubuntu.out
sujx@ubuntu:~$ jps
11414 SecondaryNameNode
10923 NameNode
11141 DataNode
12038 Jps
11586 ResourceManager
11811 NodeManager
sujx@ubuntu:~$
3.启动脚本三：
sujx@ubuntu:~$ start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on
hd2-single: starting namenode, logging to /opt/hadoop-2.2.0/logs/hadoop-sujx-namenode-ubuntu.out
hd2-single: starting datanode, logging to /opt/hadoop-2.2.0/logs/hadoop-sujx-datanode-ubuntu.out
Starting secondary namenodes
0.0.0.0: starting secondarynamenode, logging to /opt/hadoop-2.2.0/logs/hadoop-sujx-secondarynamenode-ubuntu.out
starting yarn daemons
starting resourcemanager, logging to /opt/hadoop-2.2.0/logs/yarn-sujx-resourcemanager-ubuntu.out
hd2-single: starting nodemanager, logging to /opt/hadoop-2.2.0/logs/yarn-sujx-nodemanager-ubuntu.out
sujx@ubuntu:~$ jps
14156 NodeManager
14445 Jps
13267 NameNode
13759 SecondaryNameNode
13485 DataNode
13927 ResourceManager
sujx@ubuntu:~$
其实这三种方式最终效果都是相同，他们内部也都是相互调用关系。对应的结束脚本也简单：
1. 结束脚本一：
sujx@ubuntu:~$ hadoop-daemon.sh stop nodemanager
sujx@ubuntu:~$ hadoop-daemon.sh stop resourcemanager
sujx@ubuntu:~$ hadoop-daemon.sh stop secondarynamenode
sujx@ubuntu:~$ hadoop-daemon.sh stop datanode
sujx@ubuntu:~$ hadoop-daemon.sh stop namenode
2. 结束脚本二：
sujx@ubuntu:~$ stop-yarn.sh
sujx@ubuntu:~$ stop-dfs.sh
3. 结束脚本三：
sujx@ubuntu:~$ stop-all.sh
至此，单机伪分布就已经部署完毕。
转自：http://www.linuxidc.com/Linux/2014-01/95570p2.htm
页:
[1]