大数据系列12：Hadoop2 – 全新的Hadoop

阿娇开发978

wget http://mirrors.hust.edu.cn/apache/hadoop/common/stable2/hadoop-2.2.0.tar.gz
tar -xzvf hadoop-2.2.0.tar.gz
cd hadoop-2.2.0
vi etc/hadoop/hadoop-env.sh
修改：
export  JAVA_HOME=/home/ysc/jdk1.7.0_17
vi etc/hadoop/slaves
修改localhost为host001
vi etc/hadoop/core-site.xml
增加：
<property>
       <name>fs.defaultFS</name>
       <value>hdfs://host001:9000</value>
</property>
cp etc/hadoop/mapred-site.xml.template etc/hadoop/mapred-site.xml
vi etc/hadoop/mapred-site.xml
增加：
<property>
       <name>mapreduce.framework.name</name>
       <value>yarn</value>
</property>
<property>
       <name>mapreduce.cluster.local.dir</name>
       <value>/home/ysc/mapreduce/local</value>
</property>
vi etc/hadoop/yarn-site.xml
增加：
<property>
       <name>yarn.resourcemanager.hostname</name>  
       <value>host001</value>
</property>
<property>
       <name>yarn.nodemanager.aux-services</name> 
       <value>mapreduce_shuffle</value> 
</property>
vi etc/hadoop/hdfs-site.xml
增加：
<property>
       <name>dfs.name.dir</name>
       <value>/home/ysc/dfs/filesystem/name</value>
</property>
<property>
       <name>dfs.data.dir</name>
       <value>/home/ysc/dfs/filesystem/data</value>
</property>
<property>
       <name>dfs.replication</name>
       <value>3</value>
</property>
sudo vi /etc/profile
增加：
export  HADOOP_PREFIX=/home/ysc/hadoop-2.2.0
export  HADOOP_COMMON_HOME=/home/ysc/hadoop-2.2.0
export  HADOOP_MAPRED_HOME=/home/ysc/hadoop-2.2.0
export  HADOOP_CONF_DIR=/home/ysc/hadoop-2.2.0/etc/hadoop
export  HADOOP_HDFS_HOME=/home/ysc/hadoop-2.2.0
export  HADOOP_YARN_HOME=/home/ysc/hadoop-2.2.0
source /etc/profile
格式化：
bin/hdfs namenode -format
启动dfs：
sbin/start-dfs.sh
启动yarn：
sbin/start-yarn.sh
启动historyserver：
sbin/mr-jobhistory-daemon.sh start  historyserver
运行wordcount：
echo  "APDPlat is a java open source project,  Application Product Development Platform."  >  text1.txt
echo  "Yang Shangchuan is the founder of APDPlat which won the "2013 Outstanding Open Source Project" award."  >  text2.txt
bin/hadoop  fs  -mkdir input
bin/hadoop  fs  -put text1.txt  input
bin/hadoop  fs  -put text2.txt  input
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount  input  output
访问管理页面：
ResourceManager: http://host001:8088
NodeManager: http://host001:8042  
NameNode:http://host001:50070
NameNode:http://host001:50070
 
停止dfs：
sbin/stop-dfs.sh
停止yarn：
sbin/stop-yarn.sh
停止historyserver：
sbin/mr-jobhistory-daemon.sh stop  historyserver
 
 注意：要把这里的主机host001和用户ysc替换为你自己的，在/etc/hosts文件中要把127.0.0.1 localhost改为实际IP地址 host001
 
 
 

 
APDPlat旗下十大开源项目