hadoop2.7.2修改配置文件，配置linux java环境变量
1.修改core-site.xml，加上fs.defaultFS
hdfs://backup02:9000
hadoop.tmp.dir
file:/home/zhongml/hadoop-2.7.2/tmp
io.file.buffer.size
131702
2.修改hdfs-site.xml，加上
dfs.namenode.name.dir
file:/home/zhongml/hadoop-2.7.2/hdfs/name
dfs.datanode.data.dir
file:/home/zhongml/hadoop-2.7.2/hdfs/data
dfs.replication
2
dfs.namenode.secondary.http-address
backup02:9001
dfs.webhdfs.enabled
true
3.先复制一个mapred-site.xml
cp mapred-site.xml.template mapred-site.xml
修改mapred-site.xml，加上
mapreduce.framework.name
yarn
mapreduce.jobhistory.address
backup02:10020
mapreduce.jobhistory.webapp.address
backup02:19888
4.修改yarn-site.xml，加上
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.auxservices.mapreduce.shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
yarn.resourcemanager.address
backup02:8032
yarn.resourcemanager.scheduler.address
backup02:8030
yarn.resourcemanager.resource-tracker.address
backup02:8031
yarn.resourcemanager.admin.address
backup02:8033
yarn.resourcemanager.webapp.address
backup02:8088
yarn.nodemanager.resource.memory-mb
768
5.配置/home/zhongml/hadoop-2.7.2/etc/hadoop目录下hadoop-env.sh、yarn-env.sh的JAVA_HOME
6.配置/home/zhongml/hadoop-2.7.2/etc/hadoop目录下slaves
7.验证启动是否成功
bin/hadoop fs -ls
http://localhost:50030(MapReduce的页面)
http://localhost:50070(HDFS的页面)
8.如果是64位的linux，需要覆盖native
9.配置java环境变量/etc/profile
先卸载linux自带的jdk，
执行以下命令查看需要卸载的选项:
rpm -qa | grep gcj
执行卸载操作:
yum -y remove java-1.4.2-gcj-compat-1.4.2.0-40jpp.115
将以下内容添加到/etc/profile末尾
JAVA_HOME=/jdk/jdk1.8.0_101/
PATH=$JAVA_HOME/bin:$PATH
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_HOME
export PATH
export> 注意:装好hadoop环境必须关闭防火墙:
关闭防火墙服务
# service iptables stop
关闭开机自动启动
# chkconfig iptables off
安装以下软件:
yum install svn
yum install autoconfautomake libtool cmake
yum install ncurses-devel
yum install openssl-devel
yum install gcc*
页:
[1]