hadoop学习笔记之二：1.0.1搭建集群
安装环境写道
两台主机：
192.168.1.2 bigdata2
192.168.1.4 bigdata3
1 安装JDK （两台主机安装一个版本的JDK）
# echo $JAVA_HOME
/opt/java/jdk1.6.0_45
2 安装SSH，配置面密码登录（bigdata2 可以免密码登录到bigdata3）
在bigdata2上面执行如下指令
ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
id_rsa.pub复制到bigdata3机器下的.ssh/authorized_keys文件里
scpid_rsa.pub root@bigdata3:/root/.ssh/authorized_keys
ssh bigdata3
3 进入conf目录，修改配置文件
core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://bigdata2:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/root/u/tmp/hadoop-${user.name}</value>
</property>
</configuration>
假如/root/u/tmp目录不存在，需要建立起来 (两台机器上目录位置相同)
mkdir -p /root/u/tmp
hdfs-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>bigdata2:9001</value>
</property>
</configuration>
masters
# more masters
bigdata2
slaves
# more slaves
bigdata3
修改hadoop-env.sh 设置JAVA_HOME
# The java implementation to use.Required.
# export JAVA_HOME=/usr/lib/j2sdk1.5-sun
export JAVA_HOME=/opt/java/jdk1.6.0_45
# Extra Java CLASSPATH elements.Optional.
# export HADOOP_CLASSPATH=
# The maximum amount of heap to use, in MB. Default is 1000.
# export HADOOP_HEAPSIZE=2000
# Extra Java runtime options.Empty by default.
# export HADOOP_OPTS=-server
4 拷贝配置好的hadoop目录到bigdata3，放到与bigdata2相同的目录
5 格式化HDFS文件系统
/bin/hadoop namenode -format
6 启动
./bin/start-all.sh
7 jps 检测
# jps
6096 Jps
5452 NameNode
5758 JobTracker
5630 SecondaryNameNode
8 进入bigdata3
# ssh bigdata3
Last login: Thu Nov 27 11:36:27 2014 from bigdata2
# jps
19584 DataNode
20226 Jps
19713 TaskTracker
9 访问50070和50030端口查看集群和HDFS的状态
页:
[1]