mac Hadoop安装

云深处

mac hadoop安装
在mac上安装hadoop伪分布式
 
杜伟
目录


一、下载jdk...................................................................................... 1
二、配置hadoop................................................................................ 1
三、配置ssh，无密码登录.................................................................... 2
四、启动hadoop................................................................................ 3
五、环境配置...................................................................................... 4
 



一、 下载jdk
选择最新版本下载，地址：http://www.oracle.com/technetwork/java/javase/downloads/index.html
安装完成之后，打开终端，输入java -version ,出现类似如下说明安装成功。
java version "1.8.0"
Java(TM) SE Runtime Environment (build 1.8.0-b132)
Java HotSpot(TM) 64-Bit Server VM (build 25.0-b70, mixed mode)
 
二、 配置hadoop
下载hadoop，自己可到官网下载稳定版本，本次搭建用的是hadoop-2.6.0
配置hadoop 里面hadoop-2.6.0/etc/hadoop文件夹四个文件（hadoop-env.sh,core-site.xml,mapred-site.xml,hdfs-site.xml）
下载完hadoop之后，把它解压到你想存放的文件夹，然后进入hadoop的etc/hadoop目录
1.   配置hadoop-env.sh
修改为：
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home
export HADOOP_HEAPSIZE=2000
export HADOOP_OPTS=-server
即去掉前面的解释符#，mac的jdk安装在根目录Libray文件夹下面。
2.   配置core-site.xml
<configuration>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/Users/duwei/software/hadoop-2.6.0/tmp</value>
    </property>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://hadoop0:9000</value>
    </property>
</configuration>
3.   配置mapred-site.xml
<configuration>
    <property>
        <name>mapred.job.tracker</name>
        <value>hdfs://localhost:9001</value>
    </property>
</configuration>
4.   配置hdfs-site.xml
<configuration>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>/Users/duwei/software/hadoop-2.6.0/hdfs/namenode</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>/Users/duwei/software/hadoop-2.6.0/hdfs/datanode</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>
三、 配置ssh，无密码登录
mac上已经ssh了，在终端输入ssh-keygen -t rsa命令，碰到需要输入密码的直接按enter健即可。出现如下成功
·       Enter passphrase (empty for no passphrase):
·       Enter same passphrase again:
·       Your identification has been saved in /Users/jia/.ssh/id_rsa.
·       Your public key has been saved in /Users/jia/.ssh/id_rsa.pub.
·       The key fingerprint is:
·       d4:85:aa:83:ae:db:50:48:0c:5b:dd:80:bb:fa:26:a7 jia@JIAS-MacBook-Pro.local
·       The key's randomart image is:
·       +--[ RSA 2048]----+
·       |. .o.o     ..    |
·       | =. . .  ...     |
·       |. o.    ...      |
·       | ...   ..        |
·       |  .... .S        |
·       |  ... o          |
·       | ...   .         |
·       |o oo.            |
·       |E*+o.            |
·       +-----------------+
在终端输入cd ~进入用户目录，再输入cd .ssh目录，输入命令
cp id_rsa.pub authorized_keys即可
 
四、 启动hadoop
1.    进入hadoop文件夹，用如下命令格式化
cd bin
./hadoop namenode –format
 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = JIAS-MacBook-Pro.local/192.168.1.3
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 0.20.2
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
Re-format filesystem in /tmp/hadoop-jia/dfs/name ? (Y or N) Y
14/07/14 13:55:17 INFO namenode.FSNamesystem: fsOwner=jia,staff,everyone,localaccounts,_appserverusr,admin,_appserveradm,_lpadmin,com.apple.sharepoint.group.1,_appstore,_lpoperator,_developer,com.apple.access_screensharing,com.apple.access_ssh
14/07/14 13:55:17 INFO namenode.FSNamesystem: supergroup=supergroup
14/07/14 13:55:17 INFO namenode.FSNamesystem: isPermissionEnabled=true
14/07/14 13:55:17 INFO common.Storage: Image file of size 93 saved in 0 seconds.
14/07/14 13:55:17 INFO common.Storage: Storage directory /tmp/hadoop-jia/dfs/name has been successfully formatted.
14/07/14 13:55:17 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at JIAS-MacBook-Pro.local/192.168.1.3
************************************************************/
2.   启动hadoop守护进程
bin/start-all.sh
3.   停止hadoop守护进程
bin/stop-all.sh
五、 环境配置
~  cat .bash_profile
 
export JAVA_HOME=$(/usr/libexec/java_home -v 1.7)
export HADOOP_HOME=/Users/duwei/software/hadoop-2.6.0
export PATH=$PATH:$HADOOP_HOME/bin
 
用如下命令使其生效
source ~/.bash_profile

 
 参照：http://www.cnblogs.com/aijianiula/p/3842581.html