82870034 发表于 2018-10-29 11:07:33

hadoop的HA实现,超详细(一)

  网上的转载,但超详细http://blog.csdn.net/xiaojin21cen/article/details/42611073
  我主要关注第三个问题,他们是主、备关系,但是datanode必须要求一致的吧
  问题导读:
  1、hadoop ha是通过什么配置实现自动切换的?
  2、配置中mapred与mapreduce的区别是什么?
  3、hadoop ha两个namenode之间的关系是什么?
  -- hadoop 版本:2.4.0
  -- 安装包名:
  hadoop-2.4.0.tar.gz 或者源码版本 hadoop-2.4.0-src.tar.gz(我hadoop、hbase、hive均是用的源码编译安装)
  -- 安装参考:
  http://www.netfoucs.com/article/book_mmicky/79985.html
  http://www.byywee.com/page/M0/S934/934356.html
  http://www.itpub.net/thread-1631536-1-1.html
  http://demo.netfoucs.com/u014393917/article/details/25913363
  http://www.aboutyun.com/thread-8294-1-1.html
  -- 找不到本地库
  参考:http://www.ercoppa.org/Linux-Com ... -hadoop-library.htm
  -- lzo支持,
  参考:http://blog.csdn.net/zhangzhaokun/article/details/17595325
  http://slaytanic.blog.51cto.com/2057708/1162287/
  http://hi.baidu.com/qingchunranzhi/item/3662ed5ed29d37a1adc85709
  -- 安装以下RPM包:
  yum -y install openssh*
  yum -y install man*
  yum -y install compat-libstdc++-33*
  yum -y install libaio-0.*
  yum -y install libaio-devel*
  yum -y install sysstat-9.*
  yum -y install glibc-2.*
  yum -y install glibc-devel-2.* glibc-headers-2.*
  yum -y install ksh-2*
  yum -y install libgcc-4.*
  yum -y install libstdc++-4.*
  yum -y install libstdc++-4.*.i686*
  yum -y install libstdc++-devel-4.*
  yum -y install gcc-4.*x86_64*
  yum -y install gcc-c++-4.*x86_64*
  yum -y install elfutils-libelf-0*x86_64* elfutils-libelf-devel-0*x86_64*
  yum -y install elfutils-libelf-0*i686* elfutils-libelf-devel-0*i686*
  yum -y install libtool-ltdl*i686*
  yum -y install ncurses*i686*
  yum -y install ncurses*
  yum -y install readline*
  yum -y install unixODBC*
  yum -y install zlib
  yum -y install zlib*
  yum -y install openssl*
  yum -y install patch
  yum -y install git
  yum -y -y installlzo-devel zlib-devel gcc autoconf automake libtool
  yum -y install lzop
  yum -y install lrzsz
  yum -y -y installlzo-develzlib-develgcc autoconf automake libtool
  yum -y install nc
  yum -y install glibc
  yum -y install java-1.7.0-openjdk
  yum -y install gzip
  yum -y install zlib
  yum -y install gcc
  yum -y install gcc-c++
  yum -y install make
  yum -y install protobuf
  yum -y install protoc
  yum -y install cmake
  yum -y install openssl-devel
  yum -y install ncurses-devel
  yum -y install unzip
  yum -y install telnet
  yum -y install telnet-server
  yum -y install wget
  yum -y install svn
  yum -y install ntpdate
  -- hive 安装,参考:http://kicklinux.com/hive-deploy/
5台服务器设计图
IP地址主机名NameNodeJournalNodeDataNodeZookeeperHbaseHive192.168.117.194funshion-hadoop194是是否是是否192.168.117.195funshion-hadoop195是是否是是否192.168.117.196funshion-hadoop196否是是是是(Master)是(Mysql)192.168.117.197funshion-hadoop197否是是是是否192.168.117.198funshion-hadoop198否是是是是否  --配置Linux、安装JDK
  --参考:linux(ubuntu)安装Java jdk环境变量设置及小程序测试
-- Step 1. 建立用户hadoop的ssh无密码登陆
  --参考:
  linux(ubuntu)无密码互通、相互登录高可靠文档
  CentOS6.4之图解SSH无验证双向登陆配置
  -- Step 2. zookeeper配置(配置奇数台zk集群,我用的5台)
  -- 参考:Zookeeper集群环境安装过程详解


页: [1]
查看完整版本: hadoop的HA实现,超详细(一)