Heartbeat+Ldirectord+LVS+FreeNas实现HA和LB群集
heartbeat的四个主要层次：1、messaging layer 用来实现HA群集中成员存活情况的探测
2、ccm 成员管理，维护HA成员的一致性
3、crm 群集资源管理，用来声明要接管的资源
4、resources agent 资源代理，控制要代理的一些服务
ldirectord 用来提供heartbeat与LVS之间的结合和对后方LB群集中Server的健康探测
LVS Linux虚拟服务，用来管控后方LB群集中的成员，实现客户端服务请求的定向
FreeNas 提供网络附加存储服务，保持LB群集中所提供服务的数据一致性
大致思路：
LVS将其权力移交至ldirectord服务，而ldirectord实际上由heartbeat掌控，后方LB群集成员的健康探测也由ldirectord服务完成。
安装步骤：
LVSàldirectordàheartbeatà安装配置FreeNasà配置LB群集中的Server
拓扑：
http://blog.运维网.com/attachment/201310/130526615.jpg
步骤：(Director1-->Director2-->FreeNas-->Server1-->Server2-->测试)
Director1：
# vim /etc/hosts
添加两行解析记录：
192.168.2.1 Director1.a.com
192.168.2.2 Director2.a.com
为了便于安装软件包，使用本地yum
# vim/etc/yum.repos.d/rhel-debuginfo.repo
1 Server]
2 name=Red Hat Enterprise Linux Server
3 baseurl=file:///mnt/cdrom/Server
4 enabled=1
5 gpgcheck=1
6 gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
7 Cluster]
8 name=Red Hat Enterprise Linux Cluster
9 baseurl=file:///mnt/cdrom/Cluster
10 enabled=1
11 gpgcheck=1
12 gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
挂载光盘
# mkdir /mnt/cdrom
# mount /dev/cdrom/mnt/cdrom/
mount: block device /dev/cdrom iswrite-protected, mounting read-only
安装ipvsadm
# yum install ipvsadm –y
确保ipvsadm服务关闭并且不随系统启动自启动
# service ipvsadm stop
Clearing the current IPVS table: [ OK ]
# chkconfig ipvsadm off
上传安装ldirectord和heartbeat所需的软件包
http://blog.运维网.com/attachment/201310/114846602.jpg
安装所有软件包
# yumlocalinstall *.rpm --nogpgcheck –y
设置heartbeat接管ldirectord服务
拷贝几个需要的配置文件
# cd/usr/share/doc/heartbeat-2.1.4/
#cp ha.cf haresources authkeys /etc/ha.d/
#cd /etc/ha.d/
编辑ha.cf文件来声明HA群集的成员以及心跳探测的方法
# vim ha.cf
91 bcast eth1# Linux /////在eth1口发送广播来进行Director之间的心跳探测
声明HA群集内的成员
211 node Director1.a.com
212 node Director2.a.com
设置Director加入HA群集的验证方式
# chmod 600 authkeys
# vim authkeys
23 auth 3/////使用第三种验证方式
24#1 crc
25#2 sha1 HI!
26 3 md5 Director /////启用这种验证方式
# vim haresources /////设置heartbeat的资源清单
45 Director1.a.com 192.168.2.100/24/eth0 ldirectord::ldirectord.cf /////使Director1成为主Director
# cp /usr/share/doc/heartbeat-ldirectord-2.1.4/ldirectord.cf ./
设置LVS移交权力至ldirectord服务
# vim ldirectord.cf
21 quiescent=no/////不使用静默，探测不到后方Server就把Server踢出ipvsadm的定向规则
22
23 #Sample for an http virtual service
24 virtual=192.168.2.100:80
25 real=192.168.2.3:80 gate
26 real=192.168.2.4:80 gate
27 service=http
28 request=".test.html" /////后方Server上的探测页面，ldirectord探测不到该页面即视为Server失效
29 receive="ok"/////探测页面的内容
30 #virtualhost=some.domain.com.au
31 scheduler=rr /////LVS选择的算法
32 #persistent=600
33 #netmask=255.255.255.255
34#protocol=tcp
35 #checktype=negotiate
36 #checkport=80
37 #request="index.html"
38 #receive="Test Page"
确保ldirectord服务关闭并且不随系统启动自启动
# service ldirectord stop
Stopping ldirectord... success
# chkconfig ldirectord off
启动heartbeat服务并设置其随系统启动而自启动
# service heartbeat start
Starting High-Availability services:
2013/10/08_20:40:27 INFO: Resource isstopped
[ OK ]
# chkconfig heartbeat on
Director2：
# vim /etc/hosts
添加两行解析记录：
192.168.2.1 Director1.a.com
192.168.2.2 Director2.a.com
为了便于安装软件包，使用本地yum
# vim/etc/yum.repos.d/rhel-debuginfo.repo
1 Server]
2 name=Red Hat Enterprise Linux Server
3 baseurl=file:///mnt/cdrom/Server
4 enabled=1
5 gpgcheck=1
6 gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
7 Cluster]
8 name=Red Hat Enterprise Linux Cluster
9 baseurl=file:///mnt/cdrom/Cluster
10 enabled=1
11 gpgcheck=1
12 gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
挂载光盘
# mkdir /mnt/cdrom
# mount /dev/cdrom/mnt/cdrom/
mount: block device /dev/cdrom iswrite-protected, mounting read-only
安装ipvsadm
# yum install ipvsadm –y
确保ipvsadm服务关闭并且不随系统启动自启动
# service ipvsadm stop
Clearing the current IPVS table: [ OK ]
# chkconfig ipvsadm off
上传安装ldirectord和heartbeat所需的软件包
http://blog.运维网.com/attachment/201310/115907958.jpg
安装所有软件包
# yumlocalinstall *.rpm --nogpgcheck –y
设置heartbeat接管ldirectord服务并且将LVS权力移交至ldirectord服务
拷贝几个需要的配置文件
在Director1上
# scp ha.cf haresources authkeys ldirectord.cfDirector2.a.com:/etc/ha.d
确保ldirectord服务关闭并且不随系统启动自启动
# service ldirectord stop
Stopping ldirectord... success
# chkconfig ldirectordoff
启动heartbeat服务并设置其随系统启动而自启动
# service heartbeat start
Starting High-Availability services:
2013/10/08_20:41:17 INFO: Resource isstopped
[ OK ]
# chkconfig heartbeat on
FreeNas：
安装FreeNas并设置共享目录
http://blog.运维网.com/attachment/201310/120543191.jpg
启动虚拟机
http://blog.运维网.com/attachment/201310/120552461.jpg
在这里要注意，安装FreeNas系统的硬盘不能作为数据存储盘
http://blog.运维网.com/attachment/201310/120559177.jpg
http://blog.运维网.com/attachment/201310/120603204.jpg
http://blog.运维网.com/attachment/201310/120609329.jpg
http://blog.运维网.com/attachment/201310/120613261.jpg
此时取出系统镜像
下面为系统成功启动后的界面
http://blog.运维网.com/attachment/201310/120626981.jpg
http://blog.运维网.com/attachment/201310/120632895.jpg
可以登录web界面进行管理了
http://blog.运维网.com/attachment/201310/120650391.jpg
http://blog.运维网.com/attachment/201310/120701133.jpg
将第二块，大小为10G的盘作为数据存储盘
http://blog.运维网.com/attachment/201310/120719334.jpg
http://blog.运维网.com/attachment/201310/120738269.jpg
开启NFS服务，并将刚才设置好的盘共享出来
http://blog.运维网.com/attachment/201310/120759318.jpg
http://blog.运维网.com/attachment/201310/120809182.jpg
最后在共享出来的目录中创建两个文件，一个站点的默认页面，一个ldirectord的探测页面，这个目录在后面的服务器上要挂载到web站点的主目录
http://blog.运维网.com/attachment/201310/120810440.jpg
Server1：
设置参数使Server1不宣告自己的IP
# echo"net.ipv4.conf.all.arp_announce = 2" >>/etc/sysctl.conf
# echo"net.ipv4.conf.all.arp_ignore = 1" >>/etc/sysctl.conf
# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route =0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 4294967295
kernel.shmall = 268435456
net.ipv4.conf.all.arp_announce= 2
net.ipv4.conf.all.arp_ignore= 1
要在lo:0接口设置VIP，保证Server能与Director通信即可，不需要将网关指向Director。
# setup
http://blog.运维网.com/attachment/201310/121616695.jpg
http://blog.运维网.com/attachment/201310/121618657.jpg
http://blog.运维网.com/attachment/201310/121624411.jpg
http://blog.运维网.com/attachment/201310/121629649.jpg
http://blog.运维网.com/attachment/201310/121632502.jpg
要注意子网掩码为“255.255.255.255”
# service network restart
安装Apache
# mkdir /mnt/cdrom
# mount /dev/cdrom/mnt/cdrom/
mount: block device /dev/cdrom iswrite-protected, mounting read-only
# rpm -ivh/mnt/cdrom/Server/httpd-2.2.3-31.el5.i386.rpm
warning: /mnt/cdrom/Server/httpd-2.2.3-31.el5.i386.rpm:Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...###########################################
1:httpd ###########################################
# service httpd start
Starting httpd: [ OK ]
# chkconfig httpd on
挂载后方FreeNas共享的目录
# showmount -e 192.168.3.3
Export list for 192.168.3.3:
/mnt/web (everyone)
# mount192.168.3.3:/mnt/web /var/www/html/ /////临时挂载
实现永久挂载
# vim /etc/fstab
192.168.3.3:/mnt/web /var/www/htmlnfs defaults,soft,intr 0 0 /////重启系统后生效
Server2上的操作与Server1相同，不再列出
测试：
正常状态下：
http://blog.运维网.com/attachment/201310/121841215.jpg
http://blog.运维网.com/attachment/201310/121844392.jpg
http://blog.运维网.com/attachment/201310/121848765.jpg
模拟Director1成为备份设备：
# cd /usr/lib/heartbeat
# ./hb_standby
2013/10/09_11:03:42 Going standby .
http://blog.运维网.com/attachment/201310/121940720.jpg
http://blog.运维网.com/attachment/201310/121942318.jpg
模拟Dirctor1恢复正常：
# ./hb_takeover
http://blog.运维网.com/attachment/201310/122025259.jpg
http://blog.运维网.com/attachment/201310/122034698.jpg
模拟Server1失效：
http://blog.运维网.com/attachment/201310/122037572.jpg
http://blog.运维网.com/attachment/201310/122042298.jpg
http://blog.运维网.com/attachment/201310/122312742.jpg
模拟Server1恢复正常：
http://blog.运维网.com/attachment/201310/122312155.jpg
http://blog.运维网.com/attachment/201310/122315979.jpg
这样就实现了高可用性和负载均衡群集。
页:
[1]