nginx+keepalived高可用及双主模式
高可用有2中方式。1、Nginx+keepalived 主从配置
这种方案，使用一个vip地址，前端使用2台机器，一台做主，一台做备，但同时只有一台机器工作，另一台备份机器在主机器不出现故障的时候，永远处于浪费状态，对于服务器不多的网站，该方案不经济实惠。
2、Nginx+keepalived 双主配置
这种方案，使用两个vip地址，前端使用2台机器，互为主备，同时有两台机器工作，当其中一台机器出现故障，两台机器的请求转移到一台机器负担，非常适合于当前架构环境。
nginx+keepalived高可用
1.环境如下
lb-01:192.168.75.136/24nginx+keepalived-master
lb-02:192.168.75.137/24nginx+keepalived-backup
VIP：192.168.75.135
rs-01:192.168.75.133/24 apache
rs-02:192.168.75.13424 apache
lb操作系统centos7、rs操作系统ubuntu14.04
2.lb-01/02安装nginx切配置文件一致
nginx-repo仓库源
# cat /etc/yum.repos.d/nginx.repo
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1
安装nginx
# yum install nginx -y 启动nginx服务命令
# systemctl start nginx.service nginx配置文件内容如下
# pwd
/etc/nginx/conf.d
# cat upstream.conf
upstream pools {
server 192.168.75.133:80 weight=3;
server 192.168.75.134:80 weight=3;
}
server {
listen 80;
server_name www.zxl.com;
location / {
proxy_pass http://pools;
include /etc/nginx/conf.d/a.conf;
}
}
include a.conf配置文件内容
# cat a.conf
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 30;
proxy_send_timeout 15;
proxy_read_timeout 15;
把nginx相关配置文件拷贝到对端lb-02机器相应的目录，然后检查nginx -t语法以及重新加载即可
3.测试2台lb-01/02是否负载均衡
打开客户端浏览器分别访问lb-01/02的ip地址，结果如下
lb-01访问结果如下
lb-02访问结果如下
从上面可以看到lb-01/02已经均衡访问了
4.nginx结合keepalived高可用
为什么使用keepalived呢？使用keepalived就用来做高可用的，提供虚拟VIP
4.1分别在2台lb上安装keepalived
# yum install keepalived -y 查看keepalived版本
# keepalived -v
Keepalived v1.2.13 (11/20,2015)
4.2关于2台keepalived配置文件
在lb-01-master keepalived配置文件内容如下
# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
19872672@qq.com
}
notification_email_from root@localhost.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_MASTER
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.75.135
}
}
在lb-02-backup keepalived配置文件内容如下
# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
19872672@qq.com
}
notification_email_from root@localhost.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_BACKUP
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.75.135
}
}
然后分别启动2台keepalived服务
# systemctl start keepalived.service
# systemctl start keepalived.service
4.3查看虚拟VIP
lb-01-master机器查看
# ip a
1: lo:mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:4f:23:ba brd ff:ff:ff:ff:ff:ff
inet 192.168.75.136/24 brd 192.168.75.255 scope global dynamic eth0
valid_lft 1133sec preferred_lft 1133sec
inet 192.168.75.135/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe4f:23ba/64 scope link
valid_lft forever preferred_lft forever
从上面可以看到虚拟VIP地址192.168.75.135
lb-02-backup机器查看
root@lb-02 ~]# ip a
1: lo:mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:9d:61:b5 brd ff:ff:ff:ff:ff:ff
inet 192.168.75.137/24 brd 192.168.75.255 scope global dynamic eth0
valid_lft 1306sec preferred_lft 1306sec
inet6 fe80::20c:29ff:fe9d:61b5/64 scope link
valid_lft forever preferred_lft forever
从上面可以看到没有虚拟VIP地址
4.4测试访问虚拟VIP
打开浏览器访问
此时虚拟VIP可以轮询访问了
4.5模拟故障
把lb-01-master nginx和keepalived停止查看是否还能正常提供服务
# nginx -s stop
# systemctl stop keepalived.service
查看nginx服务是否停止
或者ps -ef|grep nginx查看也可以
# netstat -ntpl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1034/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1793/master
tcp6 0 0 :::22 :::* LISTEN 1034/sshd
tcp6 0 0 ::1:25 :::* LISTEN 1793/master
此时虚拟VIP已经不再lb-01-master上了
# ip a
1: lo:mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:4f:23:ba brd ff:ff:ff:ff:ff:ff
inet 192.168.75.136/24 brd 192.168.75.255 scope global dynamic eth0
valid_lft 1361sec preferred_lft 1361sec
inet6 fe80::20c:29ff:fe4f:23ba/64 scope link
valid_lft forever preferred_lft forever
4.6客户端打开浏览器访问是否正常访问
lb-01nginx和keepalived挂了也不影响服务
4.7查看lb-02-backup机器VIP情况
# ip a
1: lo:mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:9d:61:b5 brd ff:ff:ff:ff:ff:ff
inet 192.168.75.137/24 brd 192.168.75.255 scope global dynamic eth0
valid_lft 1366sec preferred_lft 1366sec
inet 192.168.75.135/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe9d:61b5/64 scope link
valid_lft forever preferred_lft forever
此时虚拟VIP 已经在lb-02机器上
4.8分别查看2台lb机器日志关于keepalived切换过程
lb-01的keepalived日志
# tail -f /var/log/messages
Jun 30 17:01:01 node1 systemd: Started Session 1159 of user root.
Jun 30 17:05:32 node1 systemd: Stopped nginx - high performance web server.
Jun 30 17:05:44 node1 systemd: Stopping LVS and VRRP High Availability Monitor...
Jun 30 17:05:44 node1 Keepalived: Stopping Keepalived v1.2.13 (11/20,2015)
Jun 30 17:05:44 node1 Keepalived_vrrp: VRRP_Instance(VI_1) sending 0 priority
Jun 30 17:05:44 node1 Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
Jun 30 17:05:44 node1 Keepalived_healthcheckers: Netlink reflector reports IP 192.168.75.135 removed
Jun 30 17:05:44 node1 systemd: Stopped LVS and VRRP High Availability Monitor.
可以看到服务停止后会发送一个检测试剂以及以及删除VIP等等情况
lb-02keepalived日志
# tail -f messages
Jun 30 17:01:35 node2 systemd: Started Network Manager Script Dispatcher Service.
Jun 30 17:01:35 node2 nm-dispatcher: Dispatching action 'dhcp4-change' for eth0
Jun 30 17:05:44 node2 Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Jun 30 17:05:45 node2 Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Jun 30 17:05:45 node2 Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Jun 30 17:05:45 node2 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.75.135
Jun 30 17:05:45 node2 Keepalived_healthcheckers: Netlink reflector reports IP 192.168.75.135 added
Jun 30 17:05:50 node2 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.75.135
可以看到日志输出过程中设置为master虚拟机VIP:192.168.75.135等信息
4.9上面故障模拟是人为手工操作，那么有木有自动检测切换呢？
检测脚本的内容如下
# cat /data/scripts/check_nginx_status.sh
#!/bin/bash
start_nginx=`which nginx`
nginx_status1=`ps -C nginx --no-header |wc -l`
if [ $nginx_status1 -eq 0 ];then
$start_nginx
sleep 3
nginx_status2=`ps -C nginx --no-header |wc -l`
if [ $nginx_status2 -eq 0 ];then
systemctl stop keepalived.service
fi
fi
注：lb-01/02脚本内容一样，放入计划任务即可，比如每三秒检测一下
* * * * * sleep 3; /bin/bash /data/scripts/check_nginx_status.sh
那么如何实现nginx+keepalived双主模式呢？
1.其实只是需要更改下keepalived配置文件即可，配置文件实例如下
增加新的VIP192.168.75.150,192.168.75.135是lb-01机器上主虚拟VIP，192.168.75.150是lb-02机器上主虚拟VIP
lb-01的keepalived配置文件内容如下
# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
1987277672@qq.com
}
notification_email_from root@localhost.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_MASTER
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.75.135
}
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 52
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.75.150
}
}
lb-02的keepalived配置文件内容如下
# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
1987277672@qq.com
}
notification_email_from root@localhost.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_BACKUP
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.75.135
}
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.75.150
}
}
2.分别启动2台lb上nginx和keepalived服务
lb-01启动
# nginx
# systemctl start keepalived.service
lb-01查看服务是否启动
# ps -ef|grep ninx
root 6336 61280 22:39 pts/1 00:00:00 grep --color=auto ninx
# ps -ef|grep nginx
root 6298 10 22:28 ? 00:00:00 nginx: master process nginx
nginx 6299 62980 22:28 ? 00:00:00 nginx: worker process
root 6338 61280 22:39 pts/1 00:00:00 grep --color=auto nginx
# ps -ef|grep keepalived
root 6304 10 22:29 ? 00:00:00 /usr/sbin/keepalived -D
root 6305 63040 22:29 ? 00:00:00 /usr/sbin/keepalived -D
root 6306 63040 22:29 ? 00:00:00 /usr/sbin/keepalived -D
root 6340 61280 22:39 pts/1 00:00:00 grep --color=auto keepalived
lb-01查看虚拟ip
# ip a
1: lo:mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:4f:23:ba brd ff:ff:ff:ff:ff:ff
inet 192.168.75.136/24 brd 192.168.75.255 scope global dynamic eth0
valid_lft 1506sec preferred_lft 1506sec
inet 192.168.75.135/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe4f:23ba/64 scope link
valid_lft forever preferred_lft forever
注：可以看到lb-01机器上虚拟VIP是192.168.75.135
lb-02启动
# nginx
# systemctl start keepalived.service
lb-02查看服务是否启动
# ps -ef|grep nginx
root 56849 10 22:27 ? 00:00:00 nginx: master process nginx
nginx 56850568490 22:27 ? 00:00:00 nginx: worker process
root 56899539010 22:41 pts/0 00:00:00 grep --color=auto nginx
# ps -ef|grep keepalived
root 56856 10 22:28 ? 00:00:00 /usr/sbin/keepalived -D
root 56857568560 22:28 ? 00:00:00 /usr/sbin/keepalived -D
root 56858568560 22:28 ? 00:00:00 /usr/sbin/keepalived -D
root 56901539010 22:41 pts/0 00:00:00 grep --color=auto keepalived
lb-02查看虚拟ip
# ip a
1: lo:mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:9d:61:b5 brd ff:ff:ff:ff:ff:ff
inet 192.168.75.137/24 brd 192.168.75.255 scope global dynamic eth0
valid_lft 1087sec preferred_lft 1087sec
inet 192.168.75.150/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe9d:61b5/64 scope link
valid_lft forever preferred_lft forever
注：可以看到lb-02机器上虚拟VIP是192.168.75.150
3.客户端测试访问虚拟VIP
访问虚拟VIP:192.168.75.135结果如下
访问虚拟VIP:192.168.75.150结果如下
3.模拟故障
把lb-01服务停止
# nginx -s stop
# systemctl stop keepalived.service
验证lb-01服务是否停止状态
# ps -ef|grep nginx
root 6355 61280 22:49 pts/1 00:00:00 grep --color=auto nginx
# ps -ef|grep keepalived
root 6373 61280 22:49 pts/1 00:00:00 grep --color=auto keepalived
查看lb-01虚拟VIP是否存在结果如下
# ip a
1: lo:mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:4f:23:ba brd ff:ff:ff:ff:ff:ff
inet 192.168.75.136/24 brd 192.168.75.255 scope global dynamic eth0
valid_lft 1798sec preferred_lft 1798sec
inet6 fe80::20c:29ff:fe4f:23ba/64 scope link
valid_lft forever preferred_lft forever
从上面结果可以看到，此时虚拟VIP已经不再lb-01机器了
4.测试访问虚拟VIP**
访问虚拟VIP:192.168.75.135结果如下
访问虚拟VIP:192.168.75.150结果如下
从上面结果可以看到，即使lb-01机器发生了故障也不影响使用，这样也利用2台lb资源了
查看一下lb-02虚拟ip结果情况如下
# ip a
1: lo:mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:9d:61:b5 brd ff:ff:ff:ff:ff:ff
inet 192.168.75.137/24 brd 192.168.75.255 scope global dynamic eth0
valid_lft 1020sec preferred_lft 1020sec
inet 192.168.75.150/32 scope global eth0
valid_lft forever preferred_lft forever
inet 192.168.75.135/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe9d:61b5/64 scope link
valid_lft forever preferred_lft forever
注：此时虚拟VIP地址都已经在lb-02机器上了，nginx+keepalived先这样了。服务级别还可以使用开源监控软件来进行监控，先这样吧。。。。
页:
[1]