我积极解决 发表于 2018-11-25 15:08:10

tomcat7+apache(mod_jk/mod_proxy)负载均衡

  操作目的
前端的web服务器接收请求,若是jsp页面,就通过mod_jk/mod_proxy连接器转发至后端的tomcat服务器,并实现tomcat的负载均衡,session会话的保持

  
简单拓扑图


  准备工作:
两台服务器
172.16.220.11安装httpd tomcat把web服务器和tomcat做在一台服务器上 主机名为tomcat1
172.16.220.12   安装tomcat   做为tomcat服务器   主机名为tomcat2
同步两台服务器的时间
hwclock -s
  一、准备两台服务器


[*]tomcat1
[*] 编译安装httpd
[*]准备的包apr-1.4.6.tar.bz2 apr-util-1.4.1.tar.bz2 httpd-2.4.2.tar.bz2
[*]
[*]安装apr
[*]#tar xf apr-1.4.6.tar.bz2
[*]#cd apr-1.4.6
[*]#./configure --prefix=/usr/local/apr
[*]#make
[*]#make install
[*]
[*]#tar xf apr-util-1.4.1.tar.bz2   
[*]#cd apr-util-1.4.1
[*]#./configure --prefix=/usr/local/apr-util --with-apr=/usr/local/apr
[*]#make
[*]#make install
[*]
[*]编译安装apache   
[*](注意:编译的时候要启用--enable-so指定apr的位置启用--enable-proxy --enable-proxy-http --enable-proxy=ajp模块)
[*]#yum -y install pcre-devel
[*]#tar xf httpd-2.4.2.tar.bz2
[*]#cd httpd-2.4.2
[*]#./configure --prefix=/usr/local/apache --sysconfdir=/etc/httpd --enable-so --enable-sll --enable-zlib --enable-cgi --enable-rewrite --with-zlib --with-pcre --with-apr=/usr/local/apr --with-apr-util=/usr/local/apr-util --enable-proxy --enable-proxy-http --enable-proxy-ajp
[*]#make   
[*]#make install

  添加PidFile文件



[*]#vim /etc/httpd/httpd.conf
[*]添加
[*]PidFile "/var/run/httpd.pid"

  提供服务脚本编辑/etc/rc.d/init.d/httpd


[*]#!/bin/bash
[*]#
[*]# httpd      Startup script for the Apache HTTP Server
[*]#
[*]# chkconfig: - 85 15
[*]# description: Apache is a World Wide Web server.It is used to serve \
[*]#      HTML files and CGI.
[*]# processname: httpd
[*]# config: /etc/httpd/conf/httpd.conf
[*]# config: /etc/sysconfig/httpd
[*]# pidfile: /var/run/httpd.pid
[*]
[*]# Source function library.
[*]. /etc/rc.d/init.d/functions
[*]
[*]if [ -f /etc/sysconfig/httpd ]; then
[*]      . /etc/sysconfig/httpd
[*]fi
[*]
[*]# Start httpd in the C locale by default.
[*]HTTPD_LANG=${HTTPD_LANG-"C"}
[*]
[*]# This will prevent initlog from swallowing up a pass-phrase prompt if
[*]# mod_ssl needs a pass-phrase from the user.
[*]INITLOG_ARGS=""
[*]
[*]# Set HTTPD=/usr/sbin/httpd.worker in /etc/sysconfig/httpd to use a server
[*]# with the thread-based "worker" MPM; BE WARNED that some modules may not
[*]# work correctly with a thread-based MPM; notably PHP will refuse to start.
[*]
[*]# Path to the apachectl script, server binary, and short-form for messages.
[*]apachectl=/usr/local/apache/bin/apachectl
[*]httpd=${HTTPD-/usr/local/apache/bin/httpd}
[*]prog=httpd
[*]pidfile=${PIDFILE-/var/run/httpd.pid}
[*]lockfile=${LOCKFILE-/var/lock/subsys/httpd}
[*]RETVAL=0
[*]
[*]start() {
[*]      echo -n $"Starting $prog: "
[*]      LANG=$HTTPD_LANG daemon --pidfile=${pidfile} $httpd $OPTIONS
[*]      RETVAL=$?
[*]      echo
[*]      [ $RETVAL = 0 ] && touch ${lockfile}
[*]      return $RETVAL
[*]}
[*]
[*]stop() {
[*] echo -n $"Stopping $prog: "
[*] killproc -p ${pidfile} -d 10 $httpd
[*] RETVAL=$?
[*] echo
[*] [ $RETVAL = 0 ] && rm -f ${lockfile} ${pidfile}
[*]}
[*]reload() {
[*]    echo -n $"Reloading $prog: "
[*]    if ! LANG=$HTTPD_LANG $httpd $OPTIONS -t >&/dev/null; then
[*]      RETVAL=$?
[*]      echo $"not reloading due to configuration syntax error"
[*]      failure $"not reloading $httpd due to configuration syntax error"
[*]    else
[*]      killproc -p ${pidfile} $httpd -HUP
[*]      RETVAL=$?
[*]    fi
[*]    echo
[*]}
[*]
[*]# See how we were called.
[*]case "$1" in
[*]start)
[*] start
[*] ;;
[*]stop)
[*] stop
[*] ;;
[*]status)
[*]      status -p ${pidfile} $httpd
[*] RETVAL=$?
[*] ;;
[*]restart)
[*] stop
[*] start
[*] ;;
[*]condrestart)
[*] if [ -f ${pidfile} ] ; then
[*]stop
[*]start
[*] fi
[*] ;;
[*]reload)
[*]      reload
[*] ;;
[*]graceful|help|configtest|fullstatus)
[*] $apachectl $@
[*] RETVAL=$?
[*] ;;
[*]*)
[*] echo $"Usage: $prog {start|stop|restart|condrestart|reload|status|fullstatus|graceful|help|configtest}"
[*] exit 1
[*]esac
[*]
[*]exit $RETVAL
[*]
[*]#chmod +x /etc/rc.d/init.d/httpd
[*]#chkconfig --add httpd
[*]#service httpd start
[*]
[*]添加apache的路径
[*]#vim /etc/profile
[*]PATH:$PATH:/usr/local/apache/bin
[*]而后重新载入/etc/profile
[*]#. /etc/profile
[*]
[*]# httpd -D DUMP_MODULES| grep proxy            //查看加载的模块
[*] proxy_http_module (shared)
[*] proxy_fcgi_module (shared)
[*] proxy_scgi_module (shared)
[*] proxy_ajp_module (shared)
[*]
[*]   
[*]tomcat1 tomcat2
[*]安装tomcat
[*]# rpm -ivh jdk-7u5-linux-i586.rpm   
[*]
[*]为JVM指定JDK路径
[*]#vim /etc/profile
[*]JAVA_HOME=/usr/java/jdk1.7.0_05
[*]PATH=$PATH:$JAVA_HOME/bin:$PATH
[*]export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE INPUTRC JAVA_HOME
[*]#. /etc/profile
[*]# printenv
[*]有如下内容
[*]JAVA_HOME=/usr/java/jdk1.7.0_05
[*]# java -version            //可以测试jdk是否生效
[*]java version "1.7.0_05"
[*]Java(TM) SE Runtime Environment (build 1.7.0_05-b05)
[*]Java HotSpot(TM) Client VM (build 23.1-b03, mixed mode, sharing)
[*]
[*]
[*]#tar xfapache-tomcat-7.0.29.tar.gz -C /usr/local
[*]# cd /usr/local
[*]# ln -sv apache-tomcat-7.0.29 tomcat
[*]# /usr/local/tomcat/bin/catalina.sh start 启动
[*]
[*]至此,两台服务器准备完毕

  tomcat1和tomcat2上统一做如下配置


[*]tomcat1
[*]#vim /usr/local/tomcat/conf/server.xml
[*]修改
[*]   
[*]演示效果,在TomcatA上某context中提供特定页面
[*]#cd /usr/local/tomcat/webapps
[*]#mkdir -pv sess/WEB-INF/{lib,classes}
[*]#cd sess
[*]#vim index.jsp
[*]添加如下内容
[*]
[*]
[*]TomcatA
[*]   
[*]    TomcatA
[*]   
[*]      
[*]      Session ID
[*]         
[*]      
[*]      
[*]      Created on
[*]         
[*]      
[*]   
[*]   
[*]
[*]#/usr/local/tomcat/conf/catalina.sh stop
[*]#/usr/local/tomcat/conf/catalina.sh stop
[*]
[*]
[*]tomcat2
[*]#vim /usr/local/tomcat/conf/server.xml
[*]修改
[*]
[*]
[*]演示效果,在TomcatB上某context中,提供特定页面
[*]#cd /usr/local/tomcat/webapps
[*]#mkdir -pv sess/WEB-INF/{lib,classes}
[*]#cd sess
[*]#vim index.jsp
[*]添加如下
[*]
[*]
[*]TomcatB
[*]   
[*]    TomcatB
[*]   
[*]      
[*]      Session ID
[*]         
[*]      
[*]      
[*]      Created on
[*]         
[*]      
[*]   
[*]   
[*]
[*]
[*]# /usr/local/tomcat/bin/catalina.sh stop
[*]# /usr/local/tomcat/bin/catalina.sh start
[*]

  二、整合apache和tomcat
2.1 基于mod_jk 实现负载均衡



[*]tomcat1:
[*]修改/etc/httpd/extra/httpd-jk.conf为如下内容:
[*]# Load the mod_jk
[*]LoadModulejk_modulemodules/mod_jk.so
[*]JkWorkersFile/etc/httpd/extra/workers.properties
[*]JkLogFilelogs/mod_jk.log
[*]JkLogLeveldebug
[*]JkMount/*lbcluster1
[*]JkMount/statusstat1
[*]保存退出                        
[*]
[*](lbcluster1这个名字是虚的,专门是为了实现负载均衡的)
[*]
[*]
[*]编辑/etc/httpd/extra/workers.properties,添加如下内容:
[*]
[*]worker.list=lbcluster1,stat1
[*]
[*]worker.TomcatA.port=8009
[*]worker.TomcatA.host=172.16.220.11
[*]worker.TomcatA.type=ajp13
[*]worker.TomcatA.lbfactor=1
[*]
[*]worker.TomcatB.port=8009
[*]worker.TomcatB.host=172.16.220.12
[*]worker.TomcatB.type=ajp13
[*]worker.TomcatB.lbfactor=1
[*]
[*]worker.lbcluster1.type=lb
[*]worker.lbcluster1.balance_workers=TomcatA,TomcatB
[*]worker.stat1.type=status
[*]worker.lbcluster1.sticky_session = 0
[*]
[*]#service httpd restart
[*]

  测试:172.16.220.11/sess 刷新访问,可以看到两台tomcat主机轮询访问,实现了负载均衡




  
问题依然存在,就是会话无法保持。接下来会解决这个问题
(把/etc/httpd/extra/workers.properties中的worker.lbcluster1.sticky_session 的值改为1,发现请求总是定向到一台tomcat服务器上)

2.2 基于mod_proxy实现负载均衡


[*]tomcat1:
[*]
[*]#vim /etc/httpd/httpd.conf
[*]Include /etc/httpd/extra/httpd-vhosts.conf
[*]#Include /etc/httpd/extra/httpd-jk.conf
[*]
[*]#vim /etc/httpd/extra/httpd-vhosts.conf
[*]内容改为:
[*]ProxyRequests Off
[*]ProxyPass / balancer://lbcluster1/ stickysession=jsessionid
[*]ProxyPa***everse / balancer://lbcluster1/
[*]
[*]      BalancerMember ajp://172.16.220.11:8009 loadfactor=10 route=TomcatA
[*]      BalancerMember ajp://172.16.220.12:8009 loadfactor=10 route=TomcatB
[*]
[*]
[*]
[*]      ServerAdmin admin@magedu.com
[*]      ServerName tomcat.magedu.com
[*]
[*]
[*]#service httpd restart
[*]

  测试tomcat.magedu.com/sess 实现了负载均衡




  
但是会话依然无法保持(若配置worker.lbcluster1.sticky_session = 1 这里的sticksession=jsessionid 访问可以实现请求定向到一台tomcat上)
  四、tomcat基于session复制的集群


[*](DeltaManager 和BackupManager都可以实现)
[*]在tomcat1主机上:
[*]#vim /usr/local/tomcat/conf/server.xml
[*]行下面添加如下内容:
[*]
[*]
[*]      
[*]      
[*]         
[*]            
[*]            
[*]               
[*]            
[*]      
[*]      
[*]      
[*]      
[*]      
[*]      
[*]      
[*]      
[*]
[*]
[*]#cp /usr/local/tomcat/conf/web.xml /usr/local/tomcat/webapps/sess/WEB-INF
[*]#vim /usr/local/tomcat/webapps/sess/WEB-INF/web.xml
[*]在
[*] 后面添加如下
[*]
[*]
[*]#/usr/local/tomcat/bin/catalina.sh stop
[*]#/usr/local/tomcat/bin/catalina.sh start
[*]#service httpd restart
[*]
[*]tomcat2上的配置和tocmat1上的配置一样,只不过是把tomcat1的地址改为tomcat2的地址为172.16.220.12,然后复制web.xml,重启服务

  访问 tomcat.magedu.com/sess 效果是负载均衡,并且会话是保持的


  
此时前端也可以是mod_jk(把apache的配置文件中的虚拟主机取消,mod_jk启用即可)


[*]操作过程中遇到了不少问题,这里分享一下:
[*]
[*]1、第五步中启动httpd时,启动成功了,但是端口并没有监听,什么原因?
[*]# tail /usr/local/apache/logs/error_log   
[*] AH01177: Failed to lookup provider 'shm' for 'slotmem': is mod_slotmem_shm loaded?
[*]编译配置文件启用这个模块即可
[*]LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
[*]
[*]2、 使用mod_jk总是实现不了复制均衡,什么问题?页面访问的一直是TomcatA,郁闷啊
[*]原因1:worker.lbcluster1.balance_workers = TomcatA,TomcatB中 “TomcatA和TomcatB之间多了一个空格,导致TomcatA出不来
[*] 2:没有加worker.lbcluster1.sticky_session = 0 ,导致无法负载均衡,添加上之后就OK了
[*]
[*]3、做mod_proxy时遇到了错误
[*]重启httpd 时错误
[*]# service httpd restart
[*]Stopping httpd:                                          
[*]Starting httpd:                                          
[*]# tail /usr/local/apache/logs/error_log
[*] (22)Invalid argument: AH01186: worker slotmem_grab failed
[*] [:emerg] AH00020: Configuration Failed, exiting
[*]
[*]解决:在虚拟主机的配置文件中,第一次是把
[*]ProxyPass / balancer://lbcluster1/ stickysession=jsessionid
[*]ProxyPa***everse / balancer://lbcluster1/ 这两项内容定义在了虚拟主机内部,出了错误
[*]
[*]定义在全局位置就可以了
[*]
[*]4、在实现session复制时,定义时,放在其他地方就不行,即便是放在
[*] 的前面就不行,一定要放在后面才可以,可以是下面的任何一定地方,这是文件读取时加载的顺序
[*]




页: [1]
查看完整版本: tomcat7+apache(mod_jk/mod_proxy)负载均衡