【APP & Web架构】企业web高可用集群实战之Nginx+nginx_ajp实现Tomcat负载均衡集群+Session
一、环境：系统环境：centos 5.8 x86_64应用软件：所以应用软件都放在/usr/local/src目录下nginx-1.2.2.tar.gzyaoweibin-nginx_ajp_module-v0.2.5-17-ge971b84.zip pcre-8.12.tar.gz apache-tomcat-7.0.27.tar.gzjdk-7u4-linux-x64.gzapr-1.4.5.tar.gzapr-iconv-1.2.1.tar.gzapr-util-1.3.10.tar.gztomcat-native-1.1.23各应用软件官方站点：nginx: http://www.nginx.orgnginx_ajp: https://github.com/yaoweibin/nginx_ajp_modulepcre: http://pcre.org/apache-tomcat: http://tomcat.apache.orgjdk: http://www.oracle.com/technetwork/java/javase/downloads/index.html所需服务器： server1:nginx服务器192.168.8.10 -->安装nginx，作为分发器server2:jvm1服务器192.168.8.11 -->安装apache-tomcat、jdkserver3:jvm2服务器192.168.8.12 -->安装apache-tomcat、jdk二、安装nginx+ajp[*] 1.建立运行用户
[*] groupadd www
[*] useradd -g www -s /sbin/nologin -M www
[*] cd /usr/local/src
[*] 2.安装pcre
[*] tar zxf pcre-8.12.tar.gz
[*] cd pcre-8.12
[*] ./configure
[*] make ; make install
[*] 3.安装nginx_ajp和nginx
[*] unzip yaoweibin-nginx_ajp_module-v0.2.5-17-ge971b84.zip
[*] tar zxf nginx-1.2.2.tar.gz
[*] cd nginx-1.2.2
[*] patch -p1 < ../yaoweibin-nginx_ajp_module-e971b84/ajp.patch
[*] ./configure --prefix=/usr/local/nginx --user=www --group=www--with-http_stub_status_module--with-http_ssl_module --without-mail_pop3_module --without-mail_imap_module --without-http_uwsgi_module --without-http_scgi_module--without-http_memcached_module --without-http_autoindex_module --without-http_ssi_module --add-module=../yaoweibin-nginx_ajp_module-e971b84
[*] make ; make install
三、安装tomcat、jdk
[*] 1.安装tomcat
[*] cd /usr/local/src
[*] tar zxf apache-tomcat-7.0.27.tar.gz
[*] mv apache-tomcat-7.0.27 /usr/local/tomcat
[*] 2.安装jdk
[*] tar zxf jdk-7u4-linux-x64.gz
[*] mv jdk1.7.0_04 /usr/local/jdk
[*] 3.设置java、tomcat运行环境变量
[*] vi /etc/profile
[*] 在文件最后加入如下内容：
[*] #Set java & tomcat Environment
[*] JAVA_HOME=/usr/local/jdk
[*] export JAVA_HOME
[*] JRE_HOME=/usr/local/jdk/jre
[*] export JRE_HOME
[*] CLASSPATH=/usr=/usr/local/tomcat/common/lib/:/usr/local/jdk/lib:/usr/local/jdk/jre/lib
[*]
export>
[*] PATH=$PATH:/usr/local/tomcat/bin/:/usr/local/apache/bin:/usr/local/jdk/bin:/usr/local/jdk/jre/bin
[*] export PATH
[*] TOMCAT_HOME=/usr/local/tomcat
[*] export TOMCAT_HOME
[*] 让环境变量立即生效：
[*] source /etc/profile
四、安装apr、native
[*] (1)安装apr
[*] wget http://archive.apache.org/dist/apr/apr-1.4.5.tar.gz
[*] tar zxf apr-1.4.5.tar.gz
[*] cd apr-1.4.5
[*] ./configure --prefix=/usr/local/apr
[*] make
[*] make install
[*] (2)安装apr-iconv
[*] wget http://archive.apache.org/dist/apr/apr-iconv-1.2.1.tar.gz
[*] tar -zxf apr-iconv-1.2.1.tar.gz
[*] cd apr-iconv-1.2.1
[*] ./configure --prefix=/usr/local/apr-iconv --with-apr=/usr/local/apr
[*] make
[*] make install
[*] (3)安装apr-util
[*] wget http://archive.apache.org/dist/apr/apr-util-1.3.10.tar.gz
[*] tar zxf apr-util-1.3.10.tar.gz
[*] cd apr-util-1.3.10
[*] ./configure --prefix=/usr/local/apr-util--with-apr=/usr/local/apr --with-apr-iconv=/usr/local/apr-iconv/bin/apriconv
[*] make
[*] make install
[*] (4)安装tomcat-native
[*] 安装好tomcat7之后，tomcat-native已经在tomcat的bin目录下了，可以直接使用
[*] cd /usr/local/tomcat/bin
[*] tar zxf tomcat-native.tar.gz
[*] cd tomcat-native-1.1.23-src/jni/native
[*] ./configure --with-apr=/usr/local/apr --with-java-home=/usr/local/jdk
[*] make
[*] make install
[*]
[*] (5)设置 apr 的环境变量：
[*] vi /etc/profile
[*] 在文件后面添加以下内容:
[*] export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/apr/lib
[*]
[*] 让配置生效： source /etc/profile
PS：对于为什么要安装apr和native，是apache-tomcat-7.0.27这个版本需要这些包的支持，不然启动会出现如下错误:cat /usr/local/tomcat/logs/catalina.out Jul 27, 2012 7:19:34 PM org.apache.catalina.core.AprLifecycleListener initINFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib安装上面几个包即可解决！！！！！！！！！！！四、配置nginx在192.168.8.10服务器上操作所以环境都搭建好，下面开始配置！
[*] 1.创建 nginx 配置文件
[*] cd /usr/local/nginx/conf/
[*] mv nginx.conf nginx.conf.bak
[*] vi nginx.conf
[*] 加入如下内容：
[*] userwww www;
[*] worker_processes 4;
[*] worker_cpu_affinity 0001 0010 0100 1000;
[*] error_log/usr/local/nginx/logs/nginx_error.logcrit;
[*] pid /usr/local/nginx/nginx.pid;
[*] worker_rlimit_nofile 65535;
[*] events
[*] {
[*] use epoll;
[*] worker_connections 65535;
[*] }
[*] http
[*] {
[*] include mime.types;
[*] default_typeapplication/octet-stream;
[*] charsetutf-8;
[*] server_names_hash_bucket_size 128;
[*] client_header_buffer_size 2k;
[*] large_client_header_buffers 4 4k;
[*] client_max_body_size 8m;
[*] sendfile on;
[*] tcp_nopush on;
[*] keepalive_timeout 60;
[*] open_file_cache max=204800 inactive=20s;
[*] open_file_cache_min_uses 1;
[*] open_file_cache_valid 30s;
[*] tcp_nodelay on;
[*] gzip on;
[*] gzip_min_length1k;
[*] gzip_buffers 4 16k;
[*] gzip_http_version 1.0;
[*] gzip_comp_level 2;
[*] gzip_types text/plain application/x-javascript text/css application/xml;
[*] gzip_vary on;
[*] upstream tomcats {
[*] server 192.168.8.11:8009 srun_id=jvm1;
[*] server 192.168.8.12:8009 srun_id=jvm2;
[*]
[*] #jvm_route $cookie_JSESSIONID reverse;
[*] keepalive 10;
[*] }
[*] server
[*] {
[*] listen 80 default_server;
[*] server_name_;
[*] index index.htm index.html;
[*] root/data/www/;
[*] location / {
[*] ajp_pass tomcats;
[*] }
[*] location /status
[*] {
[*] stub_status on;
[*] }
[*] location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|js|css)$
[*] {
[*] expires 30d;
[*] }
[*] }
[*] log_formataccess'$remote_addr - $remote_user [$time_local] "$request" '
[*] '$status $body_bytes_sent "$http_referer" '
[*] '"$http_user_agent" $http_x_forwarded_for';
[*] access_log/usr/local/nginx/logs/access.logaccess;
[*] }
五、配置tomcat集群在11、12服务器操作修改server.xml配置文件：
[*] 192.168.8.11服务器：
[*] 1.在/usr/local/tomcat/conf/server.xml 中修改Enginx节点信息，大概在103行：
[*] 把
[*] 修改成：
[*]
[*] 2.加入Cluster节点信息，大概在110行左右处加入如下内容：
[*]
[*]
[*] expireSessionsOnShutdown="false"
[*] notifyListenersOnReplication="true"/>
[*]
[*]
[*] address="228.0.0.4"
[*] port="45564"
[*] frequency="500"
[*] dropTime="3000"/>
[*]
[*] address="192.168.8.11"
[*] port="4000"
[*] autoBind="100"
[*] selectorTimeout="5000"
[*] maxThreads="6"/>
[*]
[*]
[*]
[*]
[*]
[*]
[*]
[*] filter=""/>
[*]
[*]
[*] tempDir="/tmp/war-temp/"
[*] deployDir="/tmp/war-deploy/"
[*] watchDir="/tmp/war-listen/"
[*] watchEnabled="false"/>
[*]
[*]
[*]
[*] 192.168.8.12服务器：
[*] 1.在/usr/local/tomcat/conf/server.xml 中修改Enginx节点信息，大概在103行：
[*] 把
[*] 修改成：
[*]
[*] 2.加入Cluster节点信息，大概在110行左右处加入如下内容：
[*]
[*]
[*] expireSessionsOnShutdown="false"
[*] notifyListenersOnReplication="true"/>
[*]
[*]
[*] address="228.0.0.4"
[*] port="45564"
[*] frequency="500"
[*] dropTime="3000"/>
[*]
[*] address="192.168.8.12"
[*] port="4000"
[*] autoBind="100"
[*] selectorTimeout="5000"
[*] maxThreads="6"/>
[*]
[*]
[*]
[*]
[*]
[*]
[*]
[*] filter=""/>
[*]
[*]
[*] tempDir="/tmp/war-temp/"
[*] deployDir="/tmp/war-deploy/"
[*] watchDir="/tmp/war-listen/"
[*] watchEnabled="false"/>
[*]
[*]
[*]
[*] 3.测试目录负载均衡
[*] 在11、12服务器上建立测试目录，观察负载均衡情况。
[*] mkdir -p /data/www/
[*] cat /data/www/test.jsp
[*]
[*] 4.启动各服务器服务
[*] 10服务器：
[*] /usr/local/nginx/sbin/nginx
[*] 11、12服务器
[*] startup.sh
[*] 在浏览器上输入：192.168.8.10/test.jsp,然后分别在11、12服务器查看输出日志：
[*] 11服务器：
12服务器：
5.测试负载session
在/data/www目录下创建admin目录，再在admin目录下创建WEB-INF目录
mkdir -p /data/www/admin/WEB-INF
[*] cd /data/www/admin/WEB-INF
[*] vi web.xml
[*] 加入如下内容：
[*]
[*] TomcatDemo
[*]
[*]
[*] PS: 这个必须加进去，不然也实现不了复制！
[*] 在admin目录下创建测试代码：
[*] cat index.jsp
[*]
[*]
[*] Cluster App Test
[*]
[*] Server Info:
[*]
[*]
页:
[1]