MySQL MHA配置

cundeng

　　MySQL环境：
　　master：192.168.202.129:3306
　　slave：192.168.202.129:3307，192.168.202.129:3307，192.168.202.130:3306，192.168.202.130:3307，192.168.202.130:3308
　　MariaDB-10.0.13
　　MHA环境：
　　manager：192.168.202.129
　　node：192.168.202.130(3306端口监听的实例作为备用主库)，192.168.202.129
　　下载地址：http://code.google.com/p/mysql-master-ha/ （这个需要FQ），我在百度网盘里共享了一份：http://pan.baidu.com/s/1pJ0VkSz
　　一 MySQL安装准备：
　　1 建立MySQL安装目录：mkdir -p /usr/local/mysql
　　2 建立mysql用户：
　　



groupadd mysql
useradd -g mysql mysql
　　3 下载依赖包：
　　



yum  install make apr* autoconf automake curl curl-devel gcc gcc-c++ gtk+-devel zlib-devel openssl openssl-devel pcre-devel gd kernel keyutils patch perl kernel-headers compat*  cpp glibc libgomp libstdc++-devel keyutils-libs-devel libsepol-devel libselinux-devel krb5-devel  libXpm* freetype freetype-devel freetype* fontconfig fontconfig-devel  libjpeg* libpng* php-common php-gd gettext gettext-devel ncurses* libtool* libxml2 libxml2-devel patch policycoreutils bison
　　4 下载cmake：yum install cmake -y
　　5 开始编译：
　　



cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/home/mysql/data -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci -DEXTRA_CHARSETS=all -DENABLED_LOCAL_INFILE=1
　　6 结束之后，输入下面的命令：make && make install
　　上面的过程结束之后就安装好了MySQL。下面是配置过程：
　　二 编译安装MySQL：
　　1 修改环境变量：vi /etc/profile，添加下面的语句：
　　



export PATH=$PATH:/usr/local/mysql/bin
　　2 source /etc/profile加载环境变量
　　3 将/usr/local/mysql/support_files下的my-default.cnf复制到/etc下：cp /usr/local/mysql/support_files/my-default.cnf /etc/my.cnf，修改这个文件加入下面的内容：
　　



[mysqld_multi]
mysqld = /usr/local/mysql/bin/mysqld_safe
mysqladmin = /usr/local/mysql/bin/mysqladmin
log = /home/mysql/mydata/log/mysqld_multi.log

[mysqld1]
socket = /home/mysql/mydata/data1/socket/mysqld.sock
port = 3306
pid-file = /home/mysql/mydata/data1/mysqld.pid
datadir = /home/mysql/mydata/data1
log_bin=/home/mysql/mydata/data1/binlog/mysql-bin
server-id = 1
[mysqld2]
socket = /home/mysql/mydata/data2/socket/mysqld.sock
port = 3307
pid-file = /home/mysql/mydata/data2/mysqld.pid
datadir = /home/mysql/mydata/data2
log_bin=/home/mysql/mydata/data2/binlog/mysql-bin
server-id = 2
relay_log = /home/mysql/mydata/data2/relay_log/mysql-relay-bin
log_slave_updates = 1
read_only = 1
[mysqld3]
socket = /home/mysql/mydata/data3/socket/mysqld.sock
port = 3308
pid-file = /home/mysql/mydata/data3/mysqld.pid
datadir = /home/mysql/mydata/data3
log_bin=/home/mysql/mydata/data3/binlog/mysql-bin
server-id = 3
relay_log = /home/mysql/mydata/data3/relay_log/mysql-relay-bin
log_slave_updates = 1
read_only = 1
　　上面的路径都需要用mysql用户建立，这里不再赘述。
　　4 下面就可以初始化数据库了：
　　



/usr/local/mysql/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/home/mysql/mydata/data1
/usr/local/mysql/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/home/mysql/mydata/data2
/usr/local/mysql/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/home/mysql/mydata/data3
　　这样就把一个服务器上所有的数据库都建立好了，在130服务器上也是如法炮制，不过在修改my.cnf的时候要注意一点就是修改server-id，从4以后开始，不要和129服务器的相同。
　　下面就可以启动数据库了，命令很简单：
　　



mysqld_multi --defaults-extra-file=/etc/my.cnf start 1,2,3
　　这就将三个数据库启动好了，可以用下面的语句检查：
　　



mysqld_multi --defaults-extra-file=/etc/my.cnf report
MySQL server from group: mysqld1 is running
MySQL server from group: mysqld2 is running
MySQL server from group: mysqld3 is running
　　
　　三 主从配置
　　1 主库建立复制用户：
　　



在192.168.202.129上：
$mysql -uroot -P3306 -p127.0.0.1
grant replication slave, replication client on *.* to repl@'192.168.202.%' identified by 'repl'; -- 这里限制在局域网内

　　2 从库配置：
　　



在192.168.202.129上：
$mysql -uroot -P3307 -p127.0.0.1
change master to master_host='192.168.202.129', MASTER_PORT=3306, master_user='repl', master_password='repl', master_log_file='mysql-bin.000001', master_log_pos=0;
start slave;

$mysql -uroot -P3308 -p127.0.0.1
change master to master_host='192.168.202.129', MASTER_PORT=3306, master_user='repl', master_password='repl', master_log_file='mysql-bin.000001', master_log_pos=0;
start slave;
在192.168.202.130上：
$mysql -uroot -P3306 -p127.0.0.1
change master to master_host='192.168.202.129', MASTER_PORT=3306, master_user='repl', master_password='repl', master_log_file='mysql-bin.000001', master_log_pos=0;
start slave;
$mysql -uroot -P3307 -p127.0.0.1
change master to master_host='192.168.202.129', MASTER_PORT=3306, master_user='repl', master_password='repl', master_log_file='mysql-bin.000001', master_log_pos=0;
start slave;
$mysql -uroot -P3308 -p127.0.0.1
change master to master_host='192.168.202.129', MASTER_PORT=3306, master_user='repl', master_password='repl', master_log_file='mysql-bin.000001', master_log_pos=0;
start slave;
　　这样子就配置结束了。
　　3 配置好了以后主库上会有slave过来的进程：
　　mysql>show processlist;

　　有几个slave，这里就会有多少个进程。
　　4



# ln -s /usr/local/mysql/bin/mysqlbinlog /usr/bin/mysqlbinlog
# ln -s /usr/local/mysql/bin/mysql /usr/bin/mysql
　　
　　三 MHA配置
　　1 依赖安装：
　　在node节点上，执行下面的命令：
　　



# yum install perl-DBD-MySQL
　　在manager节点上：
　　



# yum install perl-DBD-MySQL
# yum install perl-Config-Tiny
# yum install perl-Log-Dispatch
# yum install perl-Parallel-ForkManager
# yum install -y rrdtool perl-rrdtool rrdtool-devel perl-Params-Validate
　　2 rpm包安装：
　　manager节点：
　　



# rpm -ivh mha4mysql-manager-0.56-0.el6.noarch.rpm
# rpm -ivh mha4mysql-node-0.56-0.el6.noarch.rpm
　　node节点：
　　



# rpm -ivh mha4mysql-node-0.56-0.el6.noarch.rpm
　　3 配置
　　



mkdir -p /etc/masterha/
mkdir -p /var/log/masterha/app1/
　　将下面的内容写进/etc/masterha/app1.cnf:



[server default]
manager_workdir=/var/log/masterha/app1
manager_log=/var/log/masterha/app1/manager.log
user=admin
password=admin
ssh_user=root
repl_user=repl
repl_password=repl
ping_interval=1
shutdown_script=""
master_ip_online_change_script=""
report_script=""

[server1]
hostname=192.168.126.129
port=3306
candidate_master=1
master_binlog_dir="/home/mysql/mydata/data1/binlog"

[server2]
hostname=192.168.126.130
port=3306
candidate_master=1
master_binlog_dir="/home/mysql/mydata/data1/binlog"
[server3]
hostname=192.168.126.130
port=3307
[server4]
hostname=192.168.126.130
port=3308
[server5]
hostname=192.168.126.129
port=3307
[server6]
hostname=192.168.126.129
port=3308
　　4 配置ssh免秘钥：



# ssh-keygen -t rsa
# cd ~/.ssh
# cat id_rsa.pub >> authorized_keys
# chmod 600 authorized_keys
　　将id_rsa.pub的内容粘贴到另一台服务器的~/.ssh/authorized_keys里（保证两个机器互相信任）。
　　5 测试一下ssh：



masterha_check_ssh --conf=/etc/masterha/app1.cnf
　　下面的结果出来就对了：
　　



Sat Oct 18 14:04:43 2014 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Sat Oct 18 14:04:43 2014 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Sat Oct 18 14:04:43 2014 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Sat Oct 18 14:04:43 2014 - [info] Starting SSH connection tests..
Sat Oct 18 14:06:29 2014 - [debug]
Sat Oct 18 14:04:46 2014 - [debug]  Connecting via SSH from root@192.168.126.129(192.168.126.129:22) to root@192.168.126.129(192.168.126.129:22)..
Sat Oct 18 14:05:07 2014 - [debug]   ok.
Sat Oct 18 14:05:07 2014 - [debug]  Connecting via SSH from root@192.168.126.129(192.168.126.129:22) to root@192.168.126.130(192.168.126.130:22)..
Sat Oct 18 14:05:28 2014 - [debug]   ok.
Sat Oct 18 14:05:28 2014 - [debug]  Connecting via SSH from root@192.168.126.129(192.168.126.129:22) to root@192.168.126.130(192.168.126.130:22)..
Sat Oct 18 14:05:49 2014 - [debug]   ok.
Sat Oct 18 14:05:49 2014 - [debug]  Connecting via SSH from root@192.168.126.129(192.168.126.129:22) to root@192.168.126.130(192.168.126.130:22)..
Sat Oct 18 14:06:10 2014 - [debug]   ok.
Sat Oct 18 14:06:10 2014 - [debug]  Connecting via SSH from root@192.168.126.129(192.168.126.129:22) to root@192.168.126.129(192.168.126.129:22)..
Sat Oct 18 14:06:29 2014 - [debug]
Sat Oct 18 14:04:45 2014 - [debug]  Connecting via SSH from root@192.168.126.130(192.168.126.130:22) to root@192.168.126.129(192.168.126.129:22)..
Sat Oct 18 14:05:06 2014 - [debug]   ok.
Sat Oct 18 14:05:06 2014 - [debug]  Connecting via SSH from root@192.168.126.130(192.168.126.130:22) to root@192.168.126.130(192.168.126.130:22)..
Sat Oct 18 14:05:27 2014 - [debug]   ok.
Sat Oct 18 14:05:27 2014 - [debug]  Connecting via SSH from root@192.168.126.130(192.168.126.130:22) to root@192.168.126.130(192.168.126.130:22)..
Sat Oct 18 14:05:48 2014 - [debug]   ok.
Sat Oct 18 14:05:48 2014 - [debug]  Connecting via SSH from root@192.168.126.130(192.168.126.130:22) to root@192.168.126.129(192.168.126.129:22)..
Sat Oct 18 14:06:09 2014 - [debug]   ok.
Sat Oct 18 14:06:09 2014 - [debug]  Connecting via SSH from root@192.168.126.130(192.168.126.130:22) to root@192.168.126.129(192.168.126.129:22)..
Sat Oct 18 14:06:31 2014 - [info] All SSH connection tests passed successfully.
　　6 测试一下复制：
　　



masterha_check_repl --conf=/etc/masterha/app1.cnf


Sat Oct 18 14:12:45 2014 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Sat Oct 18 14:12:45 2014 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Sat Oct 18 14:12:45 2014 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Sat Oct 18 14:12:45 2014 - [info] MHA::MasterMonitor version 0.56.
Sat Oct 18 14:12:46 2014 - [info] GTID failover mode = 0
Sat Oct 18 14:12:46 2014 - [info] Dead Servers:
Sat Oct 18 14:12:46 2014 - [info] Alive Servers:
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.129(192.168.126.129:3306)
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.130(192.168.126.130:3306)
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.130(192.168.126.130:3307)
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.130(192.168.126.130:3308)
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.129(192.168.126.129:3307)
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.129(192.168.126.129:3308)
Sat Oct 18 14:12:46 2014 - [info] Alive Slaves:
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.130(192.168.126.130:3306)  Version=10.0.13-MariaDB-log (oldest major version between slaves) log-bin:enabled
Sat Oct 18 14:12:46 2014 - [info]     Replicating from 192.168.126.129(192.168.126.129:3306)
Sat Oct 18 14:12:46 2014 - [info]     Primary candidate for the new Master (candidate_master is set)
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.130(192.168.126.130:3307)  Version=10.0.13-MariaDB-log (oldest major version between slaves) log-bin:enabled
Sat Oct 18 14:12:46 2014 - [info]     Replicating from 192.168.126.129(192.168.126.129:3306)
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.130(192.168.126.130:3308)  Version=10.0.13-MariaDB-log (oldest major version between slaves) log-bin:enabled
Sat Oct 18 14:12:46 2014 - [info]     Replicating from 192.168.126.129(192.168.126.129:3306)
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.129(192.168.126.129:3307)  Version=10.0.13-MariaDB-log (oldest major version between slaves) log-bin:enabled
Sat Oct 18 14:12:46 2014 - [info]     Replicating from 192.168.126.129(192.168.126.129:3306)
Sat Oct 18 14:12:46 2014 - [info]   192.168.126.129(192.168.126.129:3308)  Version=10.0.13-MariaDB-log (oldest major version between slaves) log-bin:enabled
Sat Oct 18 14:12:46 2014 - [info]     Replicating from 192.168.126.129(192.168.126.129:3306)
Sat Oct 18 14:12:46 2014 - [info] Current Alive Master: 192.168.126.129(192.168.126.129:3306)
Sat Oct 18 14:12:46 2014 - [info] Checking slave configurations..
Sat Oct 18 14:12:46 2014 - [info]  read_only=1 is not set on slave 192.168.126.130(192.168.126.130:3306).
Sat Oct 18 14:12:46 2014 - [warning]  relay_log_purge=0 is not set on slave 192.168.126.130(192.168.126.130:3306).
Sat Oct 18 14:12:46 2014 - [warning]  relay_log_purge=0 is not set on slave 192.168.126.130(192.168.126.130:3307).
Sat Oct 18 14:12:46 2014 - [warning]  relay_log_purge=0 is not set on slave 192.168.126.130(192.168.126.130:3308).
Sat Oct 18 14:12:46 2014 - [warning]  relay_log_purge=0 is not set on slave 192.168.126.129(192.168.126.129:3307).
Sat Oct 18 14:12:46 2014 - [warning]  relay_log_purge=0 is not set on slave 192.168.126.129(192.168.126.129:3308).
Sat Oct 18 14:12:46 2014 - [info] Checking replication filtering settings..
Sat Oct 18 14:12:46 2014 - [info]  binlog_do_db= , binlog_ignore_db=
Sat Oct 18 14:12:46 2014 - [info]  Replication filtering check ok.
Sat Oct 18 14:12:46 2014 - [info] GTID (with auto-pos) is not supported
Sat Oct 18 14:12:46 2014 - [info] Checking MHA Node version..
Sat Oct 18 14:13:40 2014 - [info]  Version check ok.
Sat Oct 18 14:13:40 2014 - [info] Checking SSH publickey authentication settings on the current master..
Sat Oct 18 14:13:45 2014 - [warning] HealthCheck: Got timeout on checking SSH connection to 192.168.126.129! at /usr/share/perl5/vendor_perl/MHA/HealthCheck.pm line 342.
Sat Oct 18 14:13:45 2014 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Sat Oct 18 14:13:45 2014 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='admin' --slave_host=192.168.126.130 --slave_ip=192.168.126.130 --slave_port=3306 --workdir=/var/tmp --target_version=10.0.13-MariaDB-log --manager_version=0.56 --relay_log_info=/home/mysql/mydata/data1/relay-log.info  --relay_dir=/home/mysql/mydata/data1/  --slave_pass=xxx
Sat Oct 18 14:13:45 2014 - [info]   Connecting to root@192.168.126.130(192.168.126.130:22)..
Checking slave recovery environment settings..
Opening /home/mysql/mydata/data1/relay-log.info ... ok.
Relay log found at /home/mysql/mydata/data1, up to mysqld-relay-bin.000002
Temporary relay log file is /home/mysql/mydata/data1/mysqld-relay-bin.000002
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Sat Oct 18 14:13:56 2014 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='admin' --slave_host=192.168.126.130 --slave_ip=192.168.126.130 --slave_port=3307 --workdir=/var/tmp --target_version=10.0.13-MariaDB-log --manager_version=0.56 --relay_log_info=/home/mysql/mydata/data2/relay-log.info  --relay_dir=/home/mysql/mydata/data2/  --slave_pass=xxx
Sat Oct 18 14:13:56 2014 - [info]   Connecting to root@192.168.126.130(192.168.126.130:22)..
Checking slave recovery environment settings..
Opening /home/mysql/mydata/data2/relay-log.info ... ok.
Relay log found at /home/mysql/mydata/data2/relay_log, up to mysql-relay-bin.000002
Temporary relay log file is /home/mysql/mydata/data2/relay_log/mysql-relay-bin.000002
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Sat Oct 18 14:14:21 2014 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='admin' --slave_host=192.168.126.130 --slave_ip=192.168.126.130 --slave_port=3308 --workdir=/var/tmp --target_version=10.0.13-MariaDB-log --manager_version=0.56 --relay_log_info=/home/mysql/mydata/data3/relay-log.info  --relay_dir=/home/mysql/mydata/data3/  --slave_pass=xxx
Sat Oct 18 14:14:21 2014 - [info]   Connecting to root@192.168.126.130(192.168.126.130:22)..
Checking slave recovery environment settings..
Opening /home/mysql/mydata/data3/relay-log.info ... ok.
Relay log found at /home/mysql/mydata/data3/relay_log, up to mysql-relay-bin.000002
Temporary relay log file is /home/mysql/mydata/data3/relay_log/mysql-relay-bin.000002
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Sat Oct 18 14:14:47 2014 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='admin' --slave_host=192.168.126.129 --slave_ip=192.168.126.129 --slave_port=3307 --workdir=/var/tmp --target_version=10.0.13-MariaDB-log --manager_version=0.56 --relay_log_info=/home/mysql/mydata/data2/relay-log.info  --relay_dir=/home/mysql/mydata/data2/  --slave_pass=xxx
Sat Oct 18 14:14:47 2014 - [info]   Connecting to root@192.168.126.129(192.168.126.129:22)..
Checking slave recovery environment settings..
Opening /home/mysql/mydata/data2/relay-log.info ... ok.
Relay log found at /home/mysql/mydata/data2/relay_log, up to mysql-relay-bin.000002
Temporary relay log file is /home/mysql/mydata/data2/relay_log/mysql-relay-bin.000002
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Sat Oct 18 14:14:58 2014 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='admin' --slave_host=192.168.126.129 --slave_ip=192.168.126.129 --slave_port=3308 --workdir=/var/tmp --target_version=10.0.13-MariaDB-log --manager_version=0.56 --relay_log_info=/home/mysql/mydata/data3/relay-log.info  --relay_dir=/home/mysql/mydata/data3/  --slave_pass=xxx
Sat Oct 18 14:14:58 2014 - [info]   Connecting to root@192.168.126.129(192.168.126.129:22)..
Checking slave recovery environment settings..
Opening /home/mysql/mydata/data3/relay-log.info ... ok.
Relay log found at /home/mysql/mydata/data3/relay_log, up to mysql-relay-bin.000002
Temporary relay log file is /home/mysql/mydata/data3/relay_log/mysql-relay-bin.000002
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Sat Oct 18 14:15:10 2014 - [info] Slaves settings check done.
Sat Oct 18 14:15:10 2014 - [info]
192.168.126.129(192.168.126.129:3306) (current master)
+--192.168.126.130(192.168.126.130:3306)
+--192.168.126.130(192.168.126.130:3307)
+--192.168.126.130(192.168.126.130:3308)
+--192.168.126.129(192.168.126.129:3307)
+--192.168.126.129(192.168.126.129:3308)
Sat Oct 18 14:15:10 2014 - [info] Checking replication health on 192.168.126.130..
Sat Oct 18 14:15:10 2014 - [info]  ok.
Sat Oct 18 14:15:10 2014 - [info] Checking replication health on 192.168.126.130..
Sat Oct 18 14:15:10 2014 - [info]  ok.
Sat Oct 18 14:15:10 2014 - [info] Checking replication health on 192.168.126.130..
Sat Oct 18 14:15:10 2014 - [info]  ok.
Sat Oct 18 14:15:10 2014 - [info] Checking replication health on 192.168.126.129..
Sat Oct 18 14:15:10 2014 - [info]  ok.
Sat Oct 18 14:15:10 2014 - [info] Checking replication health on 192.168.126.129..
Sat Oct 18 14:15:10 2014 - [info]  ok.
Sat Oct 18 14:15:10 2014 - [warning] master_ip_failover_script is not defined.
Sat Oct 18 14:15:10 2014 - [warning] shutdown_script is not defined.
Sat Oct 18 14:15:10 2014 - [info] Got exit code 0 (Not master dead).
MySQL Replication Health is OK.
　　这样就OK了。
　　7 开启masterha_manager



# nohup masterha_manager --conf=/etc/masterha/app1.cnf > /tmp/mha_manager.log  2>&1 &
　　看一下状态：



# masterha_check_status --conf=/etc/masterha/app1.cnf
app1 (pid:5161) is running(0:PING_OK), master:192.168.126.129
　　这个时候也可以检测一下日志，可以新开一个terminal：
　　tail -f /var/log/masterha/app1/manager.log
　　8 测试一下断掉129的3306以后能不能切换到130的3306：



[iyunv@bogon ~]# mysqld_multi --defaults-extra-file=/etc/my.cnf stop 1
　　但是这之前确保没有这个文件 /var/log/masterha/app1/app1.failover.complete
　　关掉实例之后就会发现刚才tail的日志不停的刷新，摘Failover Report出来：
　　



----- Failover Report -----
app1: MySQL Master failover 192.168.126.129(192.168.126.129:3306) to 192.168.126.130(192.168.126.130:3306) succeeded
Master 192.168.126.129(192.168.126.129:3306) is down!
Check MHA Manager logs at bogon:/var/log/masterha/app1/manager.log for details.
Started automated(non-interactive) failover.
The latest slave 192.168.126.130(192.168.126.130:3306) has all relay logs for recovery.
Selected 192.168.126.130(192.168.126.130:3306) as a new master.
192.168.126.130(192.168.126.130:3306): OK: Applying all logs succeeded.
192.168.126.130(192.168.126.130:3307): This host has the latest relay log events.
192.168.126.130(192.168.126.130:3308): This host has the latest relay log events.
192.168.126.129(192.168.126.129:3307): This host has the latest relay log events.
192.168.126.129(192.168.126.129:3308): This host has the latest relay log events.
Generating relay diff files from the latest slave succeeded.
192.168.126.130(192.168.126.130:3308): OK: Applying all logs succeeded. Slave started, replicating from 192.168.126.130(192.168.126.130:3306)
192.168.126.130(192.168.126.130:3307): OK: Applying all logs succeeded. Slave started, replicating from 192.168.126.130(192.168.126.130:3306)
192.168.126.129(192.168.126.129:3307): OK: Applying all logs succeeded. Slave started, replicating from 192.168.126.130(192.168.126.130:3306)
192.168.126.129(192.168.126.129:3308): OK: Applying all logs succeeded. Slave started, replicating from 192.168.126.130(192.168.126.130:3306)
192.168.126.130(192.168.126.130:3306): Resetting slave info succeeded.
Master failover to 192.168.126.130(192.168.126.130:3306) completed successfully.
　　9 下面看看是不是已经切过来了，登陆到130的3306上看一下processlist：

　　确实是四个，再看看129的3307的slave status：

　　已经切换到了130的3306上。
　　再看看130的3307上：

　　这个配置起来不是很复杂，我觉得基本所有的开源项目安装配置都不难，关键都是在参数的配置上，下面是我参考的文档，有自己的也有别人的：
　　http://ylw6006.blog.iyunv.com/470441/890360/
　　http://www.iyunv.com/LINUXjishu/128753.html
　　http://www.iyunv.com/wingsless/p/4002806.html
　　http://www.iyunv.com/wingsless/p/4002806.html