运维自动化之svn+puppet实现监控系统的版本控制与自动部署监控系统
本帖最后由 786yr 于 2014-7-22 10:24 编辑在描述之前，先把流程图给大家，希望能使大家清楚流程：
整体流程如下：
1、运维人员在修改监控系统的时候，需要先登录svn服务器里修改监控的配置；
2、在svn里修改完监控的配置，选择更新的时候，会触发svn的hook里post-commit里的内容，根据脚本里的内容，会在puppet master服务器里的/usr/local/monitor/shell目录里留下最新svn的副本；
3、由于puppet client设置的连接puppet master的时间间隔为300秒，所以在300秒后能发现puppet client会从puppet master获得最新的配置，并更新自己本机的配置。
这样一个自动化的监控系统部署更新就完成了。
下面是是svn与puppet部分的详细说明
一、svn部分
安装svn主要是为了进行监控脚本的版本控制，如果不进行版本控制的话，比如我要修改甘肃web的配置的某个部分配置，过几天又修改了这个配置，在过2周呢？我还能记住之前修改什么了吗?我估计我是记不住了，而且我负责的平台过多，直接维护的服务器超过100+，平台更新的次数很频繁，而且根据用户的需求，修改的内容也不尽相同，所以进行版本控制是十分有必要了，废话不说，现在开始svn的配置。
svn服务端的安装我就不再说明
我的svn主要是为了配置我之前发布是“运维自动化之使用PHP+MYSQL+SHELL打造私有监控系统”这篇博文的shell脚本部分与下面的puppet配置部分，使用svn的hook（post-commit）功能，在svn更新的时候，就出发钩子，然后更新到我知道的目录，之后puppet会从这个里取得配置配置，在设置puppet的客户端连接服务端获取连接为300秒，也就是5分钟，这样的话，只有我一更新我的svn，那么puppet的客户端就会爱5分钟的时候，更新配置，减少人工操作，还能版本控制。
下面是我的svn里hook的post-commit的配置
REPOS="$1"
REV="$2"
export LANG=zh_CN.UTF-8
exec /usr/bin/svn update /usr/local/monitor/shell --username *** --password ** --no-auth-cache
echo "please check client config after in the 300s"|/bin/mail -s "svn has been update" denglei@ctfo.com
现在展示一下我的svn界面与内容
在看看我的更新记录
这个svn由于是新建立的，所以更新的日志比较少。
二、puppet部分
为了安装最新的puppe，如果您想使用yum安装的话，需要在主机上添加epel库
[*]rpm -Uvh http://download.fedora.redhat.co ... ease-5-4.noarch.rpm
如果此源失效，可以使用
[*]wget http://dl.fedoraproject.org/pub/ ... ease-5-4.noarch.rpm
[*]rpm -ivh epel-release-5-4.noarch.rpm
下面开始安装puppet服务端与客户端
1、在服务端
（1）使用yum安装puppet
[*]yum -y install puppet-server openssl*
（2）开启puppet服务端的服务
[*]service puppetmaster start
（3）添加到开机启动
[*]chkconfig --level 345 puppet on
[*]chkconfig --level 345 puppetmaster on
（4）在hosts里添加包含本身以及需要自动化的客户端机器主机名与ip地址
（5）接下来需要去客户端安装puppet、启动服务与进行认证了
2、在客户端
[*]yum -y install puppet openssl*
[*]service puppet start
[*]chkconfig --level 345 puppet on
然后在与服务端进行认证
[*]puppetd --server master --test
3、接下来在服务端进行允许认证
[*]puppetca -s -a
这样就完成了认证，之后就可以在puppet的client里进行获取服务端的配置，并自动的在本机更新了。
下面是我的配置，提供给大家参考（我已经贵州的puppet为例）
贵州的puppet结构为：
贵州备用为puppetmaster服务端
贵州web、贵州引擎、贵州引擎、贵州备用为客户端（主要贵州备用既为服务端与客户端是没有问题）
先看一下我的puppet服务端贵州备用的文件结构
[*]# tree /etc/puppet/
[*]/etc/puppet/
[*]|-- auth.conf
[*]|-- fileserver.conf #puppet文件服务器配置文件
[*]|-- manifests #puppet主文件所在目录
[*]| |-- guizhou_beiyong.pp #该模块对应的文件资源，可能是要发送给slave的配置文件等
[*]| |-- guizhou_jiaohuan.pp
[*]| |-- guizhou_web.pp
[*]| |-- guizhou_yinqing.pp
[*]| `-- site.pp #puppet主文件（入口文件）
[*]`-- puppet.conf
[*]
[*]1 directory, 8 files
客户端连接的顺序：site.pp=》*.pp
比如贵州web的接收顺序为：site.pp=》guizhou_web.pp
首先，slave向发起master连接请求，进行证书验证；
接着，证书验证通过后，master会直接找到入口文件manifests目录下的site.pp文件，该文件可能包含一些全局变量，参数缺省值（当各个模块没有设置这些参数时，它们的缺省值）以及其它pp文件的调用（在该例子中，会调用manifests下的各个pp文件）；
然后，master通过manifests下的各个pp文件定位到该slave要执行的模块（site.pp是各个模块的入口），汇总这些模块代码返回给slave；
最后，slave根据master发过来的manifest，配置信息。
注意：在配置之前一定要求的服务端与客户端的时间一致，并且hosts里都加入了对方的ip与主机名，并且能互相ping通，最好在改好hosts文件后重启服务器，同时selinux与iptables关闭。
4、先看一下我fileserver的设置
[*]# cat /etc/puppet/fileserver.conf
[*]# This file consists of arbitrarily named sections/modules
[*]# defining where files are served from and to whom
[*]
[*]# Define a section 'files'
[*]# Adapt the allow/deny settings to your needs. Order
[*]# for allow/deny does not matter, allow always takes precedence
[*]# over deny
[*]#
[*]#path /var/lib/puppet/files
[*]#allow *.example.com
[*]#deny *.evil.example.com
[*]#allow 192.168.0.0/24
[*]
[*]path /usr/local/monitor/shell
[*]allow 172.16.6.0/24
这个path就是puppet master的配置文件里从哪里获得需要给客户端传输程序的地址，同时也是svn的hook里post-commit更新的地址。
allow后面的网段是我服务器网卡的ip段，也就是需要我这个网段的ip能从这个path里或者程序。
5、贵州web的配置
[*]# cat /etc/puppet/manifests/guizhou_web.pp
[*]node web {
[*] file { "/usr/local/monitor/shell/GuiZhou_alter80.sh":
[*] source=>"puppet:///puppet/GuiZhou_web/GuiZhou_web_alter80.sh",
[*] mode=>755,
[*]}
[*] file { "/usr/local/monitor/shell/GuiZhou_alter90.sh":
[*] source=>"puppet:///puppet/GuiZhou_web/GuiZhou_web_alter90.sh",
[*] mode=>755,
[*]}
[*] file { "/usr/local/monitor/shell/GuiZhou_web_check.sh":
[*] source=>"puppet:///puppet/GuiZhou_web/GuiZhou_web_check.sh",
[*] mode=>755,
[*]}
[*] file { "/usr/local/monitor/shell/web_tomcat.sh":
[*] source=>"puppet:///puppet/GuiZhou_web/web_tomcat.sh",
[*] mode=>755,
[*] owner=>"lbs",
[*] group=>"lbs",
[*]}
[*] file { "/usr/local/monitor/shell/web_yz_tomcat.sh":
[*] source=>"puppet:///puppet/GuiZhou_web/web_yz_tomcat.sh",
[*] mode=>755,
[*] owner=>"lbs",
[*] group=>"lbs",
[*]}
[*] cron {GuiZhou_web_alert80_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_web_alter80.sh",
[*] hour=>'*/6',
[*] minute=>0,
[*]}
[*] cron {GuiZhou_web_alert90_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_web_alter90.sh",
[*] hour=>'*/1',
[*] minute=>0,
[*]}
[*] cron {GuiZhou_web_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_web_check.sh",
[*] minute=>'*/3',
[*]}
[*] cron {GuiZhou_web_sendmail_restart:
[*] command=>"/etc/init.d/sendmail restart",
[*] hour=>'23',
[*] minute=>'00',
[*]}
[*]}
6、贵州交换的配置
[*]# cat /etc/puppet/manifests/guizhou_jiaohuan.pp
[*]node jiaohuan {
[*] file { "/usr/local/monitor/shell/GuiZhou_jiaohuan_alter80.sh":
[*] source=>"puppet:///puppet/GuiZhou_jiaohuan/GuiZhou_jiaohuan_alter80.sh",
[*] mode=>755,
[*]}
[*] file { "/usr/local/monitor/shell/GuiZhou_jiaohuan_alter90.sh":
[*] source=>"puppet:///puppet/GuiZhou_jiaohuan/GuiZhou_jiaohuan_alter90.sh",
[*] mode=>755,
[*]}
[*] file { "/usr/local/monitor/shell/GuiZhou_jiaohuan_check.sh":
[*] source=>"puppet:///puppet/GuiZhou_jiaohuan/GuiZhou_jiaohuan_check.sh",
[*] mode=>755,
[*]}
[*] file { "/usr/local/monitor/shell/GuiZhou_jiaohuan_ping.sh":
[*] source=>"puppet:///puppet/GuiZhou_jiaohuan/GuiZhou_jiaohuan_ping.sh",
[*] mode=>755,
[*]}
[*] cron {GuiZhou_jiaohuan_alert80_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_jiaohuan_alter80.sh",
[*] hour=>'*/6',
[*] minute=>0,
[*]}
[*] cron {GuiZhou_jiaohuan_alert90_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_jiaohuan_alter90.sh",
[*] hour=>'*/1',
[*] minute=>0,
[*]}
[*] cron {GuiZhou_jiaohuan_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_jiaohuan_check.sh",
[*] minute=>'*/3',
[*]}
[*] cron {GuiZhou_jiaohuan_ping:
[*] command=>"/usr/local/monitor/shell/GuiZhou_jiaohuan_ping.sh",
[*] minute=>'*/15',
[*]}
[*] cron {GuiZhou_jiaohuan_sendmail_restart:
[*] command=>"/etc/init.d/sendmail restart",
[*] hour=>'23',
[*] minute=>'00',
[*]}
[*]}
7、贵州引擎的配置
[*]# cat /etc/puppet/manifests/guizhou_yinqing.pp
[*]node savecenter {
[*] file { "/usr/local/monitor/shell/GuiZhou_yinqing_alter80.sh":
[*] source=>"puppet:///puppet/GuiZhou_yinqing/GuiZhou_yinqing_alter80.sh",
[*] mode=>755,
[*]}
[*] file { "/usr/local/monitor/shell/GuiZhou_yinqing_alter90.sh":
[*] source=>"puppet:///puppet/GuiZhou_yinqing/GuiZhou_yinqing_alter90.sh",
[*] mode=>755,
[*]}
[*] file { "/usr/local/monitor/shell/GuiZhou_yinqing_check.sh":
[*] source=>"puppet:///puppet/GuiZhou_yinqing/GuiZhou_yinqing_check.sh",
[*] mode=>755,
[*] owner=>lbs,
[*] group=>lbs,
[*]}
[*] cron {GuiZhou_yinqing_alert80_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_yinqing_alter80.sh",
[*] hour=>'*/6',
[*] minute=>0,
[*]}
[*] cron {GuiZhou_yinqing_alert90_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_yinqing_alter90.sh",
[*] hour=>'*/1',
[*] minute=>0,
[*]}
[*] cron {GuiZhou_yinqing_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_yinqing_check.sh",
[*] user=>lbs,
[*] minute=>'*/3',
[*]}
[*] cron {GuiZhou_yinqing_sendmail_restart:
[*] command=>"/etc/init.d/sendmail restart",
[*] hour=>'23',
[*] minute=>'00',
[*]}
[*]}
8、贵州备用的配置
[*]# cat /etc/puppet/manifests/guizhou_beiyong.pp
[*]node beiyong {
[*] file { "/usr/local/monitor/shell/GuiZhou_beiyong_alter80.sh":
[*] source=>"puppet:///puppet/GuiZhou_beiyong/GuiZhou_beiyong_alter80.sh",
[*] mode=>755,
[*]}
[*] file { "/usr/local/monitor/shell/GuiZhou_beiyong_alter90.sh":
[*] source=>"puppet:///puppet/GuiZhou_beiyong/GuiZhou_beiyong_alter90.sh",
[*] mode=>755,
[*]}
[*] file { "/usr/local/monitor/shell/GuiZhou_beiyong_check.sh":
[*] source=>"puppet:///puppet/GuiZhou_beiyong/GuiZhou_beiyong_check.sh",
[*] mode=>755,
[*]}
[*] cron {GuiZhou_beiyong_alert80_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_beiyong_alter80.sh",
[*] hour=>'*/6',
[*] minute=>0,
[*]}
[*] cron {GuiZhou_beiyong_alert90_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_beiyong_alter90.sh",
[*] hour=>'*/1',
[*] minute=>0,
[*]}
[*] cron {GuiZhou_beiyong_check:
[*] command=>"/usr/local/monitor/shell/GuiZhou_beiyong_check.sh",
[*] minute=>'*/3',
[*]}
[*] cron {GuiZhou_beiyong_sendmail_restart:
[*] command=>"/etc/init.d/sendmail restart",
[*] hour=>'23',
[*] minute=>'00',
[*]}
[*]}
9、在查看一个site.pp的配置
[*]# cat /etc/puppet/manifests/site.pp
[*]import "guizhou_web.pp"
[*]import "guizhou_jiaohuan.pp"
[*]import "guizhou_yinqing.pp"
[*]import "guizhou_beiyong.pp"
现在服务端的配置已经展现完成，在去客户端查看一下配置是否生效
1、贵州web
[*]# crontab -l
[*]# HEADER: This file was autogenerated at Fri Oct 12 15:59:14 +0800 2012 by puppet.
[*]# HEADER: While it can still be managed manually, it is definitely not recommended.
[*]# HEADER: Note particularly that the comments starting with 'Puppet Name' should
[*]# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
[*]# Puppet Name: GuiZhou_web_sendmail_restart
[*]0 23 * * * /etc/init.d/sendmail restart
[*]# Puppet Name: GuiZhou_web_alert90_check
[*]0 */1 * * * /usr/local/monitor/shell/GuiZhou_web_alter90.sh
[*]# Puppet Name: GuiZhou_web_alert80_check
[*]#0 */6 * * * /usr/local/monitor/shell/GuiZhou_web_alter80.sh
[*]# Puppet Name: GuiZhou_web_check
[*]*/3 * * * * /usr/local/monitor/shell/GuiZhou_web_check.sh
2、贵州交换
[*]# crontab -l
[*]# HEADER: This file was autogenerated at Fri Oct 12 16:03:24 +0800 2012 by puppet.
[*]# HEADER: While it can still be managed manually, it is definitely not recommended.
[*]# HEADER: Note particularly that the comments starting with 'Puppet Name' should
[*]# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
[*]# Puppet Name: GuiZhou_jiaohuan_sendmail_restart
[*]0 23 * * * /etc/init.d/sendmail restart
[*]# Puppet Name: GuiZhou_jiaohuan_alert80_check
[*]0 */6 * * * /usr/local/monitor/shell/GuiZhou_jiaohuan_alter80.sh
[*]# Puppet Name: GuiZhou_jiaohuan_alert90_check
[*]0 */1 * * * /usr/local/monitor/shell/GuiZhou_jiaohuan_alter90.sh
[*]# Puppet Name: GuiZhou_jiaohuan_ping
[*]#*/15 * * * * /usr/local/monitor/shell/GuiZhou_jiaohuan_ping.sh
[*]# Puppet Name: GuiZhou_jiaohuan_check
[*]*/3 * * * * /usr/local/monitor/shell/GuiZhou_jiaohuan_check.sh
3、贵州引擎
由于贵州引擎有root账号与lbs用户的配置，所以我们先查看root用户的
[*]# crontab -l
[*]# HEADER: This file was autogenerated at Fri Oct 12 16:16:41 +0800 2012 by puppet.
[*]# HEADER: While it can still be managed manually, it is definitely not recommended.
[*]# HEADER: Note particularly that the comments starting with 'Puppet Name' should
[*]# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
[*]# Puppet Name: GuiZhou_yinqing_sendmail_restart
[*]0 23 * * * /etc/init.d/sendmail restart
[*]# Puppet Name: GuiZhou_yinqing_alert80_check
[*]0 */6 * * * /usr/local/monitor/shell/GuiZhou_yinqing_alter80.sh
[*]# Puppet Name: GuiZhou_yinqing_alert90_check
[*]0 */1 * * * /usr/local/monitor/shell/GuiZhou_yinqing_alter90.sh
在查看lbs用户的
[*]# crontab -u lbs -l
[*]# HEADER: This file was autogenerated at Fri Oct 12 16:16:41 +0800 2012 by puppet.
[*]# HEADER: While it can still be managed manually, it is definitely not recommended.
[*]# HEADER: Note particularly that the comments starting with 'Puppet Name' should
[*]# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
[*]#MIN HOUR DAY MONTH DAYOFWEEK COMMAND
[*]# Puppet Name: GuiZhou_yinqing_check
[*]*/3 * * * * /usr/local/monitor/shell/GuiZhou_yinqing_check.sh
4、贵州备用
[*]# crontab -l
[*]# HEADER: This file was autogenerated at Fri Oct 12 16:03:37 +0800 2012 by puppet.
[*]# HEADER: While it can still be managed manually, it is definitely not recommended.
[*]# HEADER: Note particularly that the comments starting with 'Puppet Name' should
[*]# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
[*]# Puppet Name: GuiZhou_beiyong_check
[*]*/3 * * * * /usr/local/monitor/shell/GuiZhou_beiyong_check.sh
[*]# Puppet Name: GuiZhou_beiyong_sendmail_restart
[*]0 23 * * * /etc/init.d/sendmail restart
[*]# Puppet Name: GuiZhou_beiyong_alert90_check
[*]0 */1 * * * /usr/local/monitor/shell/GuiZhou_beiyong_alter90.sh
[*]# Puppet Name: GuiZhou_beiyong_alert80_check
[*]0 */6 * * * /usr/local/monitor/shell/GuiZhou_beiyong_alter80.sh
现在puppet的配置已经完成，当前的配置已经在我们的生产服务器里部署。
如果大家想要在客户端的puppe设置一个时间来连接puppetmaster获取文件？
可以再客户端里的/etc/puppet/puppet.conf文件里加上：
[*]runinterval =1800
默认是30分钟，可以修改此处的值，单位为秒。
修改后需要重启服务。
需要注意的：
1、在客户端申请证书的时候，建议把客户端与服务器的时间与时区调整为一样的，可以使用ntpdate 210.72.145.44来更新；
2、在/etc/resolv.conf里，一定要把#search localdomain这一行给注销，否则在申请证书的时候，会带有.localdomain，在证书申请完成，服务端给予授权的时候，你在获得资源的时候，会出现hostname not match with the server certificate错误。
在hosts里添加包含本身以及需要自动化的服务端机器主机名与ip地址
3、大家可能发现我写的puppet里的资源配置非常的简单，没有那么花哨、各个模块依赖的，对于这个我的解释是：简单的配置大家都能看懂，运行的时候如果出现问题，那么我们排错的话，也很简单，只有能够顺利的运行，就是一个好的资源配置。
4、主机名使用，建议使用FQDN方式
5、每次生成证书的id均不相同。
页:
[1]