论坛 › Linux运维 › Cloud in Action: Learn and Practice Chef on Ubuntu

qmya00 发表于 2019-2-19 06:37:48

Cloud in Action: Learn and Practice Chef on Ubuntu

　　xueguofeng2011@gmail.com

　　
　　I heard Chef many times but never took it seriously. Recently I’m learning AWS OpsWorks -the hosted services to use Chef cookbooks and solutions for configration management and automation, and fully stuck by Chef this time. I didsome research on Internet and was confused by many terminologies used by Chef, such as kitchen, cookbook, recipe and knife etc, which makes me a little bit distracted. For most of blogs and articles on Internet, they are helpful to review Chef instead of learning it at the first time, I was anxious with learning the new stuff and facing a steep learning curve. After investing the whole two days on learning Ruby – another OOP and explanation language like Python, installing and playing with Chef on my virtual lab environment, finally I was able to understand the core concepts of Chef and move forward with AWS OpsWorks. Learning unknowns is always a painful and sweet journey !

　　
　　Let’s review the core concepts regarding to Chef before going to the lab and configuration:
　　
　　Chef, Puppet, Ansible and SaltStack present different paths to achieve a common goal of managing vast numbers of computers efficiently, with minimal input from developers and sysadmins. Chef is one of the most popular confiugration management tools, and it uses Ruby as its reference language to define patterns in recipes and cookbooks to configure, deploy and manage nodes across the network. Chef provides several management mode:
　　
　　1)      Client-Server. The main management mode which contains three components: A Chef Client is an agent that runs locally on every node managed by Chef, it performs the following steps to bring the node into the expected state: registering and authenticating the node with Chef Server, synchronizing cookbooks and configuring the node. The Chef Server acts as a hub for configuration data to store the cookbooks and policies that are applied to nodes, and metadata that describes each registered node that is being managed by Chef Client. Chef Development Kit on Chef Workstation is a package that contains everything needed to start using Chef, such as Chef Client, Chef and Knife CLI and test tools, to build and test cookbooks and interact with Chef Server.
http://s1.运维网.com/images/20180828/1535401299912603.png
http://s1.运维网.com/images/20180828/1535401317580022.png
　　2)      Chef-Solo. Doesn’t require the Chef Server and doesn’t support the following functionality by the Client-Server mode either: centralized distribution of cookbooks, a centralized API Gateway that interacts with and integrates infrasture components, and authentication or authorization. All the managed network nodes are only installed with the Chef Client software and use the local mode of Chef-Client to run the recipes.

　　
　　3)      Hosted Chef - the cloud-based SaaS. For example, AWS OpsWorks is a configuration management service that helps you configure and operate applications in a hybrid cloud environment by using Puppet or Chef. AWS OpsWorks for Chef Automate lets you create AWS-managed Chef servers that include Chef Automate premium features, and use the Chef DK and other Chef tooling to manage them. AWS OpsWorks Stacks provides a simple and flexible way to create and manage stacks and applications, and run s the Chef recipes using Chef-Solo to handle tasks such as installing packages on instances, deploying apps, and running scripts. It supports both AWS EC2 instances and on-premises servers that are installed with the OpsWorks Stack Agent and have connection to AWS public endpoints.
　　
　　Cookbooks are the configuration units allowing us to configure and perform specific tasks within Chef on the remote nodes and bring a node into a specific state. Cookbooks are created on the workstation and then uploaded to a Chef Server. From there, recipes and policies described within the cookbook can be assigned to nodes as part of the node's "run-list". A run-list is a sequential list of recipes and roles that are run on a node by chef-client in order to bring the node into compliance with the policy you set for it.
http://s1.运维网.com/images/20180828/1535401382385863.png
http://s1.运维网.com/images/20180828/1535401397422435.png

　　A recipe is the main workhorse of the cookbook. A cookbook can contain more than one recipe, or depend on outside recipes. Recipes are used to declare the state of different resources. A recipe is a list related resources that tell Chef how the system should look if it implements the recipe. When Chef runs the recipe, it checks each resource for compliance to the declared state. If the system matches, it moves on to the next resource; otherwise, it attempts to move the resource into the given state.
http://s1.运维网.com/images/20180828/1535401421679874.png

　　The chef-client will run a recipe only when asked. When the chef-client runs the same recipe more than once, the results will be the same system state each time. When a recipe is run against a system, but nothing has changed on either the system or in the recipe, the chef-client won’t change anything.

　　
　　For the Client-Server management mode, 4 VMs have been created by me to practice the soltuion. Below are the configurations:
　　

　　
　　Chef Server
　　Chef Workstation
　　Node1
　　Node2
// Upgrade the system – Ubuntu 16.04 LTS
sudo apt-get update
sudo apt-get upgrade
　　sudo apt-get dist-upgrade
　　
　　// Change the interace name to eth0
sudo gedit /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX="net.ifnames=0   biosdevname=0"
sudo update-grub
　　sudo grub-mkconfig -o   /boot/grub/grub.cfg
　　
// Install openssh & git
sudo apt-get install openssh-server
sudo apt-get install openssh-client
sudo apt-get install sysv-rc-conf
sudo sysv-rc-conf ssh on
sudo sysv-rc-conf --list | grep ssh
netstat -ta | grep ssh
　　
　　sudo apt-get install   git-core
// Configue interface, IP addr, hostname and DNS
sudo gedit   /etc/network/interfaces
auto eth0
iface eth0 inet static
address 10.0.0.51
netmask 255.255.255.0
gateway 10.0.0.2
dns-nameserver 10.0.1.1
sudo gedit   /etc/network/interfaces
auto eth0
iface eth0 inet static
address 10.0.0.52
netmask 255.255.255.0
gateway 10.0.0.2
　　dns-nameserver 10.0.1.1
sudo gedit   /etc/network/interfaces
auto eth0
iface eth0 inet static
address 10.0.0.61
netmask 255.255.255.0
gateway 10.0.0.2
　　dns-nameserver 10.0.1.1
sudo gedit   /etc/network/interfaces
auto eth0
iface eth0 inet static
address 10.0.0.62
netmask 255.255.255.0
gateway 10.0.0.2
dns-nameserver   10.0.1.1
　　// Configure hostname
sudo gedit   /etc/hostname
server
sudo gedit   /etc/hostname
workstation
sudo gedit   /etc/hostname
node1
sudo gedit /etc/hostname
node2
// Configure the hosts file
　　sudo   gedt /etc/hosts
　　127.0.0.1            localhost
　　127.0.1.1            server.xgf.com server
　　sudo   gedt /etc/hosts
　　127.0.0.1            localhost
127.0.1.1            workstation.xgf.com   workstation
　　sudo   gedt /etc/hosts
　　127.0.0.1                localhost
127.0.1.1                node1.xgf.com node1
　　sudo   gedt /etc/hosts
　　127.0.0.1                localhost
127.0.1.1                node2.xgf.com node2
　　10.0.0.51       server.xgf.com server
　　10.0.0.52       workstation.xgf.com workstation
　　10.0.0.61       node1.xgf.com node1
10.0.0.62         node2.xgf.com node2
　　// Install the   Chef Server
　　
　　wget https://packages.chef.io/files/stable/chef-server/12.17.33/ubuntu/16.04/chef-server-core_12.17.33-1_amd64.deb
　　
　　sudo dpkg -i   chef-server-core_12.17.33-1_amd64.deb
　　
　　sudo chef-server-ctl   reconfigure
　　
　　// Create a   Chef User and Organization
　　
　　mkdir ~/.chef
　　
　　// Create an user
　　sudo chef-server-ctl   user-create gset guofeng xue xueguofeng2011@gmail.com 'ipcc2014' --filename ~/.chef/gset.pem
　　
　　// Create an organization and   add the user to the admins
　　sudo chef-server-ctl   org-create california "California, USA" --association_user gset   --filename ~/.chef/california.pem
　　
　　gset@server:~/.chef$ ls -l
　　total 8
　　-rw-r--r-- 1 root root 1674   Aug 20 17:42 california.pem
　　-rw-r--r-- 1 root root 1674   Aug 20 17:40 gset.pem
　　// Install   ChefDK on Chef Workstation
　　
　　wget   https://packages.chef.io/files/stable/chefdk/3.1.0/ubuntu/16.04/chefdk_3.1.0-1_amd64.deb
　　
　　sudo dpkg -i   chefdk_3.1.0-1_amd64.deb
　　
　　
　　
　　
　　


　　// Install the   GUI - Chef Manage
　　
　　wget   https://packages.chef.io/files/stable/chef-manage/2.5.16/ubuntu/16.04/chef-manage_2.5.16-1_amd64.deb   
　　
　　sudo   dpkg -i chef-manage_2.5.16-1_amd64.deb
　　
　　sudo   chef-manage-ctl reconfigure
　　
　　//   Visit the GUI
　　
　　https://server.xgf.com/organizations/california/nodes
　　https://server.xgf.com/organizations/california/cookbooks



　　Chef Workstation
　　Node1
　　Node2
　　// Generate the   chef-repo and move into it
　　gset@workstation:~$ chef generate   app chef-repo
　　gset@workstation:~$ cd chef-repo
　　
　　// Copy the RSA   private keys generated when setting up the Chef Server to the workstation
　　gset@workstation:~/chef-repo$ mkdir .chef
　　gset@workstation:~/chef-repo$
　　scp   gset@10.0.0.51:~/.chef/*.pem ~/chef-repo/.chef/
　　
　　gset@workstation:~/chef-repo/.chef$   ls -l
　　total   16
　　-rw-r--r--   1 gset gset 1674 Aug 20 17:47 california.pem
　　-rw-r--r--   1 gset gset 1674 Aug 20 17:47 gset.pem
　　-rw-rw-r--   1 gset gset496 Aug 20 18:02 knife.rb
　　drwxrwxr-x   2 gset gset 4096 Aug 20 18:05 trusted_certs


　　// Add the   version control with Github
　　gset@workstation:~/chef-repo$ git config   --global user.name xueguofeng
　　gset@workstation:~/chef-repo$ git config   --global user.email xueguofeng2011@gmail.com
　　gset@workstation:~/chef-repo$ echo   ".chef" > .gitignore
　　gset@workstation:~/chef-repo$ git add .
　　gset@workstation:~/chef-repo$ git commit -m   "initial commit"
　　gset@workstation:~/chef-repo$git status
　　On   branch master
　　nothing   to commit, working directory clean


　　// Generate   knife.rb – the knife configuration file
　　gset@workstation:~/chef-repo$ cd .chef
　　gset@workstation:~/chef-repo/.chef$ gedit knife.rb
　　current_dir =   File.dirname(__FILE__)
　　log_level                :info
　　log_location             STDOUT
　　node_name                'gset'
　　client_key               "gset.pem"
　　validation_client_name   'california-validator'
　　validation_key         "california-validator.pem"
　　chef_server_url          'https://server.xgf.com/organizations/california'
　　cache_type               'BasicFile'
　　cache_options(   :path => "#{ENV['HOME']}/.chef/checksums" )
　　cookbook_path            ["#{current_dir}/../cookbooks"]
　　
　　// Copy the   needed SSL certificates from the Chef Server
　　gset@workstation:~/chef-repo/.chef$ cd ..
　　gset@workstation:~/chef-repo$ knife ssl fetch
　　gset@workstation:~/chef-repo$ knife client   list


　　// Bootstrap a   node - install the Chef Client and validate the node
　　gset@workstation:~/chef-repo/.chef$ knife bootstrap 10.0.0.61 -x gset -P ipcc2014 --sudo   --node-name node1
　　gset@workstation:~/chef-repo/.chef$ knife   bootstrap 10.0.0.62 -x gset -P ipcc2014 --sudo --node-name node2
　　gset@workstation:~/chef-repo$ knife client   list
　　gset@workstation:~/chef-repo$ knife node   list
　　
　　gset@workstation:~/chef-repo$   knife client list
　　california-validator
　　node1
　　node2
　　
　　gset@workstation:~/chef-repo$   knife node list
　　node1
　　node2


　　// Create a   cookbook – “nginx”
　　gset@workstation:~/chef-repo/cookbooks$ chef generate   cookbook nginx
　　
　　gset@workstation:~/chef-repo/cookbooks/nginx$   ls -l
　　total   40
　　-rw-rw-r--   1 gset gset   77 Aug 20 18:29 Berksfile
　　-rw-rw-r--   1 gset gset148 Aug 20 18:29   CHANGELOG.md
　　-rw-rw-r--   1 gset gset 1090 Aug 20 18:29 chefignore
　　drwxrwxr-x   3 gset gset 4096 Aug 20 18:34 files
　　-rw-rw-r--   1 gset gset   70 Aug 20 18:29 LICENSE
　　-rw-rw-r--   1 gset gset748 Aug 20 18:29   metadata.rb
　　-rw-rw-r--   1 gset gset   53 Aug 20 18:29 README.md
　　drwxrwxr-x   2 gset gset 4096 Aug 20 21:37 recipes
　　drwxrwxr-x   3 gset gset 4096 Aug 20 18:29 spec
　　drwxrwxr-x   3 gset gset 4096 Aug 20 18:29 test
　　
　　// Create the   default recipe – “default.rb”, which will install and run Nginx, and transfer   the html file into Nginx’s root
　　gset@workstation:~/chef-repo/cookbooks/nginx/recipes$ gedit   default.rb
　　package 'nginx'   do
　　action :install
　　end
　　service 'nginx'   do
　　action [ :enable, :start ]
　　end
　　cookbook_file   "/var/www/html/index.html" do
　　source "index.html"
　　mode "0644"
　　end
　　
　　// Make the   index.html file
　　gset@workstation:~/chef-repo/cookbooks/nginx/files/default$ gedit   index.html
　　
　　
　　    Hello World
　　
　　
　　    Welcome to Chef !
　　       --- Guofeng Xue
　　
　　
　　
　　// Upload the   cookbook – “nginx” to the Chef Server
　　gset@workstation:~/chef-repo$   knife   cookbook upload nginx
　　gset@workstation:~/chef-repo$   knife   cookbook upload –a
　　
　　gset@workstation:~/chef-repo/cookbooks/nginx$   knife cookbook upload -a
　　Uploading   chef-repo      
　　Uploading   learn_chef_httpd
　　Uploading   nginx         
　　Uploading   xgf            
　　Uploaded   all cookbooks.
　　
　　// Modify the   run-list of nodes
　　gset@workstation:~/chef-repo$   export   EDITOR=vi
　　gset@workstation:~/chef-repo$ knife node   edit node1
　　{
　　"name": "node1",
　　"chef_environment":   "_default",
　　"normal": {
　　    "tags": [    ]
　　},
　　"run_list": [    “recipe”]
　　}
　　gset@workstation:~/chef-repo$ knife node   edit node2
　　{
　　"name": "node2",
　　"chef_environment":   "_default",
　　"normal": {
　　    "tags": [    ]
　　},
　　"run_list": [    “recipe”,” recipe”]
　　}
　　
　　gset@workstation:~/chef-repo$   knife node show node1
　　Node   Name:   node1
　　Environment:   _default
　　FQDN:      node1.xgf.com
　　IP:          10.0.0.61
　　Run   List:    recipe, recipe
　　Roles:      
　　Recipes:   nginx, nginx::default, xgf, xgf::default
　　Platform:    ubuntu 16.04
　　Tags:   
　　
　　gset@workstation:~/chef-repo$   knife node show node2
　　Node   Name:   node2
　　Environment:   _default
　　FQDN:      node2.xgf.com
　　IP:          10.0.0.62
　　Run   List:    recipe
　　Roles:      
　　Recipes:   nginx, nginx::default
　　Platform:    ubuntu 16.04
　　Tags:
　　
　　// SSH into the   nodes and run the Chef Client
　　gset@workstation:~/chef-repo$   ssh   gset@10.0.0.61
　　gset@node1:~$   sudo   chef-client
　　gset@workstation:~/chef-repo$   ssh   gset@10.0.0.62
　　gset@node2:~$   sudo   chef-client
　　chef-client –v
　　
　　chef-solo -v
　　chef-client -v
　　
　　chef-solo –v
　　gset@workstation:~/chef-repo$   chef -v
　　Chef   Development Kit Version: 3.1.0
　　chef-client   version: 14.2.0
　　delivery   version: master (6862f27aba89109a9630f0b6c6798efec56b4efe)
　　berks   version: 7.0.4
　　kitchen   version: 1.22.0
　　inspec   version: 2.1.72
　　gset@workstation:~/chef-repo$   chef-client -v
　　Chef:   14.2.0
　　gset@workstation:~/chef-repo$   chef-solo -v
　　Chef:   14.2.0
　　sudo   chef-client
　　sudo   chef-client
http://s1.运维网.com/images/20180828/1535401548652698.png
http://s1.运维网.com/images/20180828/1535401567664629.png

http://s1.运维网.com/images/20180828/1535401584272610.png
http://s1.运维网.com/images/20180828/1535401598995794.png
　　With the Client-Server management mode, we can also use the local mode of Chef Client on the managed nodes:
　　
　　// Create a local recipe
　　gset@node2:~$   cat hello.rb
　　directory   "/home/gset/xgf111" do
　　    action :create
　　end
　　
　　// Run the recipe locally
　　gset@node2:~$   sudo chef-client --local-mode hello.rb
http://s1.运维网.com/images/20180828/1535401632182493.png
　　For the Chef-Solo management mode, below are the configurations:
　　
//   Install the Chef Client with the Chef install script
　　curl -L https://omnitruck.chef.io/install.sh | sudo bash
　　
　　// Create the recipe
　　gset@ubuntu16:~/chef-solo/cookbooks/helloworld/recipes$   cat default.rb
　　directory   "/home/gset/xgf3" do
　　    action :create
　　end
　　file   "/home/gset/xgf3/abc.txt" do
　　    content "Hello World!"
　　end
　　
　　// Build the configure file
　　gset@ubuntu16:~$   cat solo.rb
　　file_cache_path   "/home/gset/chef-solo"
　　cookbook_path   ["/home/gset/chef-solo/cookbooks"]
　　
　　// Define the run_list object
　　gset@ubuntu16:~$   cat node.json
　　{
　　   "run_list": [   "recipe" ]
　　}
　　
　　// Run the recipe with Chef-Sole
　　gset@ubuntu16:~$   chef-solo -c ~/solo.rb -j ~/node.json
http://s1.运维网.com/images/20180828/1535401663630246.png


An Overview of Chef
https://docs.chef.io/chef_overview.html
　　Install a Chef Server Workstation on Ubuntu 18.04
　　https://www.linode.com/docs/applications/configuration-management/install-a-chef-server-workstation-on-ubuntu-18-04/
　　How To Create Simple Chef Cookbooks to Manage Infrastructure on Ubuntu
　　https://www.digitalocean.com/community/tutorials/how-to-create-simple-chef-cookbooks-to-manage-infrastructure-on-ubuntu
　　Nodes - Install via URL
　　https://docs.chef.io/install_omnibus.html
　　Executing Cookbooks with Chef-Solo
　　https://aggarwalarpit.wordpress.com/2015/12/01/executing-cookbooks-with-chef-solo/comment-page-1/

查看完整版本: Cloud in Action: Learn and Practice Chef on Ubuntu