Kubernetes deployed on multiple ubuntu nodes
This document describes how to deploy kubernetes on multiple ubuntu nodes, including 1 master node and 3 minion nodes, and people uses this approach can scale to any number of minion nodes by changing some settings with ease.> Cloud team from ZJU will keep updating this work.Prerequisites：
1 The minion nodes have installed docker version 1.2+
2 All machines can communicate with each orther, no need to connect Internet (should use private docker registry in this case)
3 These guide is tested OK on Ubuntu 14.04 LTS 64bit server, but it should also work on most Ubuntu versions
4 Dependences of this guide: etcd-2.0.0, flannel-0.2.0, k8s-0.12.0, but it may work with higher versions
Main Steps
I. Make kubernetes , etcd and flanneld binaries
On your laptop, copy cluster/ubuntu-cluster directory to your workspace.
The build.sh will download and build all the needed binaries into ./binaries.
You can customize your etcd version or K8s version in the build.sh by changing variable ETCD_V and K8S_V in build.sh, default etcd version is 2.0.0 and K8s version is 0.12.0.
$ cd cluster/ubuntu-cluster
$ sudo ./build.sh
Please copy all the files in ./binaries into /opt/bin of every machine you want to run as Kubernetes cluster node.
Alternatively, if your Kubernetes nodes have access to Internet, you can copy cluster/ubuntu-cluster directory to every node and run:
# in every node
$ cd cluster/ubuntu-cluster
$ sudo ./build.sh
$ sudo cp ./binaries/* /opt/bin
We used flannel here because we want to use overlay network, but please remember it is not the only choice, and it is also not a k8s' necessary dependence. Actually you can just build up k8s cluster natively, or use flannel, Open vSwitch or any other SDN tool you like, we just choose flannel here as a example.
II. Configue and install every components upstart script
An example cluster is listed as below:
IP AddressRole10.10.103.223
minion
10.10.103.224
minion
10.10.103.162
minion
10.10.103.250
master First of all, make sure cluster/ubuntu-cluster exists on this node，and run configue.sh.
On master( infra1 10.10.103.250 ) node:
# in cluster/ubuntu-cluster
$ sudo ./configure.sh
Welcome to use this script to configure k8s setup
Please enter all your cluster node ips, MASTER node comes first
And separated with blank space like "<ip_1> <ip2> <ip3>": 10.10.103.250 10.10.103.223 10.10.103.224 10.10.103.162
This machine acts as
both MASTER and MINION: 1
only MASTER: 2
only MINION: 3
Please choose a role > 2
IP address of this machine > 10.10.103.250
Configure Success
On every minion ( e.g. 10.10.103.224 ) node:
# in cluster/ubuntu-cluster
$ sudo ./configure.sh
Welcome to use this script to configure k8s setup
Please enter all your cluster node ips, MASTER node comes first
And separated with blank space like "<ip_1> <ip2> <ip3>": 10.10.103.250 10.10.103.223 10.10.103.224 10.10.103.162
This machine acts as
both MASTER and MINION: 1
only MASTER: 2
only MINION: 3
Please choose a role > 3
IP address of this machine > 10.10.103.224
Configure Success
If you want a node acts as both running the master and minion, please choose option 1.
III. Start all components
[*] On the master node:
$ sudo service etcd start
Then on every minion node:
$ sudo service etcd start
The kubernetes commands will be started automatically after etcd
[*] On any node:
$ /opt/bin/etcdctl mk /coreos.com/network/config '{"Network":"10.0.0.0/16"}'
You can use the below command on another node to comfirm if the network setting is correct.
$ /opt/bin/etcdctl get /coreos.com/network/config
If you got {"Network":"10.0.0.0/16"}, then etcd cluster is working well. If not , please check/var/log/upstart/etcd.log to resolve etcd problem before going forward. Finally, use ifconfig to see if there is a new network interface named flannel0 coming up.
[*] On every minion node
Make sure you have brctl installed on every minion, otherwise please run sudo apt-get install bridge-utils
$ sudo ./reconfigureDocker.sh
This will make the docker daemon aware of flannel network.
All done !
IV. Validation
You can use kubectl command to see if the newly created k8s is working correctly.
For example , $ kubectl get minions to see if you get all your minion nodes comming up.
Also you can run kubernetes guest-example to build a redis backend cluster on the k8s．
V. Trouble Shooting
Generally, what of this guide did is quite simple:
[*] Build and copy binaries and configuration files to proper dirctories on every node
[*] Configure etcd using IPs based on input from user
[*] Create and start flannel network
So, whenver you have problem, do not blame Kubernetes, check etcd configuration first
Please try:
[*] Check /var/log/upstart/etcd.log for suspicisous etcd log
[*] Check /etc/default/etcd, as we do not have much input validation, a right config should be like:
ETCD_OPTS="-name infra1 -initial-advertise-peer-urls <http://ip_of_this_node:2380> -listen-peer-urls <http://ip_of_this_node:2380> -initial-cluster-token etcd-cluster-1 -initial-cluster infra1=<http://ip_of_this_node:2380>,infra2=<http://ip_of_another_node:2380>,infra3=<http://ip_of_another_node:2380> -initial-cluster-state new"
[*] Remove data-dir of etcd and run reconfigureDocker.shagain, the default path of data-dir is /infra*.etcd/
[*] You can also customize your own settings in /etc/default/{component_name} after configured success.
页:
[1]