QEMU KVM Libvirt(12): Live Migration
由于KVM的架构为Libvirt –> qemu –> KVM
所以对于live migration有两种方式，一种是qemu + KVM自己的方式，一种是libvirt的方式，当然libvirt也是基于qemu+kvm的方式
qemu + KVM自己的方式为使用monitor
KVM Migration
KVM currently supports savevm/loadvm and offline or live migration Migration commands are given when in qemu-monitor (Alt-Ctrl-2). Upon successful completion, the migrated VM continues to run on the destination host.
Requirements
[*]The VM image is accessible on both source and destination hosts (located on a shared storage, e.g. using nfs).
[*]It is recommended an images-directory would be found on the same path on both hosts (for migrations of a copy-on-write image -- an image created on top of a base-image using "qemu-image create -b ...")
[*]The src and dst hosts must be on the same subnet (keeping guest's network when tap is used).
[*]Do not use -snapshot qemu command line option.
[*]For tcp: migration protocol
the guest on the destination must be started the same way it was started on the source.
The live migration process has the following steps:
[*] The virtual machine instance is running on the source host.
[*] The virtual machine is started on the destination host in the frozen listening mode. The parameters used are the same as on the source host plus the -incoming tcp:ip:port parameter, where ip specifies the IP address and port specifies the port for listening to the incoming migration. If 0 is set as IP address, the virtual machine listens on all interfaces.
[*] On the source host, switch to the monitor console and use the migrate -d tcp:destination_ip:port command to initiate the migration.
[*] To determine the state of the migration, use the info migrate command in the monitor console on the source host.
[*] To cancel the migration, use the migrate_cancel command in the monitor console on the source host.
[*] To set the maximum tolerable downtime for migration in seconds, use the migrate_set_downtime number_of_seconds command.
[*] To set the maximum speed for migration in bytes per second, use the migrate_set_speed bytes_per_second command.
要进行live migration首先要存储共享，我们这里用nfs
在一台nfs server上安装
apt-get install nfs-kernel-server
export如下的文件夹
# cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients.See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homesgss/krb5i(rw,sync,no_subtree_check)
#
/home/cliu8/nfs *(rw,sync,no_root_squash)
在source机器和destination机器上，都mount这个文件夹
mount 16.158.166.150:/home/cliu8/nfs /home/cliu8/migrate
在文件夹里面放入两个image
root@escto-bj-hp-z620:/home/cliu8/migrate# ls -l
total 3842908
-rwxr-xr-x 1 root root 1717567488 Jul 18 00:01 ubuntu-14.04.img
-rwxr-xr-x 1 root root 2217869312 Jul 17 22:13 ubuntutest.img 在source机器上启动
qemu-system-x86_64 -enable-kvm -name ubuntutest-m 2048 -hda /home/cliu8/migrate/ubuntu-14.04.img -vnc :19 -net nic -net user
在dest机器上同样启动，但是多了listen的参数
qemu-system-x86_64 -enable-kvm -name ubuntutest-m 2048 -hda /home/cliu8/migrate/ubuntu-14.04.img -vnc :19 -net nic -net user -incoming tcp:0:4444
打开source机器的monitor
运行migrate -d tcp:16.158.166.150:4444
这个时候info migrate，显示Migration status: active
等变成complete，则migration结束
这个时候，另一面的机器已经起来了。
Libvirt Migration
Network data transports
migration的时候的数据传输有两种方式：
Hypervisor native transport
所谓native的transport，就是依赖于hypervisor，也即KVM的自有的机制去做网络传输，不支持加密，可能对于某些hypervisor来讲，网络还需要特殊的配置。
libvirt tunnelled transport
依赖于libvirt的RPC的网络通路进行网络传输，支持加密。
只要下面的命令能够执行，就能够传输
virsh -c qemu+ssh://cliu8@16.158.166.150/system list --all
virsh -c qemu+tcp://popsuper1982/system list --all
virsh -c qemu+tls://popsuper1982/system list --all 缺点是有很多额外的性能损耗
Communication control paths/flows
看完了数据通路，我们再来看控制通路
Migration一般涉及三个较色，admin, source, destination
Managed direct migration
admin节点控制整个migration的全过程，admin既控制source, 也控制destination，然而source和destination之间并不交互，因而migration过程中，如果admin挂了，就失败了。
Managed peer to peer migration
admin节点仅仅和source交互，告诉source，你要migrate到destination，然后source控制整个过程，如果admin在这个过程中挂了，不影响接着进行migration
注意的是，admin登录source的credential信息和source登录destination的credential信息是不一样的。
Unmanaged direct migration
admin和source都不控制migration的过程，而是admin的libvirt直接调用hypervisor的控制器，让hypervisor自己进行migration。
Configuration file handling
A transient guest only exists while it is running, and has no configuration file stored on disk.
A persistent guest maintains a configuration file on disk even when it is not running.
The virsh command has two flags to influence this behaviour.
The --undefine-source flag will cause the configuration file to be removed on the source host after a successful migration.
The --persist flag will cause a configuration file to be created on the destination host after a successful migration.
我们首先需要使得source和destination之间的libvirt是相互通的。
在source机器上，16.158.166.197
root@escto-bj-hp-z620:/home/cliu8/certtool# ls -l
total 60
-r--r--r-- 1 root root 1204 Jul 17 20:17 certificate_authority_certificate.pem
-r--r--r-- 1 root root 1972 Jul 17 20:17 certificate_authority_key.pem
-r--r--r-- 1 root root 37 Jul 17 20:17 certificate_authority_template.info
-r--r--r-- 1 root root 1379 Jul 17 20:17 escto-bj-hp-z620_client_certificate.pem
-r--r--r-- 1 root root 1968 Jul 17 20:17 escto-bj-hp-z620_client_key.pem
-r--r--r-- 1 root root139 Jul 17 20:17 escto-bj-hp-z620_client_template.info
-r--r--r-- 1 root root 1310 Jul 17 20:17 escto-bj-hp-z620_server_certificate.pem
-r--r--r-- 1 root root 1968 Jul 17 20:17 escto-bj-hp-z620_server_key.pem
-r--r--r-- 1 root root 91 Jul 17 20:17 escto-bj-hp-z620_server_template.info
-r--r--r-- 1 root root 1371 Jul 17 20:17 popsuper1982_client_certificate.pem
-r--r--r-- 1 root root 1972 Jul 17 20:17 popsuper1982_client_key.pem
-r--r--r-- 1 root root135 Jul 17 20:17 popsuper1982_client_template.info
-r--r--r-- 1 root root 1306 Jul 17 20:17 popsuper1982_server_certificate.pem
-r--r--r-- 1 root root 1968 Jul 17 20:17 popsuper1982_server_key.pem
-r--r--r-- 1 root root 87 Jul 17 20:17 popsuper1982_server_template.info
root@escto-bj-hp-z620:/home/cliu8/certtool# tree --charset ASCII /etc/pki/
/etc/pki/
|-- CA
| `-- cacert.pem -> /home/cliu8/certtool/certificate_authority_certificate.pem
|-- libvirt
| |-- clientcert.pem -> /home/cliu8/certtool/escto-bj-hp-z620_client_certificate.pem
| |-- private
| | |-- clientkey.pem -> /home/cliu8/certtool/escto-bj-hp-z620_client_key.pem
| | `-- serverkey.pem -> /home/cliu8/certtool/escto-bj-hp-z620_server_key.pem
| `-- servercert.pem -> /home/cliu8/certtool/escto-bj-hp-z620_server_certificate.pem
`-- nssdb -> /var/lib/nssdb
配置/etc/libvirt/libvirtd.conf
为了方便测试，我们对tcp, tls不进行密码设置
listen_tls = 1
listen_tcp = 1
tls_port = "16514"
tcp_port = "16509"
unix_sock_group = "libvirtd"
unix_sock_ro_perms = "0777"
unix_sock_rw_perms = "0770"
auth_unix_ro = "none"
auth_unix_rw = "none"
auth_tcp = "none"
auth_tls = "none"
测试下面的命令都能通过
virsh -c qemu+ssh://cliu8@16.158.166.150/system list --all
virsh -c qemu+tcp://popsuper1982/system list --all
virsh -c qemu+tls://popsuper1982/system list --all
在destination机器上，16.158.166.150
root@popsuper1982:/home/cliu8/certtool# ls -l
total 60
-r--r--r-- 1 root root 1204 Jul 15 22:31 certificate_authority_certificate.pem
-r--r--r-- 1 root root 1972 Jul 15 22:28 certificate_authority_key.pem
-r--r--r-- 1 root root 37 Jul 15 22:26 certificate_authority_template.info
-r--r--r-- 1 root root 1379 Jul 16 00:27 escto-bj-hp-z620_client_certificate.pem
-r--r--r-- 1 root root 1968 Jul 16 00:25 escto-bj-hp-z620_client_key.pem
-r--r--r-- 1 root root139 Jul 16 00:24 escto-bj-hp-z620_client_template.info
-r--r--r-- 1 root root 1310 Jul 17 20:09 escto-bj-hp-z620_server_certificate.pem
-r--r--r-- 1 root root 1968 Jul 17 20:07 escto-bj-hp-z620_server_key.pem
-r--r--r-- 1 root root 91 Jul 17 20:06 escto-bj-hp-z620_server_template.info
-r--r--r-- 1 root root 1371 Jul 17 20:14 popsuper1982_client_certificate.pem
-r--r--r-- 1 root root 1972 Jul 17 20:13 popsuper1982_client_key.pem
-r--r--r-- 1 root root135 Jul 17 20:12 popsuper1982_client_template.info
-r--r--r-- 1 root root 1306 Jul 16 00:09 popsuper1982_server_certificate.pem
-r--r--r-- 1 root root 1968 Jul 16 00:06 popsuper1982_server_key.pem
-r--r--r-- 1 root root 87 Jul 16 00:05 popsuper1982_server_template.info
root@popsuper1982:/home/cliu8/certtool# tree --charset ASCII /etc/pki/
/etc/pki/
|-- CA
| `-- cacert.pem -> /home/cliu8/certtool/certificate_authority_certificate.pem
|-- libvirt
| |-- clientcert.pem -> /home/cliu8/certtool/popsuper1982_client_certificate.pem
| |-- private
| | |-- clientkey.pem -> /home/cliu8/certtool/popsuper1982_client_key.pem
| | `-- serverkey.pem -> /home/cliu8/certtool/popsuper1982_server_key.pem
| `-- servercert.pem -> /home/cliu8/certtool/popsuper1982_server_certificate.pem
`-- nssdb -> /var/lib/nssdb
virsh -c qemu+ssh://cliu8@16.158.166.197/system list --all
virsh -c qemu+tcp://escto-bj-hp-z620/system list --all
virsh -c qemu+tls://escto-bj-hp-z620/system list –all 在source机器上，启动一个虚拟机virsh start ubuntu-14.04
# virsh dumpxml ubuntu-14.04
ubuntu-14.04
0f0806ab-531d-6134-5def-c5b495529284
2097152
2097152
1
/machine
hvm
destroy
restart
restart
/usr/bin/kvm-spice
其中cdrom和usb的都应该去掉，才能migration成功。
# virsh migrate --verbose --live --persistent ubuntu-14.04 qemu+tcp://popsuper1982/system
Migration:
页:
[1]