[zz]Scripting KVM with Python, Part 1: libvirt

qaqa12345667

http://www.ibm.com/developerworks/opensource/library/os-python-kvm-scripting1/



Paul Ferrill, CTO, ATAC



Paul Ferrill has been writing in the computer trade press for more than 20 years. He got his start writing networking reviews for PC Magazine on products like LANtastic and early versions of Novell Netware. Paul holds both BSEE and MSEE degrees and has written software for more computer platforms and architectures than he can remember.　　
　　Summary:  This two-part series explores how to use Python to create scripts for managing virtual machines using KVM. In this installment, learn the basics of using libvirt and the Python bindings to build a few simple status and display tools.


　　Tags for this article:  android, application_development, kvm, libvirt, linux, open_source, paul_ferrill, python, scripting
　　Virtualization is standard equipment with most server operating systems on the market today. In the Linux® world, there are two primary choices for server virtualization: the Kernel-based Virtual Machine (KVM) and Xen. KVM is the primary technology that Red Hat and others use. Although Citrix owns Xen, much of the core functionality remains in the public domain.
　　The Virtual Machine Manager (VMM, or virt-manager) project provides a tool for managing the creation and running of both KVM and Xen virtual machine (VM) instances. VMM is written in Python using the GTK+ library for graphical user interface construction. The real work is done through the libvirt library, which is what you'll be using for this article series. Although libvirt is a Red Hat-sponsored effort, it remains an open source project available under the GNU Lesser General Public License.
　　libvirt is made up of several different pieces, including the application programming interface (API) library, a daemon (libvirtd), and a default command-line utility (virsh). For the purposes of this article, all testing is done using Ubuntu Server version 11.04. The Installation and setup section covers everything I did to configure my server for developing the scripts presented here. Part 1 covers the basics of libvirt and Kernel-based Virtual Machine (KVM) virtualization along with a few command-line scripts to whet your appetite. Part 2 will dive deeper and show you how you can build your own virtualization management tools using libvirt, Python, and wxPython.
　　Getting started
　　Before we dive into the actual code examples, let's go over a few terms and concepts related to virtualization with KVM. When you install KVM on a server like Ubuntu Server 11.04, you're establishing a virtualization host, or hypervisor. That means that your server will be able to host multiple guest operating systems running on top of the KVM host. Each unique guest is called a domain and functions in much the same way you would expect from a single server instance on an individual machine. You can connect to the server over Secure Shell (SSH) or Virtual Network Computing just as if you were communicating with a physical machine.
　　Although KVM functions as the hypervisor or guest manager, QEMU provides the actual machine emulation, meaning that QEMU executes the native instruction set of the target machine. For x86 guests, this execution translates into native instructions capable of direct execution on the underlying hardware. For other architectures, such as ARM, a translation process must take place. The combination of KVM and QEMU provides all the support functions needed to virtualize essentially every currently available operating system plus a few that are no longer available.
　　A guest domain consists of a number of files, including one or more disk image files and an XML-based configuration file. This setup makes it extremely simple to manage multiple VMs by creating a baseline system image, and then modifying the configuration file to suit your needs. One method of configuring and communicating with KVM/QEMU is the libvirt toolkit. Multiple vendors have standardized their management products based on libvirt.
　　Look at the contents of a typical domain configuration file. Listing 1 shows the testdev.xml file from the libvirt examples.
　　
Listing 1. Device XML definition

File_test_device Libvirt Test driver 123456 11111111-2222-3333-4444-555555555555 Libvirt Test Driver 01/22/2007

　　
　　From the test domfv0.xml file shown in Listing 2, you can see a bit more detail about configuring virtual devices.
　　
Listing 2. domfv0.xml device definition file

/usr/lib/xen/bin/qemu-dm

　　
　　The key point here is the relative ease with which you can read these files and subsequently create your own. Although you could build any number of configuration files by hand, it's also possible to automate the building using a scripting language like Python.



Back to top
　　Installation and setup
　　Because this article is about scripting KVM, there is a basic assumption that you have a server with KVM installed. In the case of Ubuntu Server 11.04, you have the option of installing virtualization during the setup process by choosing the Virtual Machine Host option on the Software selection screen. You might also want to choose the OpenSSH server should you want to connect remotely to the machine.
　　The first order of business is to install the latest version of libvirt. To do this, you have to do some command-line work. When you install Ubuntu Server 11.04, you get libvirt version 0.8.8. The latest and greatest version available from the libvirt website is 0.9.5. To install a later version, you need to add a Personal Package Archive (PPA) repository to your system containing a more recent version of libvirt. A quick search on the launchpad.net site for libvirt shows a number of potential candidates. It's important to view the repository details page before you try to perform an update, as some may have broken packages. The Ubuntu Virtualization Team maintains a PPA repository with several packages, including libvirt. The latest version available at the time of this writing was 0.9.2-4.
　　Perform the following steps to install that version:

• Install the python-software-properties package as follows:
  

  sudo apt-get install python-software-properties

  
  　　This command makes available the add-apt-repository command that you need to reference the third-party source.
  
  
• Type the following commands:
  

  sudo add-apt-repository ppa:ubuntu-virt/ppa sudo apt-get update sudo apt-get install libvirt-bin

  
  
• Because you'll be using Python to do all the scripting for this article, install the IDLE shell to make it easier to write and test scripts.　　This step assumes that you have installed the desktop environment on your Ubuntu server. The quickest way to get the desktop installed is to use the following command:
  
  

  sudo apt-get install ubuntu-desktop

  
  

　　After that's done, you'll have access to any number of graphical applications along with the Ubuntu software installer. You can use the Ubuntu Software Center to install the Python IDLE tool.



Back to top
　　Sample scripts
　　At this point, let's look at some of the fundamentals of working with libvirt before getting too deeply into code. Communication between an application and the libvirt library uses a simple remote procedure call mechanism, which makes it possible to build applications to communicate with remote hypervisors over a TCP/IP connection. Uniform Resource Identifiers (URIs, defined by Internet Engineering Task Force [IETF] Request for Comments [RFC] 2396) are used to identify a specific hypervisor with which you want to establish a connection.
　　Local connections typically do not require authentication, although some remote connections do. The libvirt.conf file controls the security configuration. The most extensive control over communicating with a unique domain is through network filtering. Here's an example of how you would control network traffic using a filter:

　　
　　This snippet defines a filter named clean-traffic that will be applied to all network traffic over the specified media access control (MAC) address. If you examine the clean-traffic XML, it contains the following:

6f145c54-e3de-4c33-544a-70b69c16d9da

　　
　　The filter capabilities are quite extensive and fully documented. It takes only one command if you want to have a local copy of the libvirt documentation and sample files. Here's what you need to do:

sudo apt-get install libvirt-doc

　　
　　With that done, all the documentation is available in the /usr/share/doc/libvirt-doc directory. You'll see the Python examples a bit later. If you've updated to a more recent version of libvirt, you may need to explicitly install the Python bindings. Doing so requires a single command:

sudo apt-get install python-libvirt

　　
　　Use Python's IDLE console to examine Python code to establish a connection with a local QEMU instance, and then examine the defined domains. Listing 3 shows what you should see using this approach.
　　
Listing 3. Viewing Python code in the IDLE console

Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) [GCC 4.5.2] on linux2 Type "copyright", "credits" or "license()" for more information. ==== No Subprocess ==== >>> import libvirt >>> conn=libvirt.open("qemu:///system") >>> names = conn.listDefinedDomains() >>> print names ['Test1', 'SBSLite', 'UbuntuServer1104', 'Win7_64-bit']] >>>

　　
　　This code shows how to get a list of all defined domains. The return from the listDefinedDomains() function shows a list containing four named domains. Once you establish a connection to the hypervisor, you will have access to a laundry list of available functions. Here's a short script showing how to get a list of all available functions available on the conn object:

clist = dir(conn) for item in clist: print item

　　
　　To see a list of defined filters, you could use a similar approach:

filts = conn.listNWFilters() for item in filts: print item

　　
　　The IDLE tool is a great way to investigate the various API calls and quickly see the results returned when executed. Some of the functions operate only on running domains. The Python dir() function returns a list of valid attributes for the specified object. It's a convenient command-line tool to quickly see what a particular object provides. You can use it as shown above to get a list of functions available after establishing a connection to the hypervisor.
　　To demonstrate, you can use a few lines of Python code in the IDLE console to get an idea of the kinds of operations you can perform on a specific domain. Listing 4 provides an example of what you can do.
　　
Listing 4. Python output of a domain object

>>> import libvirt >>> import pprint >>> conn=libvirt.open("qemu:///system") >>> p = conn.lookupByName('ubuntu100403') >>> pprint.pprint(dir(p)) ['ID', 'OSType', 'UUID', 'UUIDString', 'XMLDesc', '__del__', '__doc__', '__init__', '__module__', '_conn', '_o', 'abortJob', 'attachDevice', 'attachDeviceFlags', 'autostart', 'blkioParameters', 'blockInfo', 'blockPeek', 'blockStats', 'connect', 'coreDump', 'create', 'createWithFlags', 'destroy', 'detachDevice', 'detachDeviceFlags', 'hasCurrentSnapshot', 'hasManagedSaveImage', 'info', 'injectNMI', 'interfaceStats', 'isActive', 'isPersistent',

　　
　　You can take this basic approach to building a simple script that lists information about all running domains. You use the listDomainsID() and lookupByID() function calls to do most of the work, as Listing 5 shows.
　　
Listing 5. Python list domains script

import libvirt conn=libvirt.open("qemu:///system") for id in conn.listDomainsID(): dom = conn.lookupByID(id) infos = dom.info() print 'ID = %d' % id print 'Name =  %s' % dom.name() print 'State = %d' % infos[0] print 'Max Memory = %d' % infos[1] print 'Number of virt CPUs = %d' % infos[3] print 'CPU Time (in ns) = %d' % infos[2] print ' '

　　
　　The output from this script, with one domain active and another suspended, looks like this:

ID = 3 Name =  ubuntu100403 State = 3 Max Memory = 1048576 Number of virt CPUs = 1 CPU Time (in ns) = 1048576 ID = 4 Name =  Win7_64-bit State = 1 Max Memory = 2097152 Number of virt CPUs = 2 CPU Time (in ns) = 2097152

　　
　　libvirt also implements Python docstrings for all classes and methods. You can access this information by typing help(libvirt) for the top-level help or help(libvirt.class) for a specific class. You must have imported the libvirt module before typing the help() command. The version I tested for this review implements the following 11 classes:

• libvirtError
  
• virConnect
  
• virDomain
  
• virDomainShapshot
  
• virInterface
  
• virNWFilter
  
• virNetwork
  
• virSecret
  
• virStoragePool
  
• virStorageVol
  
• virStream
  

　　This list should help you decode the syntax for accessing libvirt functions from Python. It also gives you a list of all named constants, like VIR_DOMAIN_RUNNING, which equals 1. Functions like dom.info(), used above, return an integer value and need to be decoded against this constant table.



Back to top
　　Utility scripts for automation
　　You could write any number of utility scripts to manage a KVM installation using libvirt and Python. It might not be efficient for a small number of domains but can quickly save time when the count gets into double digits. One simple task would be to make a mass change of static IP addresses for all domain images. You can do this by iterating over all the .conf files, and then making the appropriate changes. Python has many built-in features to help with this task.
　　Listing 6 shows an example of an XML network definition.
　　
Listing 6. Network configuration XML file

[table]

[tr]
[td]
         

testnetwork