GPU pass-through on KVM with Windows 7 Guest
2015年1月13日 发布OS: CentOS 6.5
Graphic Cards: Embedded Graphic Card, Grid K1, Grid K2
Hypervisor: KVM
Benchmark tool: Specviewperf 11
1. kernel recompile using linux-3.18.2
Download link:
https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.18.2.tar.gz
Before compiling kernel, install needed packages:
[*]yum install ncurses-devel hmaccalc zlib-devel binutils-devel elfutils-libelf-devel qt-devel
[*]yum groupinstall "Development Tools"
Extract to /usr/scr/ directory
[*]tar -xvzf linux-3.18.2
[*]cd linux-3.18.2
[*]make mrproper
[*]make clean
[*]cp /boot/config-'uname -r' ./.config
[*]make menuconfig
There are 3 big options to check and modify:
[*]Bus Options (PCI etc.)
[*]<*> PCI Stub driver
[*]Networking support (changing kernel from 2.x version to 3.x version)
[*]Networking options
[*]Network packet filtering framework(Netfilter)
[*]IP: Netfilter Configuration
[*]<M> IPv4 NAT
[*]IPv6: Netfilter Configuration
[*]<M> IPv6 NAT
[*]Device Drivers
[*]
[*] IOMMU Hardware Support
[*]
[*] Support for IOMMU using DMA Remapping Devices
[*]
[*] Enable DMA Remapping Devices by default
[*]<*> VFIO Non-Privileged userspace driver framework
[*]<*> VFIO support for PCI devices
[*]
[*] VFIO PCI support for VGA devices
Continue to recompile kernel
[*]make all // only when you first recompile kernel, after that, use make command
[*]make modules_install
[*]make install
[*]vi /boot/grub/grub.conf
Here modify kernel booting priority: default =0
Add intel_iommu=on to kernel line
https://3.bp.blogspot.com/-Gx23kXzJyLY/VLS2gn75OvI/AAAAAAAAAuE/9bY2rd95dIc/s1600/1.PNG
(If you are using an intel IGP on your host, you need the i915 vga arbiter patch)
i915 vga arbiter fixes file: https://lkml.org/lkml/2014/5/9/517
Continue to recompile kernel
[*]reboot
Check if your host is ready to PCI device pass-through using follow commands:
[*]dmesg | grep -e DMAR -e IOMMU
ACPI: DMAR 0x00000000BD3346F4 000158 (v01 DELL PE_SC3 00000001 DELL 00000001)
Intel-IOMMU: enabled
dmar: IOMMU 0: reg_base_addr d5000000 ver 1:0 cap d2078c106f0466 ecap f020de
dmar: IOMMU 1: reg_base_addr df900000 ver 1:0 cap d2078c106f0466 ecap f020de
IOAPIC id 2 under DRHD base0xd5000000 IOMMU 0
IOAPIC id 0 under DRHD base0xdf900000 IOMMU 1
IOAPIC id 1 under DRHD base0xdf900000 IOMMU 1
IOMMU 0 0xd5000000: using Queued invalidation
IOMMU 1 0xdf900000: using Queued invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:1d.0
IOMMU: Setting identity map for device 0000:00:1a.0
IOMMU: Setting identity map for device 0000:00:1a.0
IOMMU: Setting identity map for device 0000:00:1d.0
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0
2. Install kvm
[*]yum install kvm qemu-kvm python-virtinst libvirt libvirt-python virt-manager libguestfs-tools
(or)
[*]yum groupinstall "Virtualisation Tools" "Virtualization Platform“
[*]yum install python-virtins
Start kvm:
[*]chkconfig libvirtd on
[*]service libvirtd start
3. Create Windows 7 guest
[*]ISO image file download link: Click Here
[*]VirtIO drivers for windows guest download link: Click Here
Run virt-manager
[*]virt-manager //when using vnc to access, use dbus-launch virt-manager command
Network bridge setup
Edit->Connection Detail->Network Interface
Ad br0 (bridge) connect to em1
https://1.bp.blogspot.com/-AyoE0Q0Qjt4/VLTD5aD42xI/AAAAAAAAAuU/p1lCcwoFSHU/s1600/2.PNG
Edit /etc/sysconfig/network-scripts/ifcfg-br0
Add GATEWAY and DNS1
Restart network service
[*]service network restart
Create qcow2 Image Disk
[*]cd /home/'username'/images/
[*]qemu-img create -f qcow2-o preallocation=metadata win7.qcow2 60G
(image convert command is like this:
[*]qemu-img convert -f raw -O qcow2 win7.dsk win7.qcow2
[*]qemu-img convert -f vmdk -O raw win7.vmdk win7.img )
Create new virtual machine (windows 7)
Add Windows 7 ISO file and virtIO driver file(Load VirtIO driver)
https://4.bp.blogspot.com/-zCtMAuOe9Qs/VLTKH60AG5I/AAAAAAAAAuk/DZvt0HbQ4Xs/s1600/3.PNG
Setup public ip to virtual machine
https://1.bp.blogspot.com/-G22DnojGPD0/VLTNrpgyS_I/AAAAAAAAAuw/Wg5hjFv5m7M/s1600/4.PNG
Setup remote access with port 2222(course there is no DVI output on GridK1,K2)
My Computer(Right Click) -> Option -> Remote Setup -> Access All
Control Panel -> Windows Firewall -> Advanced Setting -> Inbound Rule -> New Rule -> Port -> TCP and Special Port(?): 2222 -> Named "Remote Poart 2222"
https://2.bp.blogspot.com/-jSfSnjsDLo0/VLTRQnACTSI/AAAAAAAAAu8/mrdXylhFCMQ/s1600/5.PNG
https://1.bp.blogspot.com/-Ayj0Wx8yuyw/VLTRs72xdAI/AAAAAAAAAvE/K-NLpKlQ3_o/s1600/6.PNG
run->regedit
[*]HKEY_LOCAL_MACHINE -> SYSTEM -> CurrentControlSet -> Control -> Terminal server -> Wds -> rdpwd -> Tds -> tcp ->PortNumber
[*]HKEY_LOCAL_MACHINE -> SYSTEM -> CurrentControlSet -> Control -> Terminal server -> WinStations -> RDP-Tcp -> PortNumber
Edit 10 value as 2222
https://2.bp.blogspot.com/-qgpVrSyIiuU/VLTTOC_wwVI/AAAAAAAAAvQ/52tyYRE-9Wo/s1600/1.PNG
we can use xxx.xxx.xxx.x:2222 to remote access.
3. Install Needed Softwares
[*]GPU-Z Click here to Download
[*]Specviewperf 11 Click Here to Download
4. Bind PCI device to virtual machine using pci-stub driver
[*]lspci -nn | grep VGA
07:00.0 VGA compatible controller : NVIDIA Corporation GK107GL (rev a1)
08:00.0 VGA compatible controller : NVIDIA Corporation GK107GL (rev a1)
09:00.0 VGA compatible controller : NVIDIA Corporation GK107GL (rev a1)
0a:00.0 VGA compatible controller : NVIDIA Corporation GK107GL (rev a1)
10:00.0 VGA compatible controller : Matrox Electronics Systems Ltd. G200eR2
[*]vi /usr/bin/pci-bind
-----------------------------------------------------------------------------------
#!/bin/bash
#modprobe pci-stub
for dev in "$@"; do
vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
device=$(cat /sys/bus/pci/devices/$dev/device)
if [ -e /sys/bus/pci/devices/$dev/driver ]; then
echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
fi
echo $vendor $device > /sys/bus/pci/drivers/pci-stub/new_id
done
------------------------------------------------------------------------------------
[*]chmod 755 /usr/bin/pci-bind
[*]pci-bind 0000:07:00.0
[*]lspci -v -s 07:00.0
------------------------------------------------------------------------------------
07:00.0 VGA compatible controller: NVIDIA Corporation GK107GL (rev a1) (prog-if 00 )
Subsystem: NVIDIA Corporation Device 1012
Flags: bus master, fast devsel, latency 0, IRQ 33
Memory at d4000000 (32-bit, non-prefetchable)
Memory at 3fff8000000 (64-bit, prefetchable)
Memory at 3fff6000000 (64-bit, prefetchable)
I/O ports at <ignored>
Expansion ROM at d5800000
Capabilities: Power Management version 3
Capabilities: MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: Express Endpoint, MSI 00
Capabilities: Vendor Specific Information: Len=14 <?>
Capabilities: Virtual Channel
Capabilities: Power Budgeting <?>
Capabilities: Advanced Error Reporting
Capabilities: Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
Capabilities: #19
Kernel driver in use: pci-stub
------------------------------------------------------------------------------------
5. Attach PCI device to virtual machine
[*]virt-manager
Add Hardware -> PCI Host Device -> 0000:07:00.0 GK107GL
https://4.bp.blogspot.com/-xfZJdk7Y_ZY/VLTdlC0P5dI/AAAAAAAAAvg/vn0AxsvZ2tg/s1600/1.PNG
[*]Start win7 (virtual machine)
[*]Install NVIDIA graphic driver 341.21
6. Accecc to Virtual Machine by Remote Desktop
[*]xxx.xxx.xxx.x:2222
[*]Run GPU-Z
https://4.bp.blogspot.com/-zbX53WmdWj0/VLTswipYSkI/AAAAAAAAAvw/I4NRPwI_gBI/s1600/1.PNG
[*]Run Specviewperf 11
https://1.bp.blogspot.com/-ZZCgyeFsx1Y/VLTtT0xI0II/AAAAAAAAAv4/iI96cWZwnc8/s1600/1.PNG
页:
[1]