unijun 发表于 2015-10-10 12:14:36

qemu-kvm如何mem 热插拔

linux内核已经实现了内存热插的支持,当一个linux系统不管运行在 物理环境 或者虚拟环境时只要宿主能提供内存热插拔机制,linux内核就能相应的增加或者减少内存。  目前已知的VMware已经可以做到内存热插。
  改变内存容量有两种方式:
view plaincopyprint?
[*]401.1 purpose of memory hotplug
[*]41------------
[*]42Memory Hotplug allows users to increase/decrease the amount of memory.
[*]43Generally, there are two purposes.
[*]44   
[*]45(A) For changing the amount of memory.
[*]46      This is to allow a feature like capacity on demand.
[*]47(B) For installing/removing DIMMs or NUMA-nodes physically.
[*]48      This is to exchange DIMMs/NUMA-nodes, reduce power consumption, etc.
[*]   
  前些天检查maillist时发现已经有人实现了DIMMs 方式更改内存大小。这种方式还没有并入mainline tree。
view plaincopyprint?
[*]https://github.com/vliaskov/seabios/commits/memhp-v3-alt
[*]https://github.com/vliaskov/qemu-kvm/commits/memhp-v3-alt
  

  可以测试下:
  1)下载带有 dimm hotplug 功能的qemu-kvm:
view plaincopyprint?
[*]git clone -bmemhp-v3-alt https://github.com/vliaskov/qemu-kvm/commits/memhp-v3-alt
  配置编译:
view plaincopyprint?
[*]$./configure --target-list="i386-softmmu x86_64-softmmu i386-linux-user x86_64-linux-user" --enable-kvm   
view plaincopyprint?
[*]$make
  
2)下载linux内核 linux-3.0.1.tar.bz2 做测试:(32bit)
  linux memory hotplug 需要内核编译进如下选项:
view plaincopyprint?
[*]- For all memory hotplug
[*]    Memory model -> Sparse Memory(CONFIG_SPARSEMEM)
[*]location:
[*]Processor type and features   
[*]    Memory model (Sparse Memory)--->
view plaincopyprint?
[*]    Allow for memory hot-add       (CONFIG_MEMORY_HOTPLUG)
[*]location:
[*]Power management and ACPI options
[*]    ACPI (Advanced Configuration and Power Interface) Support
[*]      Memory Hotplug
view plaincopyprint?
[*]- To enable memory removal, the followings are also necessary
[*]    Allow for memory hot remove    (CONFIG_MEMORY_HOTREMOVE)
[*]    Page Migration               (CONFIG_MIGRATION)
view plaincopyprint?
[*]- For ACPI memory hotplug, the followings are also necessary
[*]    Memory hotplug (under ACPI Support menu) (CONFIG_ACPI_HOTPLUG_MEMORY)
[*]    This option can be kernel module.
[*]
[*]- As a related configuration, if your box has a feature of NUMA-node hotplug
[*]via ACPI, then this option is necessary too.
[*]    ACPI0004,PNP0A05 and PNP0A06 Container Driver (under ACPI Support menu)
[*]    (CONFIG_ACPI_CONTAINER).
[*]    This option can be kernel module too.
  配置好之后就可以用make 了。
  编译完成后将得到的bzImage文件拷贝出来。
  3)制作启动所需固件:
  当内存hotplugged之后,内核需要知道新增加上的内存并且制作内存管理表,生成sysfs文件来操作内存。
  由于这需要固件的支持,而现有的qemu mainline 中包含的bios并不支持相应的功能,所以需要下载bios来重新编译:
view plaincopyprint?
[*]git clone -b memhp-v3-alt https://github.com/vliaskov/seabios/commits/memhp-v3-alt

创建一个放固件的目录:bios  里面放的固件如下所示:(这些都是编译后qemu目录下的)
view plaincopyprint?
[*].
[*]├── bios.bin
[*]├── keymaps
[*]│   ├── common
[*]│   ├── en-us
[*]│   └── modifiers
[*]├── kvmvapic.bin
[*]├── linuxboot.bin
[*]├── multiboot.bin
[*]├── optionrom
[*]│   ├── kvmvapic.bin
[*]│   ├── linuxboot.bin
[*]│   └── multiboot.bin
[*]├── ppc_rom.bin
[*]├── pxe-e1000.rom
[*]├── sgabios.bin
[*]├── slof.bin
[*]├── spapr-rtas.bin
[*]├── vgabios.bin
[*]├── vgabios-cirrus.bin
[*]├── vgabios-qxl.bin
[*]├── vgabios-stdvga.bin
[*]└── vgabios-vmware.bin
  将编译好的固件 ./out/bios.bin替换上面的bios.bin。
  4)制作文件系统busybox.img
  可参考如下连接:
view plaincopyprint?
[*]http://blog.iyunv.com/gudujianjsk/article/details/7655855
  

  现在测试用的内核文件都得到了,用如下脚本开启测试:
view plaincopyprint?
[*]./../qemu-kvm/i386-softmmu/qemu-system-i386 -enable-kvm -m 512 -kernel bzImage -L ./../bios -append "root=/dev/sda" -dimm id=dimm0,size=512M,node=0,populated=off -boot c -smp 1-hda busybox.img -monitor stdio -k en-us -vnc :2
  启动后系统显示如下:

  由于编译时配置将 CONFIG_ACPI_HOTPLUG_MEMORY 配置为内核模块了,所以这里需要首先加载一个内核模块:acpi_memhotplug.ko.
  所有的sections都在sysfs下有它们的设备信息。每一个section都是在 /sys/devices/system/memory/目录下的内存块的一部分。
view plaincopyprint?
[*]/sys/devices/system/memory/memoryXXX
  在这里XXX就是内存ID。

  在monitor中动态 增加dimm:
view plaincopyprint?
[*]sae@lx15:/var/dbg/test-dimm/busybox$ sudo ./test-box.sh   
[*]QEMU 1.2.50 monitor - type 'help' for more information
[*](qemu) device_add dimm,id=dimm0
  此时在虚拟机中可见多了个memory1,其状态是offline:

  此时新增加的内存还没有做好使用的准备。为了使用新增加的内存你需要把新增加的内存设置为online 状态

  
  此时用free查看总计的内存数量:

  
  测试完毕。
  

  http://www.mjmwired.net/kernel/Documentation/memory-hotplug.txt
页: [1]
查看完整版本: qemu-kvm如何mem 热插拔