设为首页 收藏本站
查看: 2440|回复: 1

[经验分享] kvm及libvirt、使用virsh管理kvm虚拟机、网络虚拟化技术基础...

[复制链接]
累计签到:1 天
连续签到:1 天
发表于 2017-1-3 09:13:30 | 显示全部楼层 |阅读模式
kvm及libvirt、使用virsh管理kvm虚拟机、网络虚拟化技术基础、网络名称空间netns用法详解
01kvm及libvirt

[iyunv@node1 ~]# yum install libvirt libvirt-client python-virtinst virt-manager virt-install -y
[iyunv@node1 ~]# yum -y install qemu-kvm
[iyunv@node1 ~]# systemctl start libvirtd.service

#创建桥
[iyunv@node1 ~]# virsh iface-bridge eno16777736 br0 --no-stp
[iyunv@node1 ~]# systemctl restart network.service
[iyunv@node1 ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.000c29ba0d2f       no              eno16777736
virbr0          8000.52540067a9f3       yes             virbr0-nic

#创建虚拟机
[iyunv@node1 ~]# virt-manager

02使用virsh管理kvm虚拟机

[iyunv@node1 ~]# yum -y install virt-viewer
[iyunv@node1 ~]# mkdir /images/centos -p
[iyunv@node1 ~]# virt-install -n centos6.7 -r 512 --vcpus=2,maxvcpus=4 --pxe --disk /images/centos/centos6.7.qcow2,size=120,format=qcow2,bus=virtio,sparse=yes --network bridge=br0,model=virtio --force
[iyunv@node1 ~]# virsh list
Id    名称                         状态
----------------------------------------------------
3     centos6.7                      running
[iyunv@node1 ~]# virt-viewer 3

#强制关机
[iyunv@node1 ~]# virsh destroy 1

[iyunv@node1 ~]# ls /etc/libvirt/qemu
centos6.7.xml  networks
#根据xml文件创建虚拟机
[iyunv@node1 ~]# virsh create /etc/libvirt/qemu/centos6.7.xml
域 centos6.7 被创建从 /etc/libvirt/qemu/centos6.7.xml
#创建的虚拟机已经运行了
[iyunv@node1 ~]# virsh list
Id    名称                         状态
----------------------------------------------------
2     centos6.7                      running

#删除虚拟机
[iyunv@node1 ~]# virsh undefine centos6.7 --remove-all-storage
域 centos6.7 已经被取消定义
已删除卷 'vda'(/images/centos/centos6.7.qcow2)。

[iyunv@node1 ~]# virt-install -n centos6.7 -r 512 --vcpus=2,maxvcpus=4 --pxe --disk /images/centos/centos6.7.qcow2,size=120,format=qcow2,bus=virtio,sparse=yes --network bridge=br0,model=virtio --force --nographics

[iyunv@node1 ~]# virt-install -n centos6.7 -r 512 --vcpus=2,maxvcpus=4 --pxe --disk /images/centos/centos6.7.qcow2,size=120,format=qcow2,bus=virtio,sparse=yes --network bridge=br0,model=virtio --force --graphics vnc

[iyunv@node1 ~]# mkdir /images/cirros
[iyunv@node1 ~]# mv cirros-no_cloud-0.3.0-i386-disk.img /images/cirros/
#试运行操作
[iyunv@node1 ~]# virt-install -n cirros -r 128 --disk /images/cirros/cirros-no_cloud-0.3.0-i386-disk.img  --import --dry-run
WARNING  未检测到操作系统虚拟机性能可能会受到影响。使用 --os-variant 选项指定操作系统以获得最佳性能。
试运行成功完成

[iyunv@node1 ~]# virt-install -n cirros -r 128 --disk /images/cirros/cirros-no_cloud-0.3.0-i386-disk.img  --import

[iyunv@node1 ~]# virt-install -n cirros -r 128 --vcpus=1,maxvcpus=4 --disk /images/cirros/cirros-no_cloud-0.3.0-i386-disk.img --network bridge=br0,model=virtio  --import --serial=pty --console=pty --nographics      

#创建磁盘映像文件
[iyunv@node1 ~]# qemu-img create -f qcow2 -o size=20G,preallocation=metadata /images/cirros/second.qcow2
#查看磁盘映像文件信息
[iyunv@node1 ~]# qemu-img info /images/cirros/second.qcow2
image: /images/cirros/second.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 4.1M
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false

#把新增磁盘添加到正在运行的虚拟机上
[iyunv@node1 ~]# virsh attach-disk 8 /images/cirros/second.qcow2 vda --targetbus virtio
成功附加磁盘

#拆除正在运行的虚拟机上的硬盘
[iyunv@node1 ~]# virsh detach-disk 8 vda
成功分离磁盘

#向虚拟机上添加网络接口
[iyunv@node1 ~]# virsh attach-interface 8 bridge virbr0
成功附加接口

#拆除虚拟机上网络接口
[iyunv@node1 ~]# virsh detach-interface 8 bridge --mac 52:54:00:E7:4D:25
成功分离接口

#查看虚拟机的定义信息
[iyunv@node1 ~]# virsh dumpxml cirros

#查看域CPU个数
[iyunv@node1 ~]# virsh vcpucount 8
最大值    配置         4
最大值    live           4
当前       配置         1
当前       live           1

#查看域CPU信息
[iyunv@node1 ~]# virsh vcpuinfo 8
VCPU:           0
CPU:            1
状态       running
CPU 时间   111.0s
CPU关系:      yyyy

#配置域中vcpu
[iyunv@node1 ~]# virsh setvcpus 8 2

[iyunv@node1 ~]# virsh vcpuinfo 8  
VCPU:           0
CPU:            0
状态       running
CPU 时间   111.5s
CPU关系:      yyyy

VCPU:           1
CPU:            0
状态       running
CPU关系:      yyyy

#查看域磁盘位置
[iyunv@node1 ~]# virsh domblklist 8
目标     源
------------------------------------------------
hda        /images/cirros/cirros-no_cloud-0.3.0-i386-disk.img

#查看域网络信息
[iyunv@node1 ~]# virsh domiflist 8
接口     类型     源        型号      MAC
-------------------------------------------------------
vnet0      bridge     br0        virtio      52:54:00:c1:01:14

#查看域信息
[iyunv@node1 ~]# virsh dominfo 8
Id:             8
名称       cirros
UUID:           fc2c006f-f336-47cd-a6d8-b5f474167b88
OS 类型    hvm
状态       running
CPU          2
CPU 时间   124.5s
最大内存 131072 KiB
使用的内存 131072 KiB
持久       是
自动启动 禁用
管理的保存 否
安全性模式 none
安全性 DOI 0

#显示节点信息物理机
[iyunv@node1 ~]# virsh nodeinfo
CPU 型号        x86_64
CPU               4
CPU 频率        2128 MHz
CPU socket        1
每个 socket 的内核数 4
每个内核的线程数 1
NUMA 单元       1
内存大小      8388084 KiB

#显示URI
[iyunv@node1 ~]# virsh uri
qemu:///system

#显示版本号
[iyunv@node1 ~]# virsh version
根据库编译libvirt 2.0.0
使用库libvirt 2.0.0
使用的 API: QEMU 2.0.0
运行管理程序: QEMU 1.5.3

#列出接口
[iyunv@node1 ~]# virsh iface-list
名称               状态     MAC 地址
---------------------------------------------------
br0                  活动     00:0c:29:ba:0d:2f
lo                   活动     00:00:00:00:00:00

#列出过滤器
[iyunv@node1 ~]# virsh nwfilter-list
UUID                                  名称               
------------------------------------------------------------------
13c88621-6473-4883-bb5b-ea25f0b0f8db  allow-arp           
792262fd-29f0-4fd6-8ae3-f7f320774dc1  allow-dhcp         
6175cb41-2c50-420a-b779-9bdbeaa62cb9  allow-dhcp-server   
4ac77628-0369-48b0-8544-bd694fd12994  allow-incoming-ipv4
7ee5bef8-9170-4320-ba3b-bc4d6d5fe6d8  allow-ipv4         
e712460b-3505-4ded-9b99-5ee2ea54c417  clean-traffic      
2fea4106-6c3e-49c5-8297-871bf7656992  no-arp-ip-spoofing  
2ce69339-59ba-4002-81fe-d650e9045927  no-arp-mac-spoofing
4fefe23c-b1fc-4daf-be95-4844d03ff006  no-arp-spoofing     
7d6d1c55-d6ad-44c8-aab1-50de8988db81  no-ip-multicast     
dfd46e5c-37a5-4465-aec9-38c3e8747d87  no-ip-spoofing      
7f86b7e0-9873-4174-83ad-2853af8853cb  no-mac-broadcast   
c396c95a-e1a4-44ad-95a2-c1c76bcca423  no-mac-spoofing     
d6681b9e-cfc5-483e-9f71-170cb4cecf13  no-other-l2-traffic
c7031d28-5e60-4345-ac88-a9b72acc48fb  no-other-rarp-traffic
37140bfc-595f-43f6-a831-fe223106794c  qemu-announce-self  
26522e96-db13-43ad-9bdb-6c29fbcfb9b6  qemu-announce-self-rarp

03网络虚拟化技术基础

#查看802.1q模块信息
[iyunv@node1 ~]# modinfo 8021q
filename:       /lib/modules/3.10.0-327.el7.x86_64/kernel/net/8021q/8021q.ko
version:        1.8
license:        GPL
alias:          rtnl-link-vlan
rhelversion:    7.2
srcversion:     2E63BD725D9DC11C7DA6190
depends:        mrp,garp
intree:         Y
vermagic:       3.10.0-327.el7.x86_64 SMP mod_unload modversions
signer:         CentOS Linux kernel signing key
sig_key:        79:AD:88:6A:11:3C:A0:22:35:26:33:6C:0F:82:5B:8A:94:29:6A:B3
sig_hashalgo:   sha256

#加载模块
[iyunv@node1 ~]# modprobe 8021q

[iyunv@node1 ~]# yum install vconfig


04网络名称空间netns用法详解
配置环境
node1:192.168.1.131CentOS Linux release 7.2
node2:192.168.1.132CentOS Linux release 7.2

[iyunv@node1 ~]# ip netns add r1
[iyunv@node1 ~]# ip netns add r2
[iyunv@node1 ~]# ip netns list  
r2
r1
[iyunv@node1 ~]# ip netns exec r1 ifconfig -a
lo: flags=8<LOOPBACK>  mtu 65536
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
[iyunv@node1 ~]# ip netns exec r1 route -n
[iyunv@node1 ~]# ip netns exec r1 ifconfig lo 127.0.0.1/8 up
[iyunv@node1 ~]# ip netns exec r1 ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[iyunv@node1 ~]# ip netns exec r2 ifconfig -a
lo: flags=8<LOOPBACK>  mtu 65536
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

#查看R1 iptables规则
[iyunv@node1 ~]# ip netns exec r1 iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

#创建物理桥
[iyunv@node1 ~]# brctl addbr br-ex
[iyunv@node1 ~]# ip link set br-ex up
[iyunv@node1 ~]# ip addr del 192.168.1.131/24 dev eno16777736;ip addr add 192.168.1.131/24 dev br-ex;brctl addif br-ex eno16777736

#创建内部桥
[iyunv@node1 ~]# brctl addbr br-in
[iyunv@node1 ~]# ip link set br-in up

#开启路由转发功能
[iyunv@node1 ~]# vim /etc/sysctl.conf
添加
net.ipv4.ip_forward = 1
[iyunv@node1 ~]# sysctl -p
net.ipv4.ip_forward = 1

#创建一对虚拟网卡
[iyunv@node1 ~]# ip link add veth1.1 type veth peer name veth1.2
[iyunv@node1 ~]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-ex state UP mode DEFAULT qlen 1000
    link/ether 00:0c:29:50:27:c4 brd ff:ff:ff:ff:ff:ff
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT
    link/ether 52:54:00:43:0d:84 brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 500
    link/ether 52:54:00:43:0d:84 brd ff:ff:ff:ff:ff:ff
5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 00:0c:29:50:27:c4 brd ff:ff:ff:ff:ff:ff
6: br-in: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT
    link/ether ca:89:b9:dc:cd:7d brd ff:ff:ff:ff:ff:ff
7: veth1.2@veth1.1: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether c6:66:c7:32:06:e1 brd ff:ff:ff:ff:ff:ff
8: veth1.1@veth1.2: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 5e:58:78:10:ed:a0 brd ff:ff:ff:ff:ff:ff
[iyunv@node1 ~]# ip link set veth1.1 netns r1
[iyunv@node1 ~]# ip link set veth1.2 netns r2
[iyunv@node1 ~]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-ex state UP mode DEFAULT qlen 1000
    link/ether 00:0c:29:50:27:c4 brd ff:ff:ff:ff:ff:ff
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT
    link/ether 52:54:00:43:0d:84 brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 500
    link/ether 52:54:00:43:0d:84 brd ff:ff:ff:ff:ff:ff
5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 00:0c:29:50:27:c4 brd ff:ff:ff:ff:ff:ff
6: br-in: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT
    link/ether ca:89:b9:dc:cd:7d brd ff:ff:ff:ff:ff:ff
[iyunv@node1 ~]# ip netns exec r1 ifconfig -a
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth1.1: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether 5e:58:78:10:ed:a0  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[iyunv@node1 ~]# ip netns exec r2 ifconfig -a
lo: flags=8<LOOPBACK>  mtu 65536
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth1.2: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether c6:66:c7:32:06:e1  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

#更改R1网卡显示名
[iyunv@node1 ~]# ip netns exec r1 ip link set veth1.1 name eth0
[iyunv@node1 ~]# ip netns exec r1 ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
8: eth0@if7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 5e:58:78:10:ed:a0 brd ff:ff:ff:ff:ff:ff link-netnsid 1

#更改R2网卡显示名
[iyunv@node1 ~]# ip netns exec r2 ip link set veth1.2 name eth0  
[iyunv@node1 ~]# ip netns exec r1 ip link show                 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
8: eth0@if7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 5e:58:78:10:ed:a0 brd ff:ff:ff:ff:ff:ff link-netnsid 1
#配置R1、R2的IP
[iyunv@node1 ~]# ip netns exec r1 ifconfig eth0 10.0.1.1/24 up
[iyunv@node1 ~]# ip netns exec r2 ifconfig eth0 10.0.1.2/24 up  

#测试R1、R2联通性
[iyunv@node1 ~]# ip netns exec r1 ping 10.0.1.1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.314 ms
64 bytes from 10.0.1.1: icmp_seq=2 ttl=64 time=0.057 ms
^C
--- 10.0.1.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.057/0.185/0.314/0.129 ms
[iyunv@node1 ~]# ip netns exec r1 ping 10.0.1.2
PING 10.0.1.2 (10.0.1.2) 56(84) bytes of data.
64 bytes from 10.0.1.2: icmp_seq=1 ttl=64 time=0.191 ms
64 bytes from 10.0.1.2: icmp_seq=2 ttl=64 time=0.072 ms

[iyunv@node1 ~]# route add default gw 192.168.1.254
[iyunv@node1 ~]# yum -y install qemu-kvm
[iyunv@node1 ~]# modprobe kvm

[iyunv@node1 ~]# mkdir -p /images/cirros
[iyunv@node1 ~]# cd /images/cirros/
[iyunv@node1 cirros]# ls
cirros-no_cloud-0.3.0-i386-disk.img
[iyunv@node1 cirros]# cp cirros-no_cloud-0.3.0-i386-disk.img test1.qcow2
[iyunv@node1 cirros]# cp cirros-no_cloud-0.3.0-i386-disk.img test2.qcow2
[iyunv@node1 cirros]# cd
[iyunv@node1 ~]# vim /etc/qemu-ifup
#!/bin/bash
#
bridge=br-in
if [ -n "$1" ];then
ip link set $1 up
brctl addif $bridge $1
[ $? -eq 0 ] && exit 0 || exit 1
else
echo "Error: no interface specified."
exit 1
fi  
[iyunv@node1 ~]# chmod +x /etc/qemu-ifup
[iyunv@node1 ~]# bash -n /etc/qemu-ifup
[iyunv@node1 ~]# ln -s /usr/libexec/qemu-kvm /usr/bin/

#安装第一个虚拟机
[iyunv@node1 ~]# qemu-kvm -m 128 -smp 1 -name vm1 -drive file=/images/cirros/test1.qcow2,if=virtio,media=disk -net nic,macaddr=52:54:00:aa:bb:cc -net tap,ifname=vif1.0,script=/etc/qemu-ifup --nographic

#安装第二个虚拟机
[iyunv@node1 ~]# qemu-kvm -m 128 -smp 1 -name vm2 -drive file=/images/cirros/test2.qcow2,if=virtio,media=disk -net nic,macaddr=52:54:00:aa:bb:dd -net tap,ifname=vif2.0,script=/etc/qemu-ifup --nographic

[iyunv@node1 ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
br-ex           8000.000c295027c4       no              eno16777736
br-in           8000.1a39688cf1ea       no              vif1.0

                                                        vif2.0

[iyunv@node1 ~]# ip netns delete r1
[iyunv@node1 ~]# ip netns delete r2
[iyunv@node1 ~]# ip netns list
[iyunv@node1 ~]# ip netns add r1
[iyunv@node1 ~]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-ex state UP mode DEFAULT qlen 1000
    link/ether 00:0c:29:50:27:c4 brd ff:ff:ff:ff:ff:ff
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT
    link/ether 52:54:00:43:0d:84 brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 500
    link/ether 52:54:00:43:0d:84 brd ff:ff:ff:ff:ff:ff
5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 00:0c:29:50:27:c4 brd ff:ff:ff:ff:ff:ff
6: br-in: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 1a:39:68:8c:f1:ea brd ff:ff:ff:ff:ff:ff
10: vif1.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-in state UNKNOWN mode DEFAULT qlen 500
    link/ether 6a:02:16:30:9c:82 brd ff:ff:ff:ff:ff:ff
11: vif2.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-in state UNKNOWN mode DEFAULT qlen 500
    link/ether 1a:39:68:8c:f1:ea brd ff:ff:ff:ff:ff:ff
[iyunv@node1 ~]# ip link add rinr type veth peer name rins   
[iyunv@node1 ~]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-ex state UP mode DEFAULT qlen 1000
    link/ether 00:0c:29:50:27:c4 brd ff:ff:ff:ff:ff:ff
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT
    link/ether 52:54:00:43:0d:84 brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 500
    link/ether 52:54:00:43:0d:84 brd ff:ff:ff:ff:ff:ff
5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 00:0c:29:50:27:c4 brd ff:ff:ff:ff:ff:ff
6: br-in: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 1a:39:68:8c:f1:ea brd ff:ff:ff:ff:ff:ff
10: vif1.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-in state UNKNOWN mode DEFAULT qlen 500
    link/ether 6a:02:16:30:9c:82 brd ff:ff:ff:ff:ff:ff
11: vif2.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-in state UNKNOWN mode DEFAULT qlen 500
    link/ether 1a:39:68:8c:f1:ea brd ff:ff:ff:ff:ff:ff
12: rins@rinr: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 7e:d4:16:e7:22:cc brd ff:ff:ff:ff:ff:ff
13: rinr@rins: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether da:cf:c3:b5:7b:8a brd ff:ff:ff:ff:ff:ff

[iyunv@node1 ~]# ip link set rinr up
[iyunv@node1 ~]# ip link set rins up
[iyunv@node1 ~]# brctl addif br-in rins
[iyunv@node1 ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
br-ex           8000.000c295027c4       no              eno16777736
br-in           8000.1a39688cf1ea       no              rins
                                                        vif1.0
                                                        vif2.0
[iyunv@node1 ~]# ip link set rinr netns r1

#R1网卡改名
[iyunv@node1 ~]# ip netns exec r1 ifconfig -a
lo: flags=8<LOOPBACK>  mtu 65536
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

rinr: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether da:cf:c3:b5:7b:8a  txqueuelen 1000  (Ethernet)
        RX packets 8  bytes 648 (648.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[iyunv@node1 ~]# ip netns exec r1 ip link set rinr name eth0
[iyunv@node1 ~]# ip netns exec r1 ifconfig -a
eth0: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether da:cf:c3:b5:7b:8a  txqueuelen 1000  (Ethernet)
        RX packets 8  bytes 648 (648.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=8<LOOPBACK>  mtu 65536
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
[iyunv@node1 ~]# ip netns exec r1 ip link set eth0 up
[iyunv@node1 ~]# ip netns exec r1 ip link show
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
13: eth0@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
    link/ether da:cf:c3:b5:7b:8a brd ff:ff:ff:ff:ff:ff link-netnsid 0
[iyunv@node1 ~]# ip netns exec r1 ifconfig eth0 10.0.1.254/24 up
[iyunv@node1 ~]# ip netns exec r1 ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.1.254  netmask 255.255.255.0  broadcast 10.0.1.255
        inet6 fe80::d8cf:c3ff:feb5:7b8a  prefixlen 64  scopeid 0x20<link>
        ether da:cf:c3:b5:7b:8a  txqueuelen 1000  (Ethernet)
        RX packets 8  bytes 648 (648.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 16  bytes 1296 (1.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

#配置第一台虚拟机IP
$ sudo su -
# ifconfig eth0 10.0.1.1/24 up
# ping 10.0.1.254
PING 10.0.1.254 (10.0.1.254): 56 data bytes
64 bytes from 10.0.1.254: seq=0 ttl=64 time=159.302 ms
64 bytes from 10.0.1.254: seq=1 ttl=64 time=5.182 ms
# route add default gw 10.0.1.254

#配置第二台虚拟机IP
$ sudo su -
# ifconfig eth0 10.0.1.2/24 up
# ping 10.0.1.254
PING 10.0.1.254 (10.0.1.254): 56 data bytes
64 bytes from 10.0.1.254: seq=0 ttl=64 time=11.109 ms
64 bytes from 10.0.1.254: seq=1 ttl=64 time=4.452 ms
--- 10.0.1.254 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 4.452/7.780/11.109 ms
# route add default gw 10.0.1.254

[iyunv@node1 ~]# ip link add rexr type veth peer name rexs
[iyunv@node1 ~]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-ex state UP mode DEFAULT qlen 1000
    link/ether 00:0c:29:50:27:c4 brd ff:ff:ff:ff:ff:ff
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT
    link/ether 52:54:00:43:0d:84 brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 500
    link/ether 52:54:00:43:0d:84 brd ff:ff:ff:ff:ff:ff
5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 00:0c:29:50:27:c4 brd ff:ff:ff:ff:ff:ff
6: br-in: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 1a:39:68:8c:f1:ea brd ff:ff:ff:ff:ff:ff
10: vif1.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-in state UNKNOWN mode DEFAULT qlen 500
    link/ether 6a:02:16:30:9c:82 brd ff:ff:ff:ff:ff:ff
11: vif2.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-in state UNKNOWN mode DEFAULT qlen 500
    link/ether 1a:39:68:8c:f1:ea brd ff:ff:ff:ff:ff:ff
12: rins@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-in state UP mode DEFAULT qlen 1000
    link/ether 7e:d4:16:e7:22:cc brd ff:ff:ff:ff:ff:ff link-netnsid 0
14: rexs@rexr: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether c2:6a:a0:96:0e:27 brd ff:ff:ff:ff:ff:ff
15: rexr@rexs: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether ce:6d:e3:9e:d1:1c brd ff:ff:ff:ff:ff:ff
[iyunv@node1 ~]# brctl addif br-ex rexs
[iyunv@node1 ~]# ip link set rexs up
[iyunv@node1 ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
br-ex           8000.000c295027c4       no              eno16777736
                                                        rexs
br-in           8000.1a39688cf1ea       no              rins
                                                        vif1.0
                                                        vif2.0

[iyunv@node1 ~]# ip link set rexr netns r1
[iyunv@node1 ~]# ip netns exec r1 ip link set rexr name eth1
[iyunv@node1 ~]# ip netns exec r1 ifconfig eth1 192.168.1.134/24 up
[iyunv@node1 ~]# ip netns exec r1 ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.1.254  netmask 255.255.255.0  broadcast 10.0.1.255
        inet6 fe80::d8cf:c3ff:feb5:7b8a  prefixlen 64  scopeid 0x20<link>
        ether da:cf:c3:b5:7b:8a  txqueuelen 1000  (Ethernet)
        RX packets 27  bytes 2174 (2.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 32  bytes 2696 (2.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.134  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::cc6d:e3ff:fe9e:d11c  prefixlen 64  scopeid 0x20<link>
        ether ce:6d:e3:9e:d1:1c  txqueuelen 1000  (Ethernet)
        RX packets 33  bytes 4560 (4.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
[iyunv@node1 ~]# ip netns exec r1 ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=1.72 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.453 ms

#测试虚拟机网络
[iyunv@node1 ~]# tcpdump -i vif1.0 -nn  icmp
tcpdump: WARNING: vif1.0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vif1.0, link-type EN10MB (Ethernet), capture size 65535 bytes
09:43:35.448999 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 186, length 64
09:43:36.456217 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 187, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel

[iyunv@node1 ~]# tcpdump -i rins -nn  icmp
tcpdump: WARNING: rins: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on rins, link-type EN10MB (Ethernet), capture size 65535 bytes
09:43:06.250500 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 157, length 64
09:43:07.257803 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 158, length 64
09:43:08.265838 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 159, length 64
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

[iyunv@node1 ~]# ip netns exec r1 tcpdump -i eth0 -nn icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
09:45:20.239901 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 290, length 64
09:45:21.246984 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 291, length 64
09:45:22.254115 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 292, length 64
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

[iyunv@node1 ~]# ip netns exec r1 tcpdump -i eth1 -nn icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
09:46:18.629898 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 348, length 64
09:46:19.637394 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 349, length 64
09:46:20.644763 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 43520, seq 350, length 64
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

#从192.168.1.1上抓包
[iyunv@DNS-Server ~]# tcpdump -i eth0 -nn icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
09:53:52.872389 IP 192.168.1.1 > 114.114.114.114: ICMP 192.168.1.1 udp port 16588 unreachable, length 66
09:53:52.893624 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 44288, seq 5, length 64
09:53:52.893646 IP 192.168.1.1 > 10.0.1.1: ICMP echo reply, id 44288, seq 5, length 64
09:53:53.901275 IP 10.0.1.1 > 192.168.1.1: ICMP echo request, id 44288, seq 6, length 64
09:53:53.901299 IP 192.168.1.1 > 10.0.1.1: ICMP echo reply, id 44288, seq 6, length 64
^C
5 packets captured
5 packets received by filter
0 packets dropped by kernel

#在R1上添加iptables NAT路由转发
[iyunv@node1 ~]# ip netns exec r1 iptables -t nat -A POSTROUTING -s 10.0.1.0/24 ! -d 10.0.1.0/24 -j SNAT --to-source 192.168.1.134[iyunv@node1 ~]# ip netns exec r1 iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
SNAT       all  --  10.0.1.0/24         !10.0.1.0/24          to:192.168.1.134

测试网络成功

再次抓包
[iyunv@node1 ~]# tcpdump -i rins -nn  icmp
tcpdump: WARNING: rins: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on rins, link-type EN10MB (Ethernet), capture size 65535 bytes
10:00:11.768476 IP 10.0.1.1 > 192.168.1.132: ICMP echo request, id 45312, seq 5, length 64
10:00:11.769866 IP 192.168.1.132 > 10.0.1.1: ICMP echo reply, id 45312, seq 5, length 64
10:00:12.781633 IP 10.0.1.1 > 192.168.1.132: ICMP echo request, id 45312, seq 6, length 64
10:00:12.782421 IP 192.168.1.132 > 10.0.1.1: ICMP echo reply, id 45312, seq 6, length 64
^C
4 packets captured
4 packets received by filter
0 packets dropped by kernel

#在R1上配置dhcp
[iyunv@node1 ~]# ip netns exec r1 dnsmasq --dhcp-range 10.0.1.100,10.0.1.120

#虚拟机自动获得IP地址
# udhcpc -R
udhcpc (v1.18.5) started
Sending discover...
Sending select for 10.0.1.120...
Lease of 10.0.1.120 obtained, lease time 3600
deleting routers
route: SIOCDELRT: No such process
adding dns 10.0.1.254

# ifconfig
eth0      Link encap:Ethernet  HWaddr 52:54:00:AA:BB:DD  
          inet addr:10.0.1.120  Bcast:10.0.1.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:feaa:bbdd/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:29 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2838 (2.7 KiB)  TX bytes:3476 (3.3 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.1.254      0.0.0.0         UG    0      0        0 eth0
10.0.1.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0

[iyunv@node1 ~]# ip netns exec r1 killall dnsmasq
[iyunv@node1 ~]# ip netns exec r1 dnsmasq -F 10.0.1.151,10.0.1.160 --dhcp-option=option:router,10.0.1.254
# udhcpc -R
udhcpc (v1.18.5) started
Sending discover...
Sending select for 10.0.1.158...
Lease of 10.0.1.158 obtained, lease time 3600
deleting routers
route: SIOCDELRT: No such process
adding dns 10.0.1.254
# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.1.254      0.0.0.0         UG    0      0        0 eth0
10.0.1.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0



运维网声明 1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表,该帖子作者与运维网享有帖子相关版权
3、所有作品的著作权均归原作者享有,请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意,请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息,一经发现立即删除。若您因此触犯法律,一切后果自负,我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责,亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏,不得用于商业或者其他用途,否则,一切后果均由您自己承担,我们对此不承担任何法律责任
7、如涉及侵犯版权等问题,请您及时通知我们,我们将立即采取措施予以解决
8、联系人Email:admin@iyunv.com 网址:www.yunweiku.com

所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其承担任何法律责任,如涉及侵犯版权等问题,请您及时通知我们,我们将立即处理,联系人Email:kefu@iyunv.com,QQ:1061981298 本贴地址:https://www.yunweiku.com/thread-323060-1-1.html 上篇帖子: kvm tboot和libvirt的安装 下篇帖子: 搭建KVM虚拟化平台 虚拟机 技术 网络
累计签到:26 天
连续签到:1 天
发表于 2017-1-4 15:46:30 | 显示全部楼层
很好,学习了!

运维网声明 1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表,该帖子作者与运维网享有帖子相关版权
3、所有作品的著作权均归原作者享有,请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意,请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息,一经发现立即删除。若您因此触犯法律,一切后果自负,我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责,亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏,不得用于商业或者其他用途,否则,一切后果均由您自己承担,我们对此不承担任何法律责任
7、如涉及侵犯版权等问题,请您及时通知我们,我们将立即采取措施予以解决
8、联系人Email:admin@iyunv.com 网址:www.yunweiku.com

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

扫码加入运维网微信交流群X

扫码加入运维网微信交流群

扫描二维码加入运维网微信交流群,最新一手资源尽在官方微信交流群!快快加入我们吧...

扫描微信二维码查看详情

客服E-mail:kefu@iyunv.com 客服QQ:1061981298


QQ群⑦:运维网交流群⑦ QQ群⑧:运维网交流群⑧ k8s群:运维网kubernetes交流群


提醒:禁止发布任何违反国家法律、法规的言论与图片等内容;本站内容均来自个人观点与网络等信息,非本站认同之观点.


本站大部分资源是网友从网上搜集分享而来,其版权均归原作者及其网站所有,我们尊重他人的合法权益,如有内容侵犯您的合法权益,请及时与我们联系进行核实删除!



合作伙伴: 青云cloud

快速回复 返回顶部 返回列表