ISCSI网络存储服务
实验说明：此处只用一个SAN设备与一个WWW1节点做实验步骤www1 eth0:192.168.1.1
SAN eth0:192.168.1.2 添加5块为4G的硬盘
实验步骤：
5.1 iscsi服务器上所做的配置
1、在SAN服务器上添加5块硬盘
配置eth0网卡
# vim /etc/sysconfig/network-scripts/ifcfg-eth0
# Advanced Micro Devices 79c970
DEVICE=eth0
BOOTPROTO=static
HWADDR=00:0C:29:4C:BD:42
ONBOOT=yes
IPADDR=192.168.1.1
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
# service network restart
首先使用“fdisk”命令在每块硬盘上创建一个分区
# fdisk /dev/sdb
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n //创建
Command action
e extended
p primary partition (1-4)
p //分区
Partition number (1-4): 1 //第一块
First cylinder (1-522, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-522, default 522):
Using default value 522
Command (m for help): t //类型
Selected partition 1
Hex code (type L to list codes): fd //raid类型
Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): p //查看
Disk /dev/sdb: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 522 4192933+ fd Linux raid autodetect
Command (m for help): w //保存
# fdisk /dev/sdc
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n //创建
Command action
e extended
p primary partition (1-4)
p //分区
Partition number (1-4): 1 //第一块
First cylinder (1-522, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-522, default 522):
Using default value 522
Command (m for help): t //类型
Selected partition 1
Hex code (type L to list codes): fd //raid类型
Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): p //查看
Disk /dev/sdb: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 522 4192933+ fd Linux raid autodetect
Command (m for help): w //保存
# fdisk /dev/sdd
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n //创建
Command action
e extended
p primary partition (1-4)
p //分区
Partition number (1-4): 1 //第一块
First cylinder (1-522, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-522, default 522):
Using default value 522
Command (m for help): t //类型
Selected partition 1
Hex code (type L to list codes): fd //raid类型
Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): p //查看
Disk /dev/sdb: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 522 4192933+ fd Linux raid autodetect
Command (m for help): w //保存
# fdisk /dev/sde
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n //创建
Command action
e extended
p primary partition (1-4)
p //分区
Partition number (1-4): 1 //第一块
First cylinder (1-522, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-522, default 522):
Using default value 522
Command (m for help): t //类型
Selected partition 1
Hex code (type L to list codes): fd //raid类型
Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): p //查看
Disk /dev/sdb: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 522 4192933+ fd Linux raid autodetect
Command (m for help): w //保存
# fdisk /dev/sdf
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n //创建
Command action
e extended
p primary partition (1-4)
p //分区
Partition number (1-4): 1 //第一块
First cylinder (1-522, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-522, default 522):
Using default value 522
Command (m for help): t //类型
Selected partition 1
Hex code (type L to list codes): fd //raid类型
Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): p //查看
Disk /dev/sdb: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 522 4192933+ fd Linux raid autodetect
Command (m for help): w //保存
创建软raid 磁盘阵列
# mdadm -Cv /dev/md5 -a yes -n4 -l5 -x1 /dev/sd1 //创建软raid，-a:对应设备不存在责自动创建，-n:使用几块硬盘，-l:使用的raid级别是5，-x:有几块硬盘作为备份用
mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 64K
mdadm: size set to 4192832K
mdadm: array /dev/md5 started.
将创建好的raid应用到逻辑卷
# pvcreate /dev/md5 //创建物理卷
Physical volume "/dev/md5" successfully created
# vgcreate vol0 /dev/md5 //创建卷组
/dev/cdrom: open failed: 只读文件系统
/dev/cdrom: open failed: 只读文件系统
Attempt to close device '/dev/cdrom' which is not open.
/dev/cdrom: open failed: 只读文件系统
Attempt to close device '/dev/cdrom' which is not open.
/dev/cdrom: open failed: 只读文件系统
Attempt to close device '/dev/cdrom' which is not open.
Volume group "vol0" successfully created
# lvcreate -L 4G -n lv-host1 vol0//创建逻辑卷，-L：指定创建磁盘大小，-n:新创建的逻辑卷名字，vol0：卷组名字
/dev/cdrom: open failed: 只读文件系统
Logical volume "lv-host1" created
# lvcreate -L 4G -n lv-host2 vol0
/dev/cdrom: open failed: 只读文件系统
Logical volume "lv-host2" created
# lvdisplay /dev/vol0/lv-host1 /dev/vol0/lv-host2 //查看创建成功的逻辑卷
为了重建或恢复软raid方便，将此阵列配置保存为/etc/mdadm.conf，此文件默认不错在，手动创建
# mdadm -D -s > /etc/mdadm.conf
# vi /etc/mdadm.conf
ARRAY /dev/md5 level=raid5 num-devices=4 metadata=0.90 spares=1 UUID=325a6875:fe114112:ef89899e:2459e80b auto=yes//开机自动加载软raid
安装iscsi服务器也称为Target端，负责定义存储对象，安装相关工具,对应的系统服务为tgtd,命令行管理工具为tgtadm
# mount /dev/cdrom /media/
mount: block device /dev/cdrom is write-protected, mounting read-only
# vim /etc/yum.repos.d/rhel-debuginfo.repo //编辑下列两处
name=Red Hat Enterprise Linux $releasever - $basearch - Debug
baseurl=file:///media/ClusterStorage
enabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
name=Red Hat Enterprise Linux $releasever Beta - $basearch - Debug
baseurl=file:///media/Server
enabled=1
gpgcheck=0
# yum -y install scsi-target-utils //安装iscsi定义工具
2、创建iscsi对象（target）
# service tgtd start //启动tgtd服务
Starting SCSI target daemon: Starting target framework daemon
# netstat -anpl |grep tgtd //查看tgtd端口
tcp 0 0 0.0.0.0:3260 0.0.0.0:* LISTEN 5501/tgtd
tcp 0 0 :::3260 :::* LISTEN 5501/tgtd
unix 2 [ ACC ] STREAM LISTENING 18142 5501/tgtd /tmp/.TGT_IPC_ABSTRACT_NAMESPACE
unix 2 [ ] DGRAM 18135 5501/tgtd
创建两个iscsi对象
# tgtadm -L iscsi -o new -m target -t 1 -T iqn.2011-5-11:lvm1 //-L：设备类型，-o:目标动作，是drop、new、delete，-m:管理目标（target服务器，logicaalunit客户机）-t：创建设备的ID号，-T：创建设备的名字
# tgtadm -L iscsi -o new -m target -t 2 -T iqn.2011-5-11:lvm2//创建一个ID号为2，名字为iqn.2011-5-11的iscsi存储设备
# tgtadm -L iscsi -o delete -m target -t 3 //删除一个iscsi设备
# tgtadm -L iscsi -o show -m target //查看本机创建的iscsi设备
Target 1: iqn.2011-5-11:lvm1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: None
Account information:
ACL information:
Target 2: iqn.2011-5-11:lvm2
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00020000
SCSI SN: beaf20
Size: 0 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: None
Account information:
ACL information:
3、为客户机分配存储空间（LUN）
将逻辑卷lv-host1和lv-host2分别挂在到tid为1、2的target下，指派给客户机192.168.1.0/24网段的客户机使用，
# tgtadm -L iscsi -o new -m logicalunit -t 1 -l 1 -b /dev/vol0/lv-host1 //-l:指定LUN的ID号，表示第一个LUN。-b:指定设备位置
# tgtadm -L iscsi -o new -m logicalunit -t 2 -l 1 -b /dev/vol0/lv-host2 //表示将/dev/vol0/lv-host1分区应用到tid为2的iscsi（target服务器）上，-m logicalunit:表示管理目标为LUN，-l:表示在tid号为1上创建的第一个LUN
# tgtadm -L iscsi -o bind -m target -t 1 -I 192.168.1.0/24//允许哪些客户机使用这个tid号为1的iscsi设备。-o bind：绑定客户机地址，
# tgtadm -L iscsi -o bind -m target -t 2 -I 192.168.1.0/24
# history |tail -12 >> /etc/rc.local//将target和LUN配置追加到/etc/rc.local开机自动执行此脚本
# vim /etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
tgtadm -L iscsi -o new -m target -t 1 -T iqn.2011-5-11:lvm1 //删除其余行，只留此6行
tgtadm -L iscsi -o new -m target -t 2 -T iqn.2011-5-11:lvm2
tgtadm -L iscsi -o new -m logicalunit -t 1 -l 1 -b /dev/vol0/lv-host1
tgtadm -L iscsi -o new -m logicalunit -t 2 -l 1 -b /dev/vol0/lv-host2
tgtadm -L iscsi -o bind -m target -t 1 -I 192.168.1.0/24
tgtadm -L iscsi -o bind -m target -t 2 -I 192.168.1.0/24
# tgtadm -L iscsi -o show -m target //查看target,发现LUN类型为disk
Target 1: iqn.2011-5-11:lvm1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: None
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 4295 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dev/vol0/lv-host1
Account information:
ACL information:
192.168.1.0/24
Target 2: iqn.2011-5-11:lvm2
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00020000
SCSI SN: beaf20
Size: 0 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: None
LUN: 1
Type: disk
SCSI ID: IET 00020001
SCSI SN: beaf21
Size: 4295 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dev/vol0/lv-host2
Account information:
ACL information:
192.168.1.0/24
4、保存target和LUN配置
修改/etc/tgt/targets.conf配置文件，将target、LUN信息保存为静态配置，使重启tgtd服务后仍有效
# vim /etc/tgt/targets.conf
在文件最后添加如下行
<target iqn.2011-5-11:lvm1>
backing-store /dev/vol0/lv-host1
initiator-address 192.168.1.0/24
</target>
<target iqn.2011-5-11:lvm2>
backing-store /dev/vol0/lv-host2
initiator-address 192.168.1.0/24
</target>
# chkconfig tgtd on
5.2使用iscsi存储设备（客户机www1）节点
配置客户机eth0网卡
# vim /etc/sysconfig/network-scripts/ifcfg-eth0
# Advanced Micro Devices 79c970
DEVICE=eth0
BOOTPROTO=static
HWADDR=00:0C:29:AB:B0:02
ONBOOT=yes
IPADDR=192.168.1.2
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
Iscsi客户机通常也称为initiator，负责连接iscsi设备，在iscsi上创建文件系统，以及存储数据。安装客户端所需工具
# mount /dev/cdrom /media/
mount: block device /dev/cdrom is write-protected, mounting read-only
# vim /etc/yum.repos.d/rhel-debuginfo.repo
name=Red Hat Enterprise Linux $releasever - $basearch - Debug
baseurl=file:///media/Server
enabled=1
gpgcheck=0
# yum -y install iscsi-initiator-utils lsscsi
1、发现并连接iscsi设置
使用iscsiadm工具之前，先启动iscsi服务。查找iscsi设备，其中 “–m discovery” 表示发现/查找，“-t sendtargets ”表示查询目标（iscsi设备），“-p ip地址”用来指定iscsi服务器的ip地址
# service iscsi restart //启动iscsi服务
Stopping iSCSI daemon: iscsiadm: can not connect to iSCSI daemon (111)!
iscsiadm: initiator reported error (20 - could not connect to iscsid)
iscsiadm: Could not stop iscsid. Trying sending iscsid SIGTERM or SIGKILL signals manually
iscsid 已停 [确定]
Starting iSCSI daemon: [确定]
[确定]
设置 iSCSI 目标：iscsiadm: No records found!
[确定]
# iscsiadm -m discovery -t sendtargets -p 192.168.1.1
192.168.1.1:3260,1 iqn.2011-5-11:lvm1
192.168.1.1:3260,1 iqn.2011-5-11:lvm2
# iscsiadm -m node -T iqn.2011-5-11:lvm1 --login //将上面查找到的地址复制下来即可
Logging in to
Login to : successful
# iscsiadm -m node -T iqn.2011-5-11:lvm2 --login //登陆iscsi成功
Logging in to
Login to : successful
# lsscsi //查看iscsi设备列表，发现已有（IET代表是iscsi设备）
disk VMware, VMware Virtual S 1.0 /dev/sda
storage IET Controller 0001 -
disk IET VIRTUAL-DISK 0001/dev/sdb
storage IET Controller 0001 -
disk IET VIRTUAL-DISK 0001 /dev/sdc
2、使用iscsi存储设备
1）为磁盘分区
# fdisk -l /dev/sdb
Disk /dev/sdb: 4294 MB, 4294967296 bytes
133 heads, 62 sectors/track, 1017 cylinders
Units = cylinders of 8246 * 512 = 4221952 bytes
Disk /dev/sdb doesn't contain a valid partition table //新的iscsi设备无分区列表
# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1017, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1017, default 1017):
Using default value 1017
Command (m for help): p
Disk /dev/sdb: 4294 MB, 4294967296 bytes
133 heads, 62 sectors/track, 1017 cylinders
Units = cylinders of 8246 * 512 = 4221952 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 1017 4193060 83 Linux
Command (m for help): w
# fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1017, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1017, default 1017):
Using default value 1017
Command (m for help): p
Disk /dev/sdc: 4294 MB, 4294967296 bytes
133 heads, 62 sectors/track, 1017 cylinders
Units = cylinders of 8246 * 512 = 4221952 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 1017 4193060 83 Linux
Command (m for help): w
2）创建文件系统（格式化）
# mkfs.ext3 /dev/sdb1
# mkfs.ext3 /dev/sdc1
3）挂载文件系统
# yum -y install http* mysql* //在www1上安装apahce、mysql
# service httpd restart
停止 httpd： [失败]
启动 httpd： [确定]
2、挂载分区
# mkdir /var/www/html/bbs/
# mkdir /var/lib/mysql/
# service mysqld start //启动MYSQL服务
# mysqladmin -uroot password 123 //设置mysql管理员密码
# mkdir /mysql/
# cp -rp /var/lib/mysql/* /mysql/
# rm -rf /var/lib/mysql/* //清空mysql下所有内容挂载
# mount /dev/sdb1 /var/www/html/bbs/
# mount /dev/sdc1 /var/lib/mysql/
# df -hT
文件系统 类型 容量已用 可用 已用% 挂载点
/dev/sda5 ext3 17G 5.1G 11G 33% /
/dev/sda2 ext3 2.0G 36M 1.9G 2% /home
/dev/sda1 ext3 99M 12M 83M 13% /boot
tmpfs tmpfs 252M 0 252M 0% /dev/shm
/dev/sdb1 ext3 4.0G 87M 3.7G 3% /var/www/html/bbs
/dev/sdc1 ext3 4.0G 93M 3.7G 3% /var/lib/mysql
# cd /var/www/html/bbs
# ls
index.htmllost+found //如出现此字，则挂载正确,/var/lib/mysql也是一样
3、保存iscsi连接配置
在某些客户机系统中，有时候会因为设备连接变动、引导顺序等因素影响，导致iscsi存储设备的设备文件异常（可能不在是/dev/sdb1或/dev/sdc1），可以通过系统的ID号，即UUID可进行定位
# chkconfig iscsi on
# blkid /dev/sdb1
/dev/sdb1: UUID="5e554c1b-bfdf-41fb-9420-691d286a6124" SEC_TYPE="ext2" TYPE="ext3"
# blkid /dev/sdc1
/dev/sdc1: UUID="e44f5d1c-a97f-4123-b286-b4708fad1d41" SEC_TYPE="ext2" TYPE="ext3"
# vim /etc/fstab
在文件最后添加如下行
UUID=5e554c1b-bfdf-41fb-9420-691d286a6124 /var/www/html/bbs ext3 defaults,_netdev 0 0
UUID=e44f5d1c-a97f-4123-b286-b4708fad1d41 /var/lib/mysql ext3 defaults,_netdev 0 0
#mount –a
# mount //查看已挂载
/dev/sda5 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/sda2 on /home type ext3 (rw)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sdb1 on /var/www/html/bbs type ext3 (rw,_netdev)
/dev/sdc1 on /var/lib/mysql type ext3 (rw,_netdev)
上传Discuz_7.2_FULL_SC_UTF8.zip论坛软件到/root下
# unzip Discuz_7.2_FULL_SC_UTF8.zip -d discuz
# mv discuz/upload/* /var/www/html/bbs/
# cp -rp /mysql/* /var/lib/mysql/
编辑httpd.conf配置文件
# vim /etc/httpd/conf/httpd.conf
DocumentRoot "/var/www/html/bbs"
AddDefaultCharset gb2312
DirectoryIndex index.php index.html index.html.var
# service httpd restart
停止 httpd： [确定]
启动 httpd： [确定]
谢谢楼主，共同发展 怀揣两块，胸怀500万！ 我的id是假冒的，大家不要相信我是骗子。 天塌下来你顶着，我垫着! 我不在江湖,但江湖中有我的传说。 商女不知亡国恨、妓女不懂婚外情。
页:
[1]