目录

  1. VolumeDeviceNotFound
    1. 解决方法
  2. iSCSI Login negotiation failed
    1. 解决方法

VolumeDeviceNotFound

实例Hard Reboot后，出现VolumeDeviceNotFound的错误。

查看计算节点上，nova-compute的日志。 

1
nova.compute.manager [req-1c2e36df-c5ff-49aa-9893-6046cd0cbfeb b29311c34edb4f179092e81cbee10682 8e4be2cb25084defb5a9f0128afd5291 - default default] [instance: d17a8b0f-759c-4f54-b265-64fc16986710] Cannot reboot instance: Volume device not found at .: os_brick.exception.VolumeDeviceNotFound: Volume device not found at .

解决方法

  1. 确定实例挂载卷的ID。

    1
    2
    3
    4
    5
    6
    7
    openstack server show d17a8b0f-759c-4f54-b265-64fc16986710 -c volumes_attached
    +------------------+-------------------------------------------+
    | Field            | Value                                     |
    +------------------+-------------------------------------------+
    | volumes_attached | id='f870fcdb-209c-4877-8de4-1b7683b98548' |
    +------------------+-------------------------------------------+

  2. 存储节点查看对应的物理设备是否被创建。

    1
    2
    3
    4
    # f870fcdb-209c-4877-8de4-1b7683b98548是volume的ID。
    lsblk | grep f870fcdb--209c--4877--8de4--1b7683b98548
    ├─cinder--volumes-volume--f870fcdb--209c--4877--8de4--1b7683b98548 252:101  0   150G  0 lvm  
    ├─cinder--volumes-volume--f870fcdb--209c--4877--8de4--1b7683b98548 252:101  0   150G  0 lvm

    卷对应物理设备就是。

    1
    ls /dev/cinder-volumes/volume-f870fcdb-209c-4877-8de4-1b7683b98548

  3. 在存储节点上查看iSCSI资源。

    1
    targetcli ls

    得到下面的结果。

    1
    2
    3
    4
    5
    6
    7
    8
    | o- iqn.2010-10.org.openstack:volume-f870fcdb-209c-4877-8de4-1b7683b98548 ... [TPGs: 1]
    | | o- tpg1 ................................................ [no-gen-acls, auth per-acl]
    | |   o- acls ................................................................ [ACLs: 1]
    | |   | o- iqn.2020-08.org.openanolis:8a5ea81155a ......... [1-way auth, Mapped LUNs: 0]
    | |   o- luns ................................................................ [LUNs: 0]
    | |   o- portals .......................................................... [Portals: 1]
    | |     o- 10.10.15.14:3260 ....................................................... [OK]

    从上面的结果可以看到acls（ACL，Access Control Lists，访问控制列表）是正常的，但luns（LUN，Logical Unit Number，逻辑单元号）中没有数据。

  4. 新建backstores

    1
    2
    3
    4
    targetcli \
    /backstores/block create \ 
    name=iqn.2010-10.org.openstack:volume-f870fcdb-209c-4877-8de4-1b7683b98548 \
    dev=/dev/cinder-volumes/volume-f870fcdb-209c-4877-8de4-1b7683b98548

  5. 新建lun

    1
    2
    3
    4
    targetcli \
    /iscsi/iqn.2010-10.org.openstack:volume-f870fcdb-209c-4877-8de4-1b7683b98548/tpg1/luns \ 
    create  \
    /backstores/block/iqn.2010-10.org.openstack:volume-f870fcdb-209c-4877-8de4-1b7683b98548

    也可以使用cinder-rtstool，更加方便。

    1
    cinder-rtstool create /backstores/block/iqn.2010-10.org.openstack:volume-f870fcdb-209c-4877-8de4-1b7683b98548

  6. 再次查看，已经正常。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    | o- iqn.2010-10.org.openstack:volume-f870fcdb-209c-4877-8de4-1b7683b98548 ............................... [TPGs: 1]
    | | o- tpg1 ............................................................................ [no-gen-acls, auth per-acl]
    | |   o- acls ............................................................................................ [ACLs: 1]
    | |   | o- iqn.2020-08.org.openanolis:8a5ea81155a ..................................... [1-way auth, Mapped LUNs: 1]
    | |   |   o- mapped_lun0 ... [lun0 block/iqn.2010-10.org.openstack:volume-f870fcdb-209c-4877-8de4-1b7683b98548 (rw)]
    | |   o- luns ............................................................................................ [LUNs: 1]
    | |   | o- lun0  [block/iqn.2010-10.org.openstack:volume-f870fcdb-209c-4877-8de4-1b7683b98548 (/dev/cinder-volumes/volume-f870fcdb-209c-4877-8de4-1b7683b98548) (default_tg_pt_gp)]
    | |   o- portals ...................................................................................... [Portals: 1]
      |   | o- 10.10.15.14:3260 ................................................................................... [OK]

  7. 最后重启实例。

    1
    openstack server reboot --hard ce979e29-198d-498b-b2e3-c0cab6c82c98

简单脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
volume=$1
if [[ -z vlolume ]]; then
  echo "empty volume id"
  exit 1
fi

targetcli \
/backstores/block create \ 
name=iqn.2010-10.org.openstack:volume-${volume} \
dev=/dev/cinder-volumes/volume-${volume}

targetcli \
/iscsi/iqn.2010-10.org.openstack:volume-${volume}/tpg1/luns \ 
create \
/backstores/block/iqn.2010-10.org.openstack:volume-${volume}

iSCSI Login negotiation failed

volume删除后，存储节点的日志里依旧会报错。

1
2
Jan 17 10:14:22 compute3 kernel: Unable to locate Target IQN: iqn.2010-10.org.openstack:volume-c6774132-65f2-47a0-a65b-9a5d06a4accb in Storage Node
Jan 17 10:14:22 compute3 kernel: iSCSI Login negotiation failed.

解决方法

到各个节点中执行iscsiadm命令，查看是否存在相应的session。

1
2
3
4
5
6
7
[root@compute2 ~] iscsiadm -m session -P 3 | grep c6774132
Target: iqn.2010-10.org.openstack:volume-c6774132-65f2-47a0-a65b-9a5d06a4accb (non-flash)

# 将session退出
[root@compute2 ~] iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c6774132-65f2-47a0-a65b-9a5d06a4accb -p 10.10.15.14:3260 --logout
Logging out of session [sid: 248, target: iqn.2010-10.org.openstack:volume-c6774132-65f2-47a0-a65b-9a5d06a4accb, portal: 10.10.15.14,3260]
Logout of [sid: 248, target: iqn.2010-10.org.openstack:volume-c6774132-65f2-47a0-a65b-9a5d06a4accb, portal: 10.10.15.14,3260] successful.
1
2
3
4
5
6
7
8
9
10
set -x -e
volume_id=9ef6d658-104a-4b59-9c20-3046dd8dfc9a
auth_user=XeL9sNhAKNFumZ7Nmai5
auth_password=FWcRnkPJ9XUF7qTC
#initiator_name=iqn.2016-04.com.open-iscsi:c81e0b7d0c7
initiator_name=iqn.2016-04.com.open-iscsi:928262e4d53
cinder-rtstool create /dev/cinder-volumes/volume-$volume_id \
iqn.2010-10.org.openstack:volume-$volume_id $auth_user $auth_password False -p3260 -a10.10.15.14

cinder-rtstool add-initiator iqn.2010-10.org.openstack:volume-$volume_id $auth_user $auth_password $initiator_name
1
2
3
4
5
6
iscsiadm -m discovery -t sendtargets -p 10.10.15.14

target_iqn


iscsiadm -m node -T iqn.2010-10.org.openstack:volume-9ef6d658-104a-4b59-9c20-3046dd8dfc9a -p 10.10.15.14 -l

编辑/var/lib/iscsi/nodes/iqn.2010-10.org.openstack:volume-9ef6d658-104a-4b59-9c20-3046dd8dfc9a/10.10.15.14,3260,1/default 

1
2
3
node.session.auth.authmethod = CHAP
node.session.auth.username = XeL9sNhAKNFumZ7Nmai5
node.session.auth.password = FWcRnkPJ9XUF7qTC
1
systemctl restart iscsi.service