Linux: Multipath Configuration In Linux

For this post I have setup an HPE DL380 Gen 10 server with two single port QLogic FC HBAs (Host Bus Adapter). This server is connected to 3PAR Storage through FC cables via SAN Switch. 

Loading QLogic Driver Modules

The QLogic HBA is shown in the OS (CentOS 7) as shown below.

 [root@localhost ~]# lspci -nn | grep -i fibre  
 12:00.0 Fibre Channel [0c04]: QLogic Corp. Device [1077:2261] (rev 01)  
 13:00.0 Fibre Channel [0c04]: QLogic Corp. Device [1077:2261] (rev 01)  

To load the QLogic kernel module we need to run the following command. The "-v" option is for verbose output.

 [root@localhost ~]# modprobe -v qla2xxx   

We can verify that the qla2xxx module is loaded as shown below.

 [root@localhost ~]# modinfo qla2xxx | grep version  
 version:    8.07.00.28.07.0-k1  
 srcversion:   3B54D31D167574AF88D5492  
 vermagic:    3.10.0-123.el7.x86_64 SMP mod_unload modversions   

The module is also listed on lsmod output.

 [root@localhost ~]# lsmod | grep qla  
 qla2xxx        652312 8   
 scsi_transport_fc   59960 1 qla2xxx  

The two hosts host0 and host3, displayed in the first output below, are the two QLogic FC adapters with the WWNN (World Wide Node Names) displayed in the second output.

 [root@localhost ~]# ls /sys/class/fc_host/  
 host0 host3  

 [root@localhost ~]# cat /sys/class/fc_host/host*/node_name  
 0x51402ec012cf8387  
 0x51402ec012cf8371  

Scanning FC Hosts

After the driver modules were loaded, the ports on SAN Switch connected to the server were all green and the server FC Host WWNN and WWPN were shown in the SAN Switch. Then zoning was configured and a 10 GB LUN was exported to the server. I have not included the commands of zoning and LUN creation done on SAN Switch and 3PAR storage respectively.

On the server we need to scan the FC Hosts for the LUNs to be present. The "- - -" are wild cards in the format "Channel Target LUN". These wild cards are used to rescan everything.

 [root@localhost ~]# echo "1" > /sys/class/fc_host/host0/issue_lip
 [root@localhost ~]# echo "- - -" > /sys/class/scsi_host/host0/scan
 [root@localhost ~]# echo "1" > /sys/class/fc_host/host3/issue_lip
 [root@localhost ~]# echo "- - -" > /sys/class/scsi_host/host3/scan

After rescanning the hosts the LUN is present and the output of lsblk shows four devices /dev/sdc, /dev/sdd, /dev/sde and /dev/sdf. These four devices represent the four FC paths.

 [root@localhost ~]# lsblk  
 NAME      MAJ:MIN RM SIZE RO TYPE MOUNTPOINT  
 sda        8:0  1 1.1T 0 disk   
 ├─sda1      8:1  1 200M 0 part /boot/efi  
 ├─sda2      8:2  1 500M 0 part /boot  
 └─sda3      8:3  1 1.1T 0 part   
  ├─centos-swap 253:0  0  4G 0 lvm  [SWAP]  
  ├─centos-root 253:1  0  50G 0 lvm  /  
  └─centos-home 253:2  0  1T 0 lvm  /home  
 sdc        8:32  0  10G 0 disk      
 sdd        8:48  0  10G 0 disk     
 sde        8:64  0  10G 0 disk     
 sdf        8:80  0  10G 0 disk     

Configuring Multipath

To enable multipath with user friendly names for the LUNs, we run the following command.

 [root@localhost ~]# mpathconf --enable --user_friendly_names y  

Then we add the option path_selector "round-robin 0" in the /etc/multipath.conf file. The default algorithm for path selection will be round-robin.

 defaults {  
     user_friendly_names yes  
     find_multipaths yes  
     path_selector "round-robin 0"  
 }  

To configure multipath we run the following command.

 [root@localhost ~]# multipath  

Now we list the multipath devices. Our LUN is present as mpatha as we have specified user_friendly_names to be used for LUNs and has sdc, sdd, sde and sdf as slaves.

 [root@localhost ~]# multipath -ll  
 mpatha (360002ac000000000000002180001cf1f) dm-3 3PARdata,VV         
 size=10G features='0' hwhandler='0' wp=rw  
 `-+- policy='round-robin 0' prio=1 status=active  
  |- 0:0:0:0 sdc 8:32 active ready running  
  |- 0:0:1:0 sdd 8:48 active ready running  
  |- 3:0:0:0 sde 8:64 active ready running  
  `- 3:0:1:0 sdf 8:80 active ready running  

Reloading and restarting the multipathd service needs to be done for the changes to be reflected. The enable option starts the service permanently on reboot.

 [root@localhost ~]# systemctl reload multipathd.service   
 [root@localhost ~]# systemctl restart multipathd.service
 [root@localhost ~]# systemctl enable multipathd.service  

Now the four devices are listed as part of mpatha.

 [root@localhost ~]# lsblk  
 NAME      MAJ:MIN RM SIZE RO TYPE MOUNTPOINT  
 sda        8:0  1 1.1T 0 disk   
 ├─sda1      8:1  1 200M 0 part /boot/efi  
 ├─sda2      8:2  1 500M 0 part /boot  
 └─sda3      8:3  1 1.1T 0 part   
  ├─centos-swap 253:0  0  4G 0 lvm  [SWAP]  
  ├─centos-root 253:1  0  50G 0 lvm  /  
  └─centos-home 253:2  0  1T 0 lvm  /home  
 sdc        8:32  0  10G 0 disk   
 └─mpatha    253:3  0  10G 0 mpath   
 sdd        8:48  0  10G 0 disk   
 └─mpatha    253:3  0  10G 0 mpath   
 sde        8:64  0  10G 0 disk   
 └─mpatha    253:3  0  10G 0 mpath   
 sdf        8:80  0  10G 0 disk   
 └─mpatha    253:3  0  10G 0 mpath   

Checking the device details of /dev/mapper/mpatha using fdisk.

 [root@localhost ~]# fdisk -l /dev/mapper/mpatha  
 Disk /dev/mapper/mpatha: 10.7 GB, 10737418240 bytes, 20971520 sectors  
 Units = sectors of 1 * 512 = 512 bytes  
 Sector size (logical/physical): 512 bytes / 512 bytes  
 I/O size (minimum/optimal): 16384 bytes / 16777216 bytes  

Other details such as multipath device name and and the slaves of the device can be viewed as given below.

 [root@localhost ~]# cat /sys/block/dm-3/dm/name   
 mpatha  
 [root@localhost ~]# ls /sys/block/dm-3/slaves/  
 sdc sdd sde sdf