Configure multiple iSCSI ifaces and use multipathing

Objective: To provide a quick overview of the steps needed to bind different NICS to iSCSI ifaces and use those ifaces to setup multipathing.

SCENARIO

SLES 11 SP3 server with 5 NICS (1 on public network, 3 on private network dedicated to iscsi).

eth0 – 151.155.55.22

eth1 – 10.1.10.1

eth2 – 10.1.10.2

eth3 – 10.1.10.3

iscsi-target – 10.1.10.100 (iqn.my-target.com)

BASIC PREP-WORK

1. Install open-iscsi

   zypper install open-iscsi

2. Run “yast iscsi-client”, do nothing but let yast configure the initiator name.

   Yast has specific parameters regarding the syntax of the initiator name. Rather than manually configuring the initiator name and run into problems with Yast later, let Yast set the initiator name up front and save time later on.

3. Check the network setup and verify that all IP addresses and subnet masks are correct.

   yast lan

4. Check network connectivity by pinging from the different interfaces.

   syntax: ping -I ethX IP.of.target

   example: ping -I eth1 10.1.1.100

5. Set /proc/sys/net/ipv4/conf/rp_filter to 0

   The rp_filter setting impacts packet filtering/routing and can cause problems with iSCSI connections on a multi-homed system.

SETUP ISCSI IFACE FILES

1. Configure iface devices

   syntax: iscsiadm -m iface -I iface# –op=new (dash dash op=new)

   example: iscsiadm -m iface -I iface1 –op=new (dash dash op=new)

   Run this command to create an iface file for each desired interface. As of today, 08/22/2014, Yast cannot create iface files thus this step must be done manually.

2. Modify iface file to uniquely identify one NIC. There are several different parameters that can be used (iface.hwaddress, iface.net_ifacename, iface.ipaddress, etc) see the README for a more complete list of parameters.

   syntax: iscsiadm -m iface -I iface# –op=update -n iface.hwaddress -v MAC:of:NIC (dash dash op=update)

   example: iscsiadm -m iface -I iface1 –op=update -n iface.hwaddress -v 00:0c:29:d9:6f:75 (dash dash op=update)

   A word of caution: use the minimum number of parameters to correlate the iface and the NIC. Adding more parameters increases the likely-hood of typos or other mistakes that can cause iscsi not to work as expected.

3. Check the contents of the iface files to ensure accuracy. Either cat the file or use iscsiadm to view the files

   Option 1

   syntax: cat /etc/iscsi/ifaces/iface#

   example: cat /etc/iscsi/ifaces/iface1

   Option 2

   syntax: iscsiadm -m iface -I iface#

   example: iscsiadm -m iface -I iface1

4. Set the initiator name.

   If not done earlier set the initiator name. As noted above Yast has some specific syntax rule for the initiator names. Run “yast iscsi-client” and let Yast set the initiator name and then exit before doing any discovery.

DISCOVER TARGETS

1. The easiest method to have iscsid scan the target device and then use the /etc/iscsi/ifaces files create records for each target and iface combination is:

   syntax: iscsiadm -m discoverydb -t st -p IP.of.target.nic:3260 –discover (dash dash discover)

   example: iscsiadm -m discoverydb -t st -p 10.1.10.100:3260 –discover (dash dash discover)

   It is also possible to just discover a target on one iface.

   syntax: iscsiadm -m discovery -t st -p IP.of.target.nic:3260 –interface=iface# –discover (dash dash discover)

   example: iscsiadm -m discovery -t st -p 10.1.10.100:3260 –interface=iface1 –discover (dash dash discover)

ISCSI LOGIN

Once iscsid has built the send_targets database a login must be initiated to access the target device. Yast or the cli can be used to manage the iscsi connections and login to each node.

Using YaST

With the iface files created in /etc/iscsi/ifaces and the target discovered YaST will read the send_targets database and display each connection between the iface and the target and will list them in the Discovered Targets tab. From there log-in options are available via the log-in button. If authentication is needed YaST may be an easier option over the cli.

Using CLI

With the targets discovered for the desired ifaces the initiator now needs to log into the target.

syntax: iscsiadm -m node -l

example: iscsiadm -m node -l

This will log the initiator into all of the nodes that are in the database. Please note this example assumes no authentication is needed. If authentication is needed use YaST or consult the documentation. More granular options, such as logging into only one node, can be found in the documentation.

NODE SETTINGS

There are many settings relating to each iscsi node. The startup method and authentication methods to name a few. Several settings are available in YaST or manually manipulating the file is an option; however, misconfiguration of the node settings may cause problems with log-in and operation.

Once everything is configured as desired verify that chkconfig open-iscsi is set to on.

MPIO

1. Start multipathing

   rcmultipathd start

2. Check mpio paths

   multipath -ll

   If only one path is listed under multipath -ll double check the value of /proc/sys/net/ipv4/conf/all/rp_filter.

For help with multipath configuration please see http://support.novell.com/techcenter/sdb/en/2005/04/sles_multipathing.html

The README for iscsi can be found at /usr/share/doc/packages/open-iscsi/README