======Overview======
iSCSI stands for Small Computer System Interface (SCSI) over IP. SCSI in a nutshell is set of commands, protocols and standards for transferring data between peripheral devices. Using iSCSI we can set storage to a remote machine using reliable protocols and methods.
======iSCSI Configuration======
The configure iSCSI you have to configure the following components:
* iSCSI Target - This is the server which will receive the data: Receiver
* With a LUN (the storage location)
* iSCSI Initiator - This is the server which will send the data: Sender
{{ :iscsi-env1.png?400 |}}
So let's get going :)
=====iSCSI Target=====
As everything, configuration depends on the OS, different wizards, utilities and methods depending on the OS, are used to configure the iSCSI target:
====Linux====
You can list the configured LUNs using the following command on Linux:
tgt-admin -s
Target 1: iqn.2015-01.new.target:disk1
System information:
Driver: iscsi
State: ready
====Asustor====
I will not go into details here but here is a picture of how it is configured on ASUSTORE NAS - AS202TE
It is important to note, that the iSCSI target is just the pointer to the LUN (Logical Unit Number).
{{ :iscsi_lun.jpg?400 |}}
So the connection will be initiated by the iSCSI initiator (on the client machine) -> iSCSI Target (on the server machine, over Internet) -> LUN (at the storage level)
====Windows====
In Windows server, you can create it via the wizzard in the Server Manager and it should be something like this at the end:
{{ :iscsitargerwindows.jpg?400 |}}
=====iSCSI Initiator=====
iSCSI initiator again depends on the operation system on the server.
====Windows====
Windows has integrated iSCSI initiator when you are installing it. Hopefully you have something higher than Windows 98 :).
The wizzard is pretty straight forward, you have to specify the iSCSI target server and it will discover the targets for you. Then you can just connect and map the device. Given no authentication is required.
{{ :iscsiinitiator_win.jpg?300 |}}
The configuration for windows is so trivial via the wizard that I don't think I should even discuss it.
====Linux====
On linux the situation is a little bit different. By default Linux doesn't install the initiator, so you have to install it yourself. Furthermore, like anything, there isn't GUI so you have to configure it yourself manually.
===Setup===
[root@matar ~]# yum install iscsi-initiator-utils -y
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Nothing to do
[root@matar ~]# yum list | grep iscsi
iscsi-initiator-utils.x86_64 6.2.0.873-32.el7 @anaconda/7.2 <- Already installed
iscsi-initiator-utils-iscsiuio.x86_64 6.2.0.873-32.el7 @anaconda/7.2
libiscsi.x86_64 1.9.0-6.el7 @anaconda/7.2
netbsd-iscsi.x86_64 20080207-9.el7 epel
You have new mail in /var/spool/mail/root
===Discover===
To discover the targets we can issue the following command:
[root@clientserver ~]# iscsiadm -m discovery -t st -p 10.200.15.30
iscsi_target_server_ip:3260,1 iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target
You have mail in /var/spool/mail/root
This will show us the exposed iSCSI targets.
In our case we have iscsi target which is on port: 3260 and with name: "iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target"
===CHAP===
Chap protection might or might not be enabled on the target server. If CHAP is configured on the iSCSI target, we have to configure it in the initiator.
**BE CAREFUL TO CONFIGURE ONLY THE INITIATOR PART OR YOU WILL HAVE this error: "The challenge was incorrect."
**
The configuration is done in the:
/etc/iscsi/iscsid.conf
# *************
# CHAP Settings
# *************
# To enable CHAP authentication set node.session.auth.authmethod
# to CHAP. The default is None.
node.session.auth.authmethod = CHAP
# To set a CHAP username and password for initiator
# authentication by the target(s), uncomment the following lines:
node.session.auth.username = username
node.session.auth.password = pass
# To set a CHAP username and password for target(s)
# authentication by the initiator, uncomment the following lines:
#node.session.auth.username_in = username
#node.session.auth.password_in = pass
# To enable CHAP authentication for a discovery session to the target
# set discovery.sendtargets.auth.authmethod to CHAP. The default is None.
#discovery.sendtargets.auth.authmethod = CHAP
# To set a discovery session CHAP username and password for the initiator
# authentication by the target(s), uncomment the following lines:
#discovery.sendtargets.auth.username = username
#discovery.sendtargets.auth.password = pass
# To set a discovery session CHAP username and password for target(s)
# authentication by the initiator, uncomment the following lines:
#discovery.sendtargets.auth.username_in = username
#discovery.sendtargets.auth.password_in = pass
Restart the iscsi deamon:
[root@matar /]# service iscsid restart
Redirecting to /bin/systemctl restart iscsid.service
[root@matar /]#
===Login===
We can login to the iSCSI target and start a session as follows:
[root@client_host /]# iscsiadm --mode node --targetname iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target --portal 10.200.15.30:3260 --login
Logging in to [iface: default, target: iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target, portal: IP,3260] (multiple)
Login to [iface: default, target: iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target, portal: IP,3260] successful.
You can also list the current setting for the initiator and target as follows:
[root@client_host /]# iscsiadm --mode node --targetname iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target --portal iscsi_target_server_ip:3260
# BEGIN RECORD 6.2.0.873-30
node.name = iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target
node.tpgt = 1
node.startup = automatic
node.leading_login = No
iface.hwaddress =
iface.ipaddress =
iface.iscsi_ifacename = default
iface.net_ifacename =
iface.transport_name = tcp
iface.initiatorname =
iface.state =
iface.vlan_id = 0
iface.vlan_priority = 0
iface.vlan_state =
iface.iface_num = 0
iface.mtu = 0
iface.port = 0
iface.bootproto =
iface.subnet_mask =
iface.gateway =
iface.dhcp_alt_client_id_state =
iface.dhcp_alt_client_id =
iface.dhcp_dns =
iface.dhcp_learn_iqn =
iface.dhcp_req_vendor_id_state =
iface.dhcp_vendor_id_state =
iface.dhcp_vendor_id =
iface.dhcp_slp_da =
iface.fragmentation =
iface.gratuitous_arp =
iface.incoming_forwarding =
iface.tos_state =
iface.tos = 0
iface.ttl = 0
iface.delayed_ack =
iface.tcp_nagle =
iface.tcp_wsf_state =
iface.tcp_wsf = 0
iface.tcp_timer_scale = 0
iface.tcp_timestamp =
iface.redirect =
iface.def_task_mgmt_timeout = 0
iface.header_digest =
iface.data_digest =
iface.immediate_data =
iface.initial_r2t =
iface.data_seq_inorder =
iface.data_pdu_inorder =
iface.erl = 0
iface.max_receive_data_len = 0
iface.first_burst_len = 0
iface.max_outstanding_r2t = 0
iface.max_burst_len = 0
iface.chap_auth =
iface.bidi_chap =
iface.strict_login_compliance =
iface.discovery_auth =
iface.discovery_logout =
node.discovery_address = iscsi_target_server_ip
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 8
node.session.xmit_thread_priority = -20
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.nr_sessions = 1
node.session.auth.authmethod = CHAP
node.session.auth.username = izoladbbackup
node.session.auth.password = ********
node.session.auth.username_in =
node.session.auth.password_in =
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 30
node.session.err_timeo.tgt_reset_timeout = 30
node.session.err_timeo.host_reset_timeout = 60
node.session.iscsi.FastAbort = Yes
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 2
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = iscsi_target_server_ip
node.conn[0].port = 3260
node.conn[0].startup = manual
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No
# END RECORD
That information is also available on:
/var/lib/iscsi/nodes/iqn
[root@matar 10.200.15.30,3260,1]# pwd
/var/lib/iscsi/nodes/iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target/10.200.15.30,3260,1
[root@matar 10.200.15.30,3260,1]# ls -lart
total 4
-rw------- 1 root root 2155 Nov 1 12:40 default
drw------- 3 root root 32 Nov 1 12:40 ..
drw------- 2 root root 20 Nov 1 12:40 .
[root@matar 10.200.15.30,3260,1]#
Once the Login was successfully, we can check where the OS mounted the device:
[root@matar 10.200.15.30,3260,1]# lsblk --scsi|grep LIO
sdf 3:0:0:0 disk LIO-ORG block1 4.0 iscsi
Or you can just grep in the /var/log/messages
[root@oraback log]# grep " Attached scsi disk" /var/log/messages
Nov 6 16:49:22 oraback kernel: sd 4:0:0:0: Attached scsi disk sdf
[root@oraback log]#
Then we can create the file system:
[root@matar 10.200.15.30,3260,1]# mkfs.ext4 -m0 /dev/sdf
[root@matar 10.200.15.30,3260,1]# blkid | grep sdf
/dev/sdf: UUID="6a1c44d0-3e2f-49fc-85ba-ced3e44bb5b0" TYPE="ext4"
Add the following to the /etc/fstab:
[root@matar 10.200.15.30,3260,1]# mkdir -p /mnt/block1
[root@matar 10.200.15.30,3260,1]# echo "UUID=6a1c44d0-3e2f-49fc-85ba-ced3e44bb5b0 /mnt/block1 ext4 _netdev 0 0" >> /etc/fstab
And then we can mount the file system:
[root@matar 10.200.15.30,3260,1]# mount /mnt/block1
[root@matar 10.200.15.30,3260,1]# df -h | grep block1
UUID=6a1c44d0-3e2f-49fc-85ba-ced3e44bb5b0 10.0G 9.9G 0.1G 1% /mnt/block1
===Logout===
You can decide to abort a session and disconnect as follows:
[root@matar dev]# iscsiadm --mode node --targetname iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target --portal 10.200.15.30:3260 --logout
Logging out of session [sid: 16, target: iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target, portal: iscsi_target_server_ip,3260]
Logout of [sid: 16, target: iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target, portal: iscsi_target_server_ip,3260] successful.
===Delete configuration===
To deleting a configuration, we need to:
- Disconnect/Logout
- Ensure that the driver won't try to re-connect
==Logout==
[root@matar ~]# iscsiadm --mode node --targetname iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target --portal 10.200.15.30:3260 --logout
Logging out of session [sid: 20, target: iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target, portal: 10.200.15.30,3260]
==Delete==
[root@matar ~]# iscsiadm -m node -o delete -T iqn.1991-05.com.microsoft:taurus-taurus-orabackup-target
[root@matar ~]# iscsiadm -m session
iscsiadm: No active sessions.
=====Performance=====
I've found that iSCSI isn't particulary fast. I've found that my environment isn't so much loaded. That can be because the clients aren't using Jumbo frames so that might be the reson.
[root@server Logs]# ip link show | grep mtu
1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
2: eno16780032: mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
3: virbr0: mtu 1500 qdisc noqueue state DOWN mode DEFAULT
4: virbr0-nic: mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 500
[root@server Logs]# ls -lart
Here is a good article of how to speed up the performance of the iSCSI:
https://www.monperrus.net/martin/performance+of+read-write+throughput+with+iscsi