Can't get IBM library to work with TSM

[anton (staff)]

It's the same all the time:

support@starwindsoftware.com

PS Just make sure you send logs zipped.

I added both files as attachments. From manage attachments I see no post. Is there an e-mail address I should send them to?

Thanks

[boxcar]

I've used the tape redirector with RHEL 6 months ago with no problem, but at that time the library controller and driver was on the Hyper-V host and only the tape drives in use on the Linux VM. The difference here is that the library control is now in the Linux VM as well as the drive paths.

I'm offering the following information in case it helps in your diagnosis. sidecar and I are working on different systems and libraries but both are IBM branded libraries. Mine is a test system and I can set up a remote session any time if that would help.

I've managed to demonstrate the problem without the TSM backup software involved by using the IBM tape diagnostic tool (ITDT). ITDT performs library functions using the iin_tape driver. ITDT has a library diagnostic function and that passes with no error.

However, when I move a tape cartridge into a drive or try to move one from a drive I get the same error every time as shown below: Errno: 5 input/output error.

Before I increased the value of node.session.timeo.replacement_timeout (300, 600, 1000, now at 3000) I couldn't move slot to slot if I remember correctly, but slot to slot move now works.

When attempting to move to a drive or from a drive the cartridge actually moves to the destination slot or drive but returns the Errno: 5 error code - so it looks like we're very close.

Here's the failure sequence output:

itdt
u (for tape utility)
1 (open device)
/dev/IBMchanger0
1
52 (element inventory)
54 (move medium - find a slot element number for Source address and move to an empty slot for destination or the i/o port, or drive)

Command Result
+-----------------------------------------------------------------------------+
| Moving medium: source 257, destination 4116... |
| Move Medium FAILED |
| |
| Errno: 5 Input/output error |
| |
| Information Field Valid Bit-------0 |
| Error Code------------------------0x70 |
| Segment Number--------------------0x0 |
| File Mark Detected Bit------------0 |
| End Of Medium Bit-----------------0 |
| Illegal Length Indicator Bit------0 |
| Sense Key/ASC/ASCQ----------------0x00/0x00/0x00 |
| Information Bytes-----------------0 (00000000h) |
| Additional Sense Length-----------0x0A |
| Command Specific Information------0x00 0x00 0x00 0x00 |
| Field Replaceable Unit Code-------0x0 |
| Sense Key Specific Valid Bit------0 |
| Sense Data, Length 96 |
| |
| 0 1 2 3 4 5 6 7 8 9 A B C D E F 0123456789ABCDEF |
| 0000 - 7000 0000 0000 000A 0000 0000 0000 0000 [p...............] |
| 0010 - 0000 0000 0000 0000 0000 0000 0000 0000 [................] |
| 0020 - 0000 0000 0000 0000 0000 0000 0000 0000 [................] |
| 0030 - 0000 0000 0000 0000 0000 0000 0000 0000 [................] |
| 0040 - 0000 0000 0000 0000 0000 0000 0000 0000 [................] |
| 0050 - 0000 0000 0000 0000 0000 0000 0000 0000 [................] |

[sidecar]

Valery,

Were you able to get anything from the files I sent?

Thanks

[Val (staff)]

Hi,

According the logs the iSCSI pings could lead to the problem.

So please try to turn off iSCSI pings by setting:
node.conn[0].timeo.noop_out_interval = 0
node.conn[0].timeo.noop_out_timeout = 0

node.session.timeo.replacement_timeout = 120

Also it's better to use separate targets for the tape and the changer as seperate sessions use separate network connections.

[sidecar]

Valery,

Same results just as fast with the 3 changes to the iscsid.conf file.

[Val (staff)]

Hi,

It's not a StarWind's issue, from the logs we can see that the Linux initiator uses too short timeouts.

Please ask your question about the Linux iSCSI timeout settings at a Linux forum.

[boxcar]

Vallery,

Please see the log files I just sent. You'll discover an illegal request in the lin_tape.trace that may help you correlate what is going wrong.

This driver and backup software works fine with literally hundreds of brands of physical tape libraryies and many VTLs (some using iSCSI protocols). I'd like to believe we can make the adjustments to make it work with your iSCSI driver so we can make that available in a HyperV environment.

boxcar


---------------------------------------------------------------------
IBMtape1-----35872 Wed Nov 26 09:20:17 2014
Scsi Path : 04 00 00 00
CDB Command : 12 01 86 00 FF 00
Status Code : 08 00 00 01
Sense Data : 70 00 05 00 00 00 00 58 00 00 00 00 24 00 30 CF
00 02 00 00 00 01 53 30 31 30 30 37 00 00 00 4C
00 00 A9 06 00 00 00 00 00 30 12 80 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Description : Illegal Request

[boxcar]

Sorry, forgot to include the error from the dmesg file:

lin_tape: IBMtape1-----35872 unable to get inquiry page 86
lin_tape: qry_blk_prot cannot get inquiry 0x86

that seems to have caused the illegal request in lin_tape.errorlog.

[Val (staff)]

Hi,
this is not an error, just a warning that the optional INQUIRY VPD page 0x86 is not supported by the device.

As I've already said, the problem is that the Linux initiator drops iSCSI sessions because of too short timeout setting somewhere in the Linux iSCSI or SCSI data path.

[anton (staff)]

Update: Valery had fixed the issue so far. Valery well done

[boxcar]

I'm happy to say that the IBM libraries, a TS3582 with LTO3s and a 3584 are working perfectly with TSM servers in physical and virtual machines.

I've tested with TSM servers in the Hyper-V host, in a Windows Storage Server 2012 R2 virtual machines, and RHEL 6.5 and 6.6. I've tested with the IBM primary library manager in the Hyper-V host and in the Linux virtual machines with equal success.

We're also using the Linux TSM instance to backup NetApp boxes with a shared NDMP library. Plenty of layers and complexity.

Thanks to Valeriy, Anton, and the StarWind engineering team!

[anton (staff)]

Excellent! Thank you very much for confirmation

I'm happy to say that the IBM libraries, a TS3582 with LTO3s and a 3584 are working perfectly with TSM servers in physical and virtual machines.

I've tested with TSM servers in the Hyper-V host, in a Windows Storage Server 2012 R2 virtual machines, and RHEL 6.5 and 6.6. I've tested with the IBM primary library manager in the Hyper-V host and in the Linux virtual machines with equal success.

We're also using the Linux TSM instance to backup NetApp boxes with a shared NDMP library. Plenty of layers and complexity.

Thanks to Valeriy, Anton, and the StarWind engineering team!