[Bacula-users] Btape Test - Append Files Test

[Aaron Greenblatt]

Hello,

I am attempting to use Bacula on a backup server and am having trouble with
btape test in the append files section. I have read the documentation,
searched on Google, and updated the firmware on my SAS card and tape drive.
I don't see any particularly helpful log messages on the system. Any
thoughts on where I should look to continue troubleshooting this? Could it
have something to do with the SCSI driver in the OS?

Configuration is as follows: Debian 8 with Atto H1280 (also tried H644) SAS
HBA and HP LTO6 tape drive. More details below.

Thank you.

Best,
Aaron

Output from btape:
=== Append files test ===

This test is essential to Bacula.

I'm going to write one record in file 0,
two records in file 1,
and three records in file 2

btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:478-0 open device "LTO-6" (/dev/nst0): OK
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1430-0 Now moving to end of medium.
btape: btape.c:633-0 Moved to end of medium.
We should be in file 3. I am at file 3. This is correct!

Now the important part, I am going to attempt to append to the tape.

btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
Done appending, there should be no I/O errors

Doing Bacula scan of blocks:
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=5, blocks=7, bytes = 451,136
End scanning the tape.
*We should be in file 4. I am at file 5. This is NOT correct!!!!*

The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
=== End sample correct output ===

If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to
the tape.



bacula-sd.conf (I tried this without the Hardware End of File line; same
result):

Device {
Name = LTO-6
Media Type = LTO-6
Device Type = Tape
Archive Device = /dev/nst0
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = yes;
RemovableMedia = yes;
RandomAccess = no;
Maximum File Size = 5GB
Hardware End of File = no
}


tapeinfo -f /dev/sg8

Product Type: Tape Drive
Vendor ID: 'HP '
Product ID: 'Ultrium 6-SCSI '
Revision: '35GD'
Attached Changer API: No
SerialNumber: 'HUJ4321BY1'
MinBlock: 1
MaxBlock: 16777215
SCSI ID: 0
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x5a
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0x1
DeCompType: 0x1
BOP: yes
Block Position: 0
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3

[Kern Sibbald]

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

[Aaron Greenblatt]

Hi Kern,

Thanks for your help!

I removed the two lines you mentioned and had the same result.

The btape output is below. I'm running Debian 8.8 (kernel 3.16.0-4-amd64).
I downloaded the Bacula 9.0.1 source from
https://sourceforge.net/projects/bacula/files/bacula/9.0.1/ and compiled
the binaries on my machine. The binaries are dynamically linked, so I added
"/home/agreenblatt/software-bacula/bacula/src/stored/.libs" to
/etc/ld.so.conf.d/bacula.conf and ran ldconfig. I assume that the warning
about plugins in /usr/local/lib can be ignored as a result? (Though, now
that I look, I see the libs are indeed in /usr/local/lib, so perhaps I
didn't need to make the extra file in ld.so.conf.d.)

Best,
Aaron

Full btape output:

***@tapebackup:/usr/local/bacula/etc# btape -c bacula-sd.conf /dev/nst0
Tape block granularity is 1024 bytes.
20-Jul 03:11 btape JobId 0: Warning: Failed to find any plugins in
/usr/local/lib
btape: butil.c:290-0 Using device: "/dev/nst0" for writing.
btape: btape.c:478-0 open device "LTO-6" (/dev/nst0): OK
*test

=== Write, rewind, and re-read test ===

I'm going to write 10000 records and an EOF
then write 10000 records and an EOF, then rewind,
and re-read the data to verify that it is correct.

This is an *essential* feature ...

btape: btape.c:1161-0 Wrote 10000 blocks of 64412 bytes.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1177-0 Wrote 10000 blocks of 64412 bytes.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1219-0 Rewind OK.
10000 blocks re-read correctly.
Got EOF on tape.
10000 blocks re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===

btape: btape.c:1286-0 Block position test
btape: btape.c:1297-0 Rewind OK.
Reposition to file:block 0:4
Block 5 re-read correctly.
Reposition to file:block 0:200
Block 201 re-read correctly.
Reposition to file:block 0:9999
Block 10000 re-read correctly.
Reposition to file:block 1:0
Block 10001 re-read correctly.
Reposition to file:block 1:600
Block 10601 re-read correctly.
Reposition to file:block 1:9999
Block 20000 re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===



=== Append files test ===

This test is essential to Bacula.

I'm going to write one record in file 0,
two records in file 1,
and three records in file 2

btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:478-0 open device "LTO-6" (/dev/nst0): OK
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1430-0 Now moving to end of medium.
btape: btape.c:633-0 Moved to end of medium.
We should be in file 3. I am at file 3. This is correct!

Now the important part, I am going to attempt to append to the tape.

btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
Done appending, there should be no I/O errors

Doing Bacula scan of blocks:
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=5, blocks=7, bytes = 451,136
End scanning the tape.
We should be in file 4. I am at file 5. This is NOT correct!!!!

The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
=== End sample correct output ===

If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to
the tape.



=== Forward space files test ===

This test is essential to Bacula.

I'm going to write five files then test forward spacing

btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1644-0 Now forward spacing 1 file.
We should be in file 1. I am at file 1. This is correct!
btape: btape.c:1656-0 Now forward spacing 2 files.
We should be in file 3. I am at file 3. This is correct!
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1669-0 Now forward spacing 4 files.
We should be in file 4. I am at file 4. This is correct!

btape: btape.c:1687-0 Now forward spacing 1 more file.
We should be in file 5. I am at file 5. This is correct!

=== End Forward space files test ===

*

Post by Aaron Greenblatt
Hello,
Hardware End of File = no
and
Device Type = Tape
then try the test again. If it fails please post the btape output, the OS
you are running, the Bacula version, and where you got the binaries.
Best regards,
Kern
Hello,
I am attempting to use Bacula on a backup server and am having trouble
with btape test in the append files section. I have read the documentation,
searched on Google, and updated the firmware on my SAS card and tape drive.
I don't see any particularly helpful log messages on the system. Any
thoughts on where I should look to continue troubleshooting this? Could it
have something to do with the SCSI driver in the OS?
Configuration is as follows: Debian 8 with Atto H1280 (also tried H644)
SAS HBA and HP LTO6 tape drive. More details below.
Thank you.
Best,
Aaron
=== Append files test ===
This test is essential to Bacula.
I'm going to write one record in file 0,
two records in file 1,
and three records in file 2
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:478-0 open device "LTO-6" (/dev/nst0): OK
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1430-0 Now moving to end of medium.
btape: btape.c:633-0 Moved to end of medium.
We should be in file 3. I am at file 3. This is correct!
Now the important part, I am going to attempt to append to the tape.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
Done appending, there should be no I/O errors
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=5, blocks=7, bytes = 451,136
End scanning the tape.
*We should be in file 4. I am at file 5. This is NOT correct!!!!*
The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
=== End sample correct output ===
If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to
the tape.
bacula-sd.conf (I tried this without the Hardware End of File line; same
Device {
Name = LTO-6
Media Type = LTO-6
Device Type = Tape
Archive Device = /dev/nst0
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = yes;
RemovableMedia = yes;
RandomAccess = no;
Maximum File Size = 5GB
Hardware End of File = no
}
tapeinfo -f /dev/sg8
Product Type: Tape Drive
Vendor ID: 'HP '
Product ID: 'Ultrium 6-SCSI '
Revision: '35GD'
Attached Changer API: No
SerialNumber: 'HUJ4321BY1'
MinBlock: 1
MaxBlock: 16777215
SCSI ID: 0
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x5a
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0x1
DeCompType: 0x1
BOP: yes
Block Position: 0
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________

[Jim Richardson]

Be sure to factory default your tape drive. I had the exact issue with my LTO7. The drive was used in a non-linux environment before. My steps to factory reset are a pain, but they got me working. Yours may be different. Manufacturer dependent.


# Steps

1. Install non-RAID HBA

2. install lin_tape & lin_taped

3. install ITDT

4. run full set of tests in ITDT

5. factory default settings - Specifically the "Read Past Filemark" setting MUST BE no

http://www-01.ibm.com/support/docview.wss?uid=ssg1S7002972&aid=1

6. remove lin_tape & lin_taped

7. rmmod st

8. modprob st

9. power cycle library

10. Bacula configuration

11. btape test



# tapeinfo -f /dev/nst0

Product Type: Tape Drive

Vendor ID: 'IBM '

Product ID: 'ULT3580-HH7 '

Revision: 'G9Q1'

Attached Changer API: No

SerialNumber: '1097000515'

MinBlock: 1

MaxBlock: 8388608

SCSI ID: 1

SCSI LUN: 0

Ready: yes

BufferedMode: yes

Medium Type: 0x78

Density Code: 0x5c

BlockSize: 0

DataCompEnabled: yes

DataCompCapable: yes

DataDeCompEnabled: yes

CompType: 0xff

DeCompType: 0xff

BOP: yes

Block Position: 0

Partition 0 Remaining Kbytes: -1

Partition 0 Size in Kbytes: -1

ActivePartition: 0

EarlyWarningSize: 0

NumPartitions: 0

MaxPartitions: 3



# lsscsi -g

[3:0:1:0] tape IBM ULT3580-HH7 G9Q1 /dev/st0 /dev/sg5

[3:0:1:1] mediumx IBM 3572-TL 0071 - /dev/sg6



Device {

Name = Drive-1 #

Drive Index = 0

Media Type = LTO-7

Archive Device = /dev/nst0

AutomaticMount = yes; # when device opened, read it

AlwaysOpen = yes;

RemovableMedia = yes;

RandomAccess = no;

AutoChanger = yes

Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

}






Jim Richardson

From: Aaron Greenblatt [mailto:***@gmail.com]
Sent: Thursday, July 20, 2017 12:15 PM
To: Kern Sibbald <***@sibbald.com>
Cc: bacula-***@lists.sourceforge.net
Subject: Re: [Bacula-users] Btape Test - Append Files Test

Hi Kern,
Thanks for your help!
I removed the two lines you mentioned and had the same result.
The btape output is below. I'm running Debian 8.8 (kernel 3.16.0-4-amd64). I downloaded the Bacula 9.0.1 source from https://sourceforge.net/projects/bacula/files/bacula/9.0.1/ and compiled the binaries on my machine. The binaries are dynamically linked, so I added "/home/agreenblatt/software-bacula/bacula/src/stored/.libs" to /etc/ld.so.conf.d/bacula.conf and ran ldconfig. I assume that the warning about plugins in /usr/local/lib can be ignored as a result? (Though, now that I look, I see the libs are indeed in /usr/local/lib, so perhaps I didn't need to make the extra file in ld.so.conf.d.)
Best,
Aaron

Full btape output:

***@tapebackup:/usr/local/bacula/etc# btape -c bacula-sd.conf /dev/nst0
Tape block granularity is 1024 bytes.
20-Jul 03:11 btape JobId 0: Warning: Failed to find any plugins in /usr/local/lib
btape: butil.c:290-0 Using device: "/dev/nst0" for writing.
btape: btape.c:478-0 open device "LTO-6" (/dev/nst0): OK
*test

=== Write, rewind, and re-read test ===

I'm going to write 10000 records and an EOF
then write 10000 records and an EOF, then rewind,
and re-read the data to verify that it is correct.

This is an *essential* feature ...

btape: btape.c:1161-0 Wrote 10000 blocks of 64412 bytes.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1177-0 Wrote 10000 blocks of 64412 bytes.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1219-0 Rewind OK.
10000 blocks re-read correctly.
Got EOF on tape.
10000 blocks re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===

btape: btape.c:1286-0 Block position test
btape: btape.c:1297-0 Rewind OK.
Reposition to file:block 0:4
Block 5 re-read correctly.
Reposition to file:block 0:200
Block 201 re-read correctly.
Reposition to file:block 0:9999
Block 10000 re-read correctly.
Reposition to file:block 1:0
Block 10001 re-read correctly.
Reposition to file:block 1:600
Block 10601 re-read correctly.
Reposition to file:block 1:9999
Block 20000 re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===



=== Append files test ===

This test is essential to Bacula.

I'm going to write one record in file 0,
two records in file 1,
and three records in file 2

btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:478-0 open device "LTO-6" (/dev/nst0): OK
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1430-0 Now moving to end of medium.
btape: btape.c:633-0 Moved to end of medium.
We should be in file 3. I am at file 3. This is correct!

Now the important part, I am going to attempt to append to the tape.

btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
Done appending, there should be no I/O errors

Doing Bacula scan of blocks:
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=5, blocks=7, bytes = 451,136
End scanning the tape.
We should be in file 4. I am at file 5. This is NOT correct!!!!

The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
=== End sample correct output ===

If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to
the tape.



=== Forward space files test ===

This test is essential to Bacula.

I'm going to write five files then test forward spacing

btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1644-0 Now forward spacing 1 file.
We should be in file 1. I am at file 1. This is correct!
btape: btape.c:1656-0 Now forward spacing 2 files.
We should be in file 3. I am at file 3. This is correct!
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1669-0 Now forward spacing 4 files.
We should be in file 4. I am at file 4. This is correct!

btape: btape.c:1687-0 Now forward spacing 1 more file.
We should be in file 5. I am at file 5. This is correct!

=== End Forward space files test ===

*


On Thu, Jul 20, 2017 at 9:01 AM, Kern Sibbald <***@sibbald.com<mailto:***@sibbald.com>> wrote:

Hello,

Please take out the:

Hardware End of File = no

and

Device Type = Tape

then try the test again. If it fails please post the btape output, the OS you are running, the Bacula version, and where you got the binaries.

Best regards,

Kern



On 07/20/2017 05:42 PM, Aaron Greenblatt wrote:
Hello,
I am attempting to use Bacula on a backup server and am having trouble with btape test in the append files section. I have read the documentation, searched on Google, and updated the firmware on my SAS card and tape drive. I don't see any particularly helpful log messages on the system. Any thoughts on where I should look to continue troubleshooting this? Could it have something to do with the SCSI driver in the OS?
Configuration is as follows: Debian 8 with Atto H1280 (also tried H644) SAS HBA and HP LTO6 tape drive. More details below.

Thank you.
Best,
Aaron

Output from btape:
=== Append files test ===

This test is essential to Bacula.

I'm going to write one record in file 0,
two records in file 1,
and three records in file 2

btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:478-0 open device "LTO-6" (/dev/nst0): OK
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1430-0 Now moving to end of medium.
btape: btape.c:633-0 Moved to end of medium.
We should be in file 3. I am at file 3. This is correct!

Now the important part, I am going to attempt to append to the tape.

btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
Done appending, there should be no I/O errors

Doing Bacula scan of blocks:
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=5, blocks=7, bytes = 451,136
End scanning the tape.
We should be in file 4. I am at file 5. This is NOT correct!!!!

The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
=== End sample correct output ===

If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to
the tape.


bacula-sd.conf (I tried this without the Hardware End of File line; same result):

Device {
Name = LTO-6
Media Type = LTO-6
Device Type = Tape
Archive Device = /dev/nst0
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = yes;
RemovableMedia = yes;
RandomAccess = no;
Maximum File Size = 5GB
Hardware End of File = no
}

tapeinfo -f /dev/sg8

Product Type: Tape Drive
Vendor ID: 'HP '
Product ID: 'Ultrium 6-SCSI '
Revision: '35GD'
Attached Changer API: No
SerialNumber: 'HUJ4321BY1'
MinBlock: 1
MaxBlock: 16777215
SCSI ID: 0
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x5a
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0x1
DeCompType: 0x1
BOP: yes
Block Position: 0
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3



------------------------------------------------------------------------------

Check out the vibrant tech community on one of the world's most

engaging tech sites, Slashdot.org! http://sdm.link/slashdot


_______________________________________________

Bacula-users mailing list

Bacula-***@lists.sourceforge.net<mailto:Bacula-***@lists.sourceforge.net>

https://lists.sourceforge.net/lists/listinfo/bacula-users


CONFIDENTIALITY: This email (including any attachments) may contain confidential, proprietary and privileged information, and unauthorized disclosure or use is prohibited. If you received this email in error, please notify the sender and delete this email from your system. Thank you.

[Kern Sibbald]

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

[Aaron Greenblatt]

Hi Everyone,

Thanks for your help!

I looked around in HP's Library and Tape Tools software and couldn't find
anything similar to a "Read Past Filemark" setting - the settings were
limited to disabling the eject button and something else that didn't look
very useful. I'm sure that HP has a this setting buried somewhere, but I'd
expect that one has to call HP support to enable it.

Meanwhile, I installed Bacula 7.4.7 and the test passes in that version. I
think the answer for now is to use version 7.4.7 to get my backup server
running.

Is there any sort of information that would be useful for you from versions
9.0.0 or 9.0.1 (both gave me the same problem)?

Thanks,
Aaron

Hello Jim,
Very interesting! Thanks for telling us.
Best regards,
Kern
Be sure to factory default your tape drive. I had the exact issue with my
LTO7. The drive was used in a non-linux environment before. My steps to
factory reset are a pain, but they got me working. Yours may be
different. Manufacturer dependent.
# Steps
1. Install non-RAID HBA
2. install lin_tape & lin_taped
3. install ITDT
4. run full set of tests in ITDT
5. factory default settings - Specifically the "Read Past Filemark" setting MUST BE no
http://www-01.ibm.com/support/docview.wss?uid=ssg1S7002972&aid=1
6. remove lin_tape & lin_taped
7. rmmod st
8. modprob st
9. power cycle library
10. Bacula configuration
11. btape test
# tapeinfo -f /dev/nst0
Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULT3580-HH7 '
Revision: 'G9Q1'
Attached Changer API: No
SerialNumber: '1097000515'
MinBlock: 1
MaxBlock: 8388608
SCSI ID: 1
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: 0x78
Density Code: 0x5c
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0xff
DeCompType: 0xff
BOP: yes
Block Position: 0
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3
# lsscsi -g
[3:0:1:0] tape IBM ULT3580-HH7 G9Q1 /dev/st0 /dev/sg5
[3:0:1:1] mediumx IBM 3572-TL 0071 - /dev/sg6
Device {
Name = Drive-1 #
Drive Index = 0
Media Type = LTO-7
Archive Device = /dev/nst0
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = yes;
RemovableMedia = yes;
RandomAccess = no;
AutoChanger = yes
Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
}
*Jim Richardson*
*Sent:* Thursday, July 20, 2017 12:15 PM
*Subject:* Re: [Bacula-users] Btape Test - Append Files Test
Hi Kern,
Thanks for your help!
I removed the two lines you mentioned and had the same result.
The btape output is below. I'm running Debian 8.8 (kernel 3.16.0-4-amd64).
I downloaded the Bacula 9.0.1 source from https://sourceforge.net/
projects/bacula/files/bacula/9.0.1/ and compiled the binaries on my
machine. The binaries are dynamically linked, so I added
"/home/agreenblatt/software-bacula/bacula/src/stored/.libs" to
/etc/ld.so.conf.d/bacula.conf and ran ldconfig. I assume that the warning
about plugins in /usr/local/lib can be ignored as a result? (Though, now
that I look, I see the libs are indeed in /usr/local/lib, so perhaps I
didn't need to make the extra file in ld.so.conf.d.)
Best,
Aaron
Tape block granularity is 1024 bytes.
20-Jul 03:11 btape JobId 0: Warning: Failed to find any plugins in /usr/local/lib
btape: butil.c:290-0 Using device: "/dev/nst0" for writing.
btape: btape.c:478-0 open device "LTO-6" (/dev/nst0): OK
*test
=== Write, rewind, and re-read test ===
I'm going to write 10000 records and an EOF
then write 10000 records and an EOF, then rewind,
and re-read the data to verify that it is correct.
This is an *essential* feature ...
btape: btape.c:1161-0 Wrote 10000 blocks of 64412 bytes.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1177-0 Wrote 10000 blocks of 64412 bytes.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1219-0 Rewind OK.
10000 blocks re-read correctly.
Got EOF on tape.
10000 blocks re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===
btape: btape.c:1286-0 Block position test
btape: btape.c:1297-0 Rewind OK.
Reposition to file:block 0:4
Block 5 re-read correctly.
Reposition to file:block 0:200
Block 201 re-read correctly.
Reposition to file:block 0:9999
Block 10000 re-read correctly.
Reposition to file:block 1:0
Block 10001 re-read correctly.
Reposition to file:block 1:600
Block 10601 re-read correctly.
Reposition to file:block 1:9999
Block 20000 re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===
=== Append files test ===
This test is essential to Bacula.
I'm going to write one record in file 0,
two records in file 1,
and three records in file 2
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:478-0 open device "LTO-6" (/dev/nst0): OK
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1430-0 Now moving to end of medium.
btape: btape.c:633-0 Moved to end of medium.
We should be in file 3. I am at file 3. This is correct!
Now the important part, I am going to attempt to append to the tape.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
Done appending, there should be no I/O errors
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=5, blocks=7, bytes = 451,136
End scanning the tape.
We should be in file 4. I am at file 5. This is NOT correct!!!!
The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
=== End sample correct output ===
If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to
the tape.
=== Forward space files test ===
This test is essential to Bacula.
I'm going to write five files then test forward spacing
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1644-0 Now forward spacing 1 file.
We should be in file 1. I am at file 1. This is correct!
btape: btape.c:1656-0 Now forward spacing 2 files.
We should be in file 3. I am at file 3. This is correct!
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1669-0 Now forward spacing 4 files.
We should be in file 4. I am at file 4. This is correct!
btape: btape.c:1687-0 Now forward spacing 1 more file.
We should be in file 5. I am at file 5. This is correct!
=== End Forward space files test ===
*
Hello,
Hardware End of File = no
and
Device Type = Tape
then try the test again. If it fails please post the btape output, the OS
you are running, the Bacula version, and where you got the binaries.
Best regards,
Kern
Hello,
I am attempting to use Bacula on a backup server and am having trouble
with btape test in the append files section. I have read the documentation,
searched on Google, and updated the firmware on my SAS card and tape drive.
I don't see any particularly helpful log messages on the system. Any
thoughts on where I should look to continue troubleshooting this? Could it
have something to do with the SCSI driver in the OS?
Configuration is as follows: Debian 8 with Atto H1280 (also tried H644)
SAS HBA and HP LTO6 tape drive. More details below.
Thank you.
Best,
Aaron
=== Append files test ===
This test is essential to Bacula.
I'm going to write one record in file 0,
two records in file 1,
and three records in file 2
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:478-0 open device "LTO-6" (/dev/nst0): OK
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
btape: btape.c:1430-0 Now moving to end of medium.
btape: btape.c:633-0 Moved to end of medium.
We should be in file 3. I am at file 3. This is correct!
Now the important part, I am going to attempt to append to the tape.
btape: btape.c:1917-0 Wrote one record of 64412 bytes.
btape: btape.c:1919-0 Wrote block to device.
btape: btape.c:612-0 Wrote 1 EOF to "LTO-6" (/dev/nst0)
btape: btape.c:582-0 Rewound "LTO-6" (/dev/nst0)
Done appending, there should be no I/O errors
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=5, blocks=7, bytes = 451,136
End scanning the tape.
*We should be in file 4. I am at file 5. This is NOT correct!!!!*
The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
=== End sample correct output ===
If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to
the tape.
Device {
Name = LTO-6
Media Type = LTO-6
Device Type = Tape
Archive Device = /dev/nst0
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = yes;
RemovableMedia = yes;
RandomAccess = no;
Maximum File Size = 5GB
Hardware End of File = no
}
tapeinfo -f /dev/sg8
Product Type: Tape Drive
Vendor ID: 'HP '
Product ID: 'Ultrium 6-SCSI '
Revision: '35GD'
Attached Changer API: No
SerialNumber: 'HUJ4321BY1'
MinBlock: 1
MaxBlock: 16777215
SCSI ID: 0
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x5a
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0x1
DeCompType: 0x1
BOP: yes
Block Position: 0
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
https://lists.sourceforge.net/lists/listinfo/bacula-users
CONFIDENTIALITY: This email (including any attachments) may contain
confidential, proprietary and privileged information, and unauthorized
disclosure or use is prohibited. If you received this email in error,
please notify the sender and delete this email from your system. Thank you.

[Kern Sibbald]

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot