The topic of this article is connecting external devices to the Mac via SAS, Fiber Channel (FC), eSATA interfaces. Let's make a reservation right away that in order to solve the problem of access to such devices, there is a way for a healthy person: to assemble a cheap PC, plug in an HBA SAS or FC controller card (for example, the simplest LSI adapter), connect your devices to this controller, install any Linux on the PC and work from a Mac over the network. But this is trite and uninteresting. We will go the hardcore way and connect our devices directly to Mac.
What we need for this:
- a decent amount of money to buy new equipment, or good luck in auctions on eBay (where, with a little effort, you can buy the required equipment of previous generations 10 times cheaper than according to the price list);
- This article.
To work with magnetic tape (now almost universally represented by the LTO format), you must have a tape drive (streamer) or tape library of the LTO standard. This is a rather expensive device for the initial purchase (from hundreds of thousands of rubles), but worth the sane money when buying a used one. Since LTO generations change approximately every two years, and compatibility is limited to two generations, the secondary market is sufficiently saturated with operable devices four years old and older, i.e. generation before last and beyond. If you buy a new device for commercial purposes, then you yourself understand why you need it. If you want to buy for home and family, you can consider this option as a way to archive information (since the media themselves are very cheap per 1 gigabyte).
Starting with the LTO-5 generation (and partly LTO-4), magnetic tape devices are connected to the computer via a SAS or FC interface (usually there are two versions of each device)
On the other hand, Apple kindly provides us with a USB-C interface in our Mac (operating via USB, Thunderbolt 3 or DisplayPort protocols), sometimes an Ethernet interface, as well as proprietary Thunderbolt 3 to Thunderbolt 2 and Thunderbolt to FireWire 800 adapters.
Stalemate? Not really. Fortunately, the Thunderbolt bus can operate in PCIe mode and allows you to connect PCIe cards in the same way as if they were installed directly inside the computer case. Due to this, any expansion of the hardware configuration of the Mac is possible, there would be an appropriate adapter and drivers.
Conceptually, the simplest way to solve the problem is an external box for PCIe adapters with a Thunderbolt interface (PCIe card expansion system), in which you can install a controller (Host bus adapter, HBA) SAS or FC. For example, such boxes are produced by the company and some others. There is a nuance here: not every controller is suitable for us, but only one with a driver for macOS. There are not many such boards, and the cheapest and most popular ones (for example, the same LSI) are not among them. Fortunately, Sonnet took the trouble to compile PCIe cards with various operating systems via the Thunderbolt interface.
Another solution is to purchase a ready-made Thunderbolt-SAS or Thunderbolt-FC interface converter, which, in fact, is a ready-made assembly of a box and a controller. The most famous company in this area , but there are also products from other companies.
Note that not all SAS and FC controllers are LTO certified, as this in itself costs money. Some manufacturers directly write that the work of their controllers with tape drives is not provided.
To complete the picture, we note that mLogic produces , which is an IBM LTO-8 drive in an external case, into which a SAS to Thunderbolt 3 converter is immediately integrated. However, this thing is even more exotic than all of the above, especially by the standards of our regions. I doubt that this device can even be legally imported into Russia (LTO drives contain cryptographic facilities, and manufacturers such as IBM and HP receive FSB import permission for each model for this reason).
Next, we will consider, for example, a specific set of equipment, the owner of which the author became as a result of several successful acquisitions, but the general principle should be preserved for all options.
So we have the following equipment for working with the tape:
- Apple Mac mini 2018 with macOS 10.15 Catalina with USB-C ports with Thunderbolt 3 support;
β Apple Thunderbolt 3 / Thunderbolt 2 adapter;
β Apple Thunderbolt 2 cable;
β ATTO ThunderLink SH 1068 interface converter (2*Thunderbolt / 2*SAS-2);
β SAS cable SFF-8088 β SFF-8088;
β tape drive LTO-5 IBM TS2350;
β LTO-5 cartridges, cleaning cartridge.
Now, as they say, with all this bullshit, we will try to take off.
We download the latest version of the ThunderLink SH 1068 driver from the ATTO website (apparently, for our convenience, it is merged with the SH 2068 driver and is located in section 2068, which is written only inside the driver archive itself) and the ATTO configuration utility.
The driver, of course, needs to be installed. Before such actions, the author advises to always take a snapshot of the APFS file system of the boot disk with the command
tmutil localsnapshotor a backup copy of the boot disk, if there is HFS +. You never know. Then it will be easy to roll back from the snapshot.
Further, the inexperienced but due diligent mind will no doubt be inclined to carefully read the ATTO instructions for installing the driver and follow it. As a result - tadam! - we get the operating system hanging at the boot stage. Here we can use a snapshot from which we can recover by calling the Time machine from the recovery partition, or we can manually delete the diseased kext from the kernel extension directory from the same recovery partition (the author generally does not recommend doing this).
Why is this happening? Because Apple took care of us. In the latest versions of macOS, you can't just pick up and inject third-party code into the boot process. The good Apple programmers have blocked such destructive behavior. More precisely, it was blocked by half, when the waiting for the driver is implemented, but the driver itself is not, so everything just hangs.
What should a sophisticated mind do before installing a driver? First, give the command:
csrutil statusIf in response to it we get:
System Integrity Protection status: enabled.
then it means that the good Apple programmers care about us, so nothing will come of us until we turn off their wonderful protection. To do this, we reboot into the recovery partition (βR), call the terminal and issue the command:
csrutil disableAfter that, we reboot into a working system, and even then we install the driver, and at the same time the ATTO configuration utility (in principle, the configuration utility is needed only for diagnostics and is not required during normal operation). Along the way, when asked, we confirm the authorization of the ATTO company in the system settings. After installation, you can again reboot into the recovery partition and give the command
csrutil enableApple is taking care of us again.
Now we have a driver-supported interface to external SAS devices (or FC if an FC converter were used). But how to work with the tape at the logical level?
As the inexperienced but erudite mind knows, any Unix-compatible system supports tape drives at the kernel level and basic system utilities, which primarily include mt (tape drive management) and tar (an archiver that has support for working with archives on tape) . However, what will the sophisticated mind clarify on this? Any Unix compatible system except macOS. Apple took care of us by removing support for tape devices from their code.
But is it really not possible to return this code by porting the standard open source Unix utilities to macOS? The good news is that Tolis (which I don't link to) has already done this with their Tolis Tape Tools product. The bad news is that the said firm estimates the use of the results of its work at $399. Estimates of this fact may be different, but personally the author is not ready to pay someone 400 bucks for a code that is mostly written by completely different people and has been in open use since the 1970s, and therefore this question is for myself considers closed. (By the way, there is a vaguely abandoned free project on github on the same subject).
Fortunately, there is an IBM corporation in the world, whose commercial appetites have a completely different scale, and therefore do not manifest themselves in every little thing. In particular, it has developed an open source LTFS tape file system, which is also distributed for macOS.
There is a nuance here, which is that different tape device manufacturers release their own versions of LTFS that support their devices. Since the author uses an IBM tape drive, I also installed LTFS from IBM. Third party drives may require their own LTFS ports. And there is a universal implementation of openLTFS on github and homebrew.
It is important for us that LTFS uses the media partitioning function, and therefore can work with devices and cartridges starting from the LTO-5 generation.
So, in our case, we download from the IBM website the IBM Spectrum Archive Single Drive Edition for macOS, which just includes the implementation of LTFS. Without any adventures, we install the product with its own installer. Along the way, he also installs the FUSE package, while in the system settings he will have to confirm the authorization of a smart programmer named Anatol Pomozov, on which the whole IBM depends in this case. Respect and respect for this man.
It is advisable to immediately write the line in the /Library/Frameworks/LTFS.framework/Versions/Current/etc/ltfs.conf.local file:
option single-drive sync_type=time@1
setting to mount the tape by default, flushing the write buffer after 1 minute of inactivity (5 minutes by default).
Finally, everything is ready to connect. We connect the chain: Mac - T3 / T2 adapter - Thunderbolt cable - ATTO converter - SAS cable - tape drive (the choice of several ports on a Mac, converter and drive is unimportant). Turn on the converter power. Turn on the tape drive. We are waiting for the end of the initialization of the drive according to its indication.
We give the command:
ltfs -o device_listHooray! We get (in the usual diagnostic manner for IBM):
307 LTFS14000I LTFS starting, LTFS version 2.4.2.0 (10418), log level 2.
307 LTFS14058I LTFS Format Specification version 2.4.0.
307 LTFS14104I Launched by "ltfs -o device_list".
307 LTFS14105I This binary is built for Mac OS X.
307 LTFS14106I GCC version is 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66)).
307 LTFS17087I Kernel version: Darwin Kernel Version 19.4.0: Wed Mar 4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64.
307 LTFS17085I Plugin: Loading "iokit" tape backend.
Tape Device list:.
Device Name = 0, Vendor ID = IBM, Product ID = ULT3580-TD5, Serial Number = **********, Product Name = [ULT3580-TD5].
We insert the cassette, wait for loading and format:
mkltfs -d 0 -nTest -r "size=10M/name=.DS_Store"Here, the -d parameter specifies the drive number (always zero if it is the only one, but cannot be omitted in this command), -n is the tape name (you can not specify it), and the -r parameter requires to place the contents of .DS_Store files that are not larger than 10 megabytes, in the index (i.e. intended for directories) section of the tape instead of the data section.
A mysterious life has gone in the tape drive. We wait a couple of minutes, we get in response:
LTFS15000I Starting mkltfs, LTFS version 2.4.2.0 (10418), log level 2.
LTFS15041I Launched by "mkltfs -d 0 -nTest -r size=10M/name=.DS_Store".
LTFS15042I This binary is built for Mac OS X.
LTFS15043I GCC version is 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66)).
LTFS17087I Kernel version: Darwin Kernel Version 19.4.0: Wed Mar 4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64.
LTFS15003I Formatting device '0'.
LTFS15004I LTFS volume blocksize: 524288.
LTFS15005I Index partition placement policy: size=10M/name=.DS_Store.LTFS11337I Update index-dirty flag (1) - NO_BARCODE (0x0x1021081e0).
LTFS17085I Plugin: Loading "iokit" tape backend.
LTFS30810I Opening a device through iokit driver (0).
LTFS30814I Vendor ID is IBM.
LTFS30815I Product ID is 'ULT3580-TD5 '.
LTFS30816I Firmware revision is H976.
LTFS30817I Drive serial is **********.
LTFS17160I Maximum device block size is 1048576.
LTFS11330I Loading cartridge.
LTFS30854I Logical block protection is disabled.
LTFS11332I Load successful.
LTFS17157I Changing the drive setting to write-anywhere mode.
LTFS15049I Checking the medium (mount).
LTFS30854I Logical block protection is disabled.
LTFS15010I Creating data partition b on SCSI partition 1.
LTFS15011I Creating index partition a on SCSI partition 0.
LTFS17165I Resetting the medium's capacity proportion.
LTFS11097I Partitioning the medium.
LTFS11100I Writing label to partition b.
LTFS11278I Writing index to partition b.
LTFS30808I READ_ATTR (0x8c) returns -20501.
LTFS30865I READ_ATTR returns Invalid Field in CDB (-20501) 0.
LTFS30836I Cannot read attribute (-20501).
LTFS11336I The attribute does not exist. Ignore the expected error.
LTFS17235I Writing index of NO_BARCODE to b (Reason: Format, 0 files) **********.
LTFS17236I Wrote index of NO_BARCODE (b, **********).
LTFS11337I Update index-dirty flag (0) - NO_BARCODE (0x0x1021081e0).
LTFS11100I Writing label to partition a.
LTFS11278I Writing index to partition a.
LTFS30808I READ_ATTR (0x8c) returns -20501.
LTFS30865I READ_ATTR returns Invalid Field in CDB (-20501) 0.
LTFS30836I Cannot read attribute (-20501).
LTFS11336I The attribute does not exist. Ignore the expected error.
LTFS17235I Writing index of NO_BARCODE to a (Reason: Format, 0 files) 9068025555.
LTFS17236I Wrote index of NO_BARCODE (a, **********).
LTFS15013I Volume UUID is: 3802a70d-bd9f-47a6-a999-eb74ffa67fc1.LTFS15019I Volume capacity is 1425 GB.
LTFS30854I Logical block protection is disabled.
LTFS15024I Medium formatted successfully.
Mount the formatted tape:
sudo mkdir /Volumes/LTFS
sudo chmod 777 /Volumes/LTFS/
sudo ltfs /Volumes/LTFSWe get a couple more minutes of drive operation, diagnostics:
307 LTFS14000I LTFS starting, LTFS version 2.4.2.0 (10418), log level 2.
307 LTFS14058I LTFS Format Specification version 2.4.0.
307 LTFS14104I Launched by "ltfs /Volumes/LTFS/".
307 LTFS14105I This binary is built for Mac OS X.
307 LTFS14106I GCC version is 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66)).
307 LTFS17087I Kernel version: Darwin Kernel Version 19.4.0: Wed Mar 4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64.
307 LTFS14063I Sync type is "time", Sync time is 60 sec.
307 LTFS17085I Plugin: Loading "iokit" tape backend.
307 LTFS17085I Plugin: Loading "unified" iosched backend.
307 LTFS14095I Set the tape device write-anywhere mode to avoid cartridge ejection.
307 LTFS30810I Opening a device through iokit driver (0).
307 LTFS30814I Vendor ID is IBM.
307 LTFS30815I Product ID is 'ULT3580-TD5 '.
307 LTFS30816I Firmware revision is H976.
307 LTFS30817I Drive serial is **********.
307 LTFS17160I Maximum device block size is 1048576.
307 LTFS11330I Loading cartridge.
307 LTFS30854I Logical block protection is disabled.
307 LTFS11332I Load successful.
307 LTFS17157I Changing the drive setting to write-anywhere mode.
307 LTFS11005I Mounting the volume.
307 LTFS30854I Logical block protection is disabled.
307 LTFS17227I Tape attribute: Vendor = IBM.
307 LTFS17227I Tape attribute: Application Name = LTFS.
307 LTFS17227I Tape attribute: Application Version = 2.4.2.0.
307 LTFS17227I Tape attribute: Medium Label =.
307 LTFS17228I Tape attribute: Text Localization ID = 0x81.
307 LTFS17227I Tape attribute: Barcode =.
307 LTFS17227I Tape attribute: Application Format Version = 2.4.0.
307 LTFS17228I Tape attribute: Volume Lock Status = 0x00.
307 LTFS17227I Tape attribute: Media Pool name =.
307 LTFS14111I Initial setup completed successfully.
307 LTFS14112I Invoke 'mount' command to check the result of final setup.
307 LTFS14113I Specified mount point is listed if succeeded.
And here it is, our ribbon on the desktop, named Test(ltfs)! An unnamed tape will be named OSXFUSE Volume 0 (ltfs).
Now you can work with her.
In general, it should be borne in mind that it is advisable not to abuse the contents of the tape directories in the finder windows, since this is an incredibly expensive operation for LTFS, but it is better to work with terminal commands, or simply dump the backup directory in bulk onto the tape, as shown in the window above.
By the way, there is a specially written IBM ltfs_copy utility and its clones, designed for more efficient copying between tape and disk, but so far the author has not been able to find them in the public domain with a superficial search.
You can unmount the tape with the command:
umount /Volumes/LTFSor just throw it in the trash.
In fact, in nature, there are some kind of graphical shells for macOS to facilitate these actions, but after such perversions, should we be afraid to type a few lines in the terminal?
As a side effect, we get the opportunity to connect external eSATA drives via a SAS / 4 * eSATA cable.
Source: habr.com