Good day.
I wanted to draw the community's attention to a characteristic feature of Linux when working with multiple NVMe SSDs on the same system. It will be especially relevant for those who like to make software RAID arrays from NVMe.
I hope that the information below will help protect your data and save you from annoying mistakes.
We are all used to the following Linux logic when dealing with block devices:
If the device is called /dev/sda then the partitions on it will be /dev/sda1, /dev/sda2, etc.
To view the SMART attributes, we use something like smartctl -a /dev/sda, and format, and add sections to the arrays, like /dev/sda1.
We are all used to the axiom that /dev/sda1 resides on /dev/sda. And, if one day SMART shows that /dev/sda is almost dead, it is /dev/sda1 that we will throw out of the RAID array for replacement.
It turns out that when working with NVMe Namespaces, this rule does not work. Proof:
nvme list && ( smartctl -a /dev/nvme0 && smartctl -a /dev/nvme1 && smartctl -a /dev/nvme2 ) | grep Serial
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 S466NX0K72XX06M Samsung SSD 970 EVO 500GB 1 96.92 GB / 500.11 GB 512 B + 0 B 1B2QEXE7
/dev/nvme1n1 S466NX0K43XX48W Samsung SSD 970 EVO 500GB 1 91.00 GB / 500.11 GB 512 B + 0 B 1B2QEXE7
/dev/nvme2n1 S466NX0K72XX01A Samsung SSD 970 EVO 500GB 1 0.00 B / 500.11 GB 512 B + 0 B 1B2QEXE7
Serial Number: S466NX0K72XX06M
Serial Number: S466NX0K72XX01A
Serial Number: S466NX0K43XX48W
The attentive reader will notice by comparing the serial numbers that /dev/nvme1n1 actually resides on /dev/nvme2, and vice versa.
PS
I wish you never remove the last living NVMe SSD from the RAID array.
Source: habr.com