Good afternoon, dear
It was Thursday evening, and one of our admins needed to resize the disk on one of the KVM virtual machines. Seemingly a trivial task, but it could lead to complete data loss... So... the whole story is under the cut
As I mentioned, on Thursday evening (it didn't seem to be raining) one of our admins decided to address a long-standing task and increase the size of the FS inside the virtual machine KVM.
Previously, I had already increased the disk size from 14GB to 60GB, and the admin just needed to expand the FS inside the virtual machine.
Around midnight, the admin sent a message asking whether the partition should be extended or primary... In response, I told him to do it as it had been done before on the virtual machine.
Time passed... and the admin reported that he was encountering errors stating that the partition could not be expanded... and it stopped mounting... it was already 2 AM...
I told him not to do anything further and to leave the virtual machine alone while I went to make a backup of the VM disk image, naming it vmname_bad
It was further complicated by the fact that the admin did not take a snapshot or copy the partition layout before his actions... Having this information would have allowed us to roll back and try again.
In the morning, with fresh thoughts, I booted a virtual machine with the same OS (Debian 9) and connected the disk. Through fdisk, I saw that the disk was already expanded to 60GB and the partition... which was slightly damaged.
I tried to find the previous layout based on the screenshots provided by the admin, but alas, it was in vain. I attempted to figure out values through fdisk, but unfortunately, all attempts failed.
Since fdisk couldn't help me... I called in parted. We loaded parted—I deleted the old partition rm 2 and, knowing the approximate partition values, I did a rescue—specifying the start and end values where the partition could be. After a moment's wait, parted found the partition and offered to put the information into the system—I agreed and exited parted.
I mounted the partition—everything was OK. The files were in place and everything was good, but the size was still the old 14GB. I unmounted /dev/sdd1 and ran resize2fs /dev/sdd1, then e2fsck /dev/sdd1 and mounted it again to see the extended partition with all the files, and it was quite alive.
Everything ended well for both me and the admin.
Source: habr.com
