Had a server that decided to drop a disk (or disk went faulty) in a RAID5 array. On reboot the array didn’t want to start. Output of mdadm --detail /dev/md0
/dev/md0: Version : 1.2 Creation Time : Wed May 22 18:17:58 2013 Raid Level : raid5 Used Dev Size : -1 Raid Devices : 3 Total Devices : 2 Persistence : Superblock is persistent Update Time : Sun Jan 3 06:50:05 2021 State : active, degraded, Not Started Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 512K Name : nebula:0 (local to host nebula) UUID : 962b8ff0:00d88161:5a030e1f:236466af Events : 31168 Number Major Minor RaidDevice State 4 8 16 0 active sync /dev/sdb 1 0 0 1 removed 3 8 48 2 active sync /dev/sdd
Try to start array with mdadm --run /dev/md0
mdadm: failed to run array /dev/md0: Input/output error
As expected, given it didn’t autorun on boot.
Trying mdadm --assemble --run /dev/md0 /dev/sdb /dev/sdd
mdadm: /dev/sdb is busy - skipping mdadm: /dev/sdd is busy - skipping
This is because the md device is using the disks. Stop it: with mdadm --stop /dev/md0
then reassemble with mdadm --assemble --run --force /dev/md0 /dev/sd[bd]
mdadm: Marking array /dev/md0 as 'clean' mdadm: /dev/md0 has been started with 2 drives (out of 3).
Ran mount -a to rerun fstab. Took a few seconds but worked. Copy data off ASAP!