Missing mdam RAID1 array during scan

mdadmswap

I have built a new server (Ubuntu 18.04.1) with 4 HDD.

My plan is to use RAID1 overall and came up with this layout after investigation:

sda, sdb four primary partitions (all as fd) with sda1/sdb1 as /boot, sda2/sdb2 as /, sda3/sdb3 as /var and sda4/sdb4 as part of /home

sdc,sdd two primary partitions (all as fd) with sdc1/sdd1 as rest of /home and sdc2/sdd2 to be used as swap.

Currently the swapfile resides in /

~$ ls -lart /swapfile 
-rw------- 1 root root 2147483648 aug 17 08:50 /swapfile

and my expectation is that this swap file should be on md2 (which is missing).

Question is: Can I just create a new array with mdadm and somehow make the OS use that area for swap instead of the swapfile ?

In my first attempts I tried to prepare all configurations during install but failed miserably and in my last attempt I was only setting up /boot and / in the raid configuration and no LVM config.

The planned part worked as expected but somehow a remnant from previous attempts has left me with a missing swap partition.

What I mean is that the mdadm.conf has entry for the swap(besides md0:/boot and md1:/):

ARRAY /dev/md/2  metadata=1.2 UUID=4b13781e:ea3a8209:74173443:77c65a9d name=LOPEZSRV:2

And checking blkid:

~$ sudo blkid
/dev/sdc2: UUID="4b13781e-ea3a-8209-7417-344377c65a9d" UUID_SUB="6ee8ecac-1e75-f5dc-3def-508cba28a512" LABEL="LOPEZSRV:2" TYPE="linux_raid_member" PARTUUID="f260af1c-02"
/dev/sdd2: UUID="4b13781e-ea3a-8209-7417-344377c65a9d" UUID_SUB="3efa76ad-3662-87e6-ed4d-b072cb0e7274" LABEL="LOPEZSRV:2" TYPE="linux_raid_member" PARTUUID="3bce6013-02"

shows them being a member of an array

Also checking the details

~$ sudo mdadm -E /dev/sdc2
/dev/sdc2:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 4b13781e:ea3a8209:74173443:77c65a9d
           Name : LOPEZSRV:2  (local to host LOPEZSRV)
  Creation Time : Thu Aug 16 14:08:06 2018
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 3906762752 (1862.89 GiB 2000.26 GB)
     Array Size : 1953381376 (1862.89 GiB 2000.26 GB)
    Data Offset : 264192 sectors
   Super Offset : 8 sectors
   Unused Space : before=264112 sectors, after=18446744073701738496 sectors
          State : active
    Device UUID : 6ee8ecac:1e75f5dc:3def508c:ba28a512

Internal Bitmap : 8 sectors from superblock
    Update Time : Thu Aug 16 14:40:25 2018
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : 19220d93 - correct
         Events : 393


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)

~$ sudo mdadm -E /dev/sdd2
/dev/sdd2:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 4b13781e:ea3a8209:74173443:77c65a9d
           Name : LOPEZSRV:2  (local to host LOPEZSRV)
  Creation Time : Thu Aug 16 14:08:06 2018
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 3906762752 (1862.89 GiB 2000.26 GB)
     Array Size : 1953381376 (1862.89 GiB 2000.26 GB)
    Data Offset : 264192 sectors
   Super Offset : 8 sectors
   Unused Space : before=264112 sectors, after=18446744073701738496 sectors
          State : active
    Device UUID : 3efa76ad:366287e6:ed4db072:cb0e7274

Internal Bitmap : 8 sectors from superblock
    Update Time : Thu Aug 16 14:40:25 2018
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : 6b6a513d - correct
         Events : 393


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)

But doing a scan:

~$ sudo mdadm --detail --scan
ARRAY /dev/md/1 metadata=1.2 name=LOPEZSRV:1 UUID=5c7e415a:fe060391:ae611987:474558ec
ARRAY /dev/md/0 metadata=1.2 name=LOPEZSRV:0 UUID=e6a5ed7b:c613d763:e8e6c8a3:8a13ecb1

I only get the arrays for /boot and /

Looking in /proc/mdstat:

~$ cat /proc/mdstat 
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid1 sda1[0] sdb1[1]
      1949696 blocks super 1.2 [2/2] [UU]

md1 : active raid1 sdb2[1] sda2[0]
      97589248 blocks super 1.2 [2/2] [UU]

also only shows the two arrays for /boot and /.

Adding outputs:

~$ sudo mdadm --assemble --scan  --verbose 
mdadm: looking for devices for /dev/md/0
mdadm: No super block found on /dev/md/0 (Expected magic a92b4efc, got 000000ef)
mdadm: no RAID superblock on /dev/md/0
mdadm: No super block found on /dev/md/1 (Expected magic a92b4efc, got 0000040d)
mdadm: no RAID superblock on /dev/md/1
mdadm: /dev/sdd2 has wrong uuid.
mdadm: No super block found on /dev/sdd1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd1
mdadm: No super block found on /dev/sdd (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd
mdadm: /dev/sdc2 has wrong uuid.
mdadm: No super block found on /dev/sdc1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc1
mdadm: No super block found on /dev/sdc (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc
mdadm: No super block found on /dev/sdb4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb4
mdadm: No super block found on /dev/sdb3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb3
mdadm: /dev/sdb2 has wrong uuid.
mdadm: /dev/sdb1 is busy - skipping
mdadm: No super block found on /dev/sdb (Expected magic a92b4efc, got a0ddb71c)
mdadm: no RAID superblock on /dev/sdb
mdadm: No super block found on /dev/sda4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda4
mdadm: No super block found on /dev/sda3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda3
mdadm: /dev/sda2 has wrong uuid.
mdadm: /dev/sda1 is busy - skipping
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got a0ddb71c)
mdadm: no RAID superblock on /dev/sda
mdadm: cannot open device /dev/sr0: No medium found
mdadm: looking for devices for /dev/md/1
mdadm: No super block found on /dev/md/0 (Expected magic a92b4efc, got 000000ef)
mdadm: no RAID superblock on /dev/md/0
mdadm: No super block found on /dev/md/1 (Expected magic a92b4efc, got 0000040d)
mdadm: no RAID superblock on /dev/md/1
mdadm: /dev/sdd2 has wrong uuid.
mdadm: No super block found on /dev/sdd1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd1
mdadm: No super block found on /dev/sdd (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd
mdadm: /dev/sdc2 has wrong uuid.
mdadm: No super block found on /dev/sdc1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc1
mdadm: No super block found on /dev/sdc (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc
mdadm: No super block found on /dev/sdb4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb4
mdadm: No super block found on /dev/sdb3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb3
mdadm: /dev/sdb2 is busy - skipping
mdadm: /dev/sdb1 has wrong uuid.
mdadm: No super block found on /dev/sdb (Expected magic a92b4efc, got a0ddb71c)
mdadm: no RAID superblock on /dev/sdb
mdadm: No super block found on /dev/sda4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda4
mdadm: No super block found on /dev/sda3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda3
mdadm: /dev/sda2 is busy - skipping
mdadm: /dev/sda1 has wrong uuid.
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got a0ddb71c)
mdadm: no RAID superblock on /dev/sda
mdadm: cannot open device /dev/sr0: No medium found
mdadm: looking for devices for /dev/md/2
mdadm: No super block found on /dev/md/0 (Expected magic a92b4efc, got 000000ef)
mdadm: no RAID superblock on /dev/md/0
mdadm: No super block found on /dev/md/1 (Expected magic a92b4efc, got 0000040d)
mdadm: no RAID superblock on /dev/md/1
mdadm: No super block found on /dev/sdd1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd1
mdadm: No super block found on /dev/sdd (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd
mdadm: No super block found on /dev/sdc1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc1
mdadm: No super block found on /dev/sdc (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc
mdadm: No super block found on /dev/sdb4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb4
mdadm: No super block found on /dev/sdb3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb3
mdadm: /dev/sdb2 has wrong uuid.
mdadm: /dev/sdb1 has wrong uuid.
mdadm: No super block found on /dev/sdb (Expected magic a92b4efc, got a0ddb71c)
mdadm: no RAID superblock on /dev/sdb
mdadm: No super block found on /dev/sda4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda4
mdadm: No super block found on /dev/sda3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda3
mdadm: /dev/sda2 has wrong uuid.
mdadm: /dev/sda1 has wrong uuid.
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got a0ddb71c)
mdadm: no RAID superblock on /dev/sda
mdadm: cannot open device /dev/sr0: No medium found
mdadm: /dev/sdd2 is identified as a member of /dev/md/2, slot 1.
mdadm: /dev/sdc2 is identified as a member of /dev/md/2, slot 0.
mdadm: failed to add /dev/sdd2 to /dev/md/2: Invalid argument
mdadm: failed to add /dev/sdc2 to /dev/md/2: Invalid argument
mdadm: failed to RUN_ARRAY /dev/md/2: Invalid argument

~$ sudo mdadm --assemble /dev/md2 --uuid=4b13781e:ea3a8209:74173443:77c65a9d --verbose
mdadm: looking for devices for /dev/md2
mdadm: no recogniseable superblock on /dev/md/0
mdadm: no recogniseable superblock on /dev/md/1
mdadm: No super block found on /dev/sdd1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd1
mdadm: No super block found on /dev/sdd (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd
mdadm: No super block found on /dev/sdc1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc1
mdadm: No super block found on /dev/sdc (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc
mdadm: No super block found on /dev/sdb4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb4
mdadm: No super block found on /dev/sdb3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb3
mdadm: /dev/sdb2 has wrong uuid.
mdadm: /dev/sdb1 has wrong uuid.
mdadm: No super block found on /dev/sdb (Expected magic a92b4efc, got a0ddb71c)
mdadm: no RAID superblock on /dev/sdb
mdadm: No super block found on /dev/sda4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda4
mdadm: No super block found on /dev/sda3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda3
mdadm: /dev/sda2 has wrong uuid.
mdadm: /dev/sda1 has wrong uuid.
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got a0ddb71c)
mdadm: no RAID superblock on /dev/sda
mdadm: cannot open device /dev/sr0: No medium found
mdadm: /dev/sdd2 is identified as a member of /dev/md2, slot 1.
mdadm: /dev/sdc2 is identified as a member of /dev/md2, slot 0.
mdadm: failed to add /dev/sdd2 to /dev/md2: Invalid argument
mdadm: failed to add /dev/sdc2 to /dev/md2: Invalid argument
mdadm: failed to RUN_ARRAY /dev/md2: Invalid argument

~$ cat /proc/mdstat 
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid1 sda1[0] sdb1[1]
      1949696 blocks super 1.2 [2/2] [UU]

md1 : active raid1 sdb2[1] sda2[0]
      97589248 blocks super 1.2 [2/2] [UU]

unused devices: <none>

~$ sudo modprobe raid1
~$ sudo mdadm --assemble /dev/md2 --uuid=4b13781e:ea3a8209:74173443:77c65a9d --verbose --force --update=summaries
mdadm: looking for devices for /dev/md2
mdadm: no recogniseable superblock on /dev/md/0
mdadm: no recogniseable superblock on /dev/md/1
mdadm: No super block found on /dev/sdd1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd1
mdadm: No super block found on /dev/sdd (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd
mdadm: No super block found on /dev/sdc1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc1
mdadm: No super block found on /dev/sdc (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc
mdadm: No super block found on /dev/sdb4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb4
mdadm: No super block found on /dev/sdb3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb3
mdadm: /dev/sdb2 has wrong uuid.
mdadm: /dev/sdb1 has wrong uuid.
mdadm: No super block found on /dev/sdb (Expected magic a92b4efc, got a0ddb71c)
mdadm: no RAID superblock on /dev/sdb
mdadm: No super block found on /dev/sda4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda4
mdadm: No super block found on /dev/sda3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda3
mdadm: /dev/sda2 has wrong uuid.
mdadm: /dev/sda1 has wrong uuid.
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got a0ddb71c)
mdadm: no RAID superblock on /dev/sda
mdadm: cannot open device /dev/sr0: No medium found
mdadm: --update=summaries not understood for 1.x metadata
double free or corruption (!prev)
Avbruten (SIGABRT)

~$ dmesg    
[lör aug 18 08:35:54 2018] md: md2 stopped.
[lör aug 18 08:35:54 2018] md: sdd2 does not have a valid v1.2 superblock, not importing!
[lör aug 18 08:35:54 2018] md: md_import_device returned -22
[lör aug 18 08:35:54 2018] md: md2 stopped.

Best Answer

To assemble drives which are not in the /etc/mdadm.conf one can use mdadm --assemble --scan. This command will let mdam look through all drives that are not active in an mdadm array and search for a mdadm superblock. If mdadm superblocks are discovered, and have the same UUID mdadm will try to automatically assemble the arrays. As your output shows from blkid that uuids are the same, this proceedure should fix your problem.

Did you try mdadm --assemble --scan?