Linux – Reconnecting secondary DRBD node

debiandrbdlinuxsoftware-raid

I have set up test DRBD installation succesfully, with switching, synchronization etc working correctly.

But I also wanted to test the reliatbility of the setup, so I gave it a good "trashing" like disconnecting network interfaces midflight, etc. In the course of such testing I have "managed" to get secondary to Diskless state:

root@h2 ~ % drbd-overview 
 0:r0/0  Connected Secondary/Primary Diskless/UpToDate

Now I cannot connect the resource back:

root@h2 ~ % drbdadm  --discard-my-data connect r0
r0: Failure: (102) Local address(port) already in use.
Command 'drbdsetup-84 connect r0 ipv4:10.0.1.90:7788 ipv4:10.0.0.90:7788 --protocol=C --max-buffers=36k --sndbuf-size=1024k --rcvbuf-size=2048k --discard-my-data' terminated with exit code 10

Even rebooting secondary does not help (I still get the message r0: Failure: (102) Local address(port) already in use.

Primary:

root@h1 ~ % drbd-overview 
 0:r0/0  Connected Primary/Secondary UpToDate/Diskless

Configuration:

root@h1 ~ % cat /etc/drbd.d/r0.res 
resource r0 {
        protocol C;
        startup {
                wfc-timeout  15;
                degr-wfc-timeout 60;
        }
        disk {
                on-io-error     detach;
                c-fill-target   10M;
                c-max-rate      700M;
                c-plan-ahead    7;
                c-min-rate      4M;
        }
        net {
                # max-epoch-size  20000;
                max-buffers       36k;
                sndbuf-size       1024k;
                rcvbuf-size       2048k;
        }
        syncer {
                rate                    400M;
                al-extents              6433;
        }
        on h1 {
                device /dev/drbd0;
                disk /dev/sdc1;
                address 10.0.0.90:7788;
                meta-disk internal;
        }
        on h2 {
                device /dev/drbd0;
                disk /dev/vg/drbdtest2;
                address 10.0.1.90:7788;
                meta-disk internal;
        }
}

OS:

% lsb_release -a
Description:    Debian GNU/Linux 9.6 (stretch)
Release:    9.6
Codename:   stretch

% dpkg -l | grep drbd
ii  drbd-utils                                    8.9.10-2                                   amd64        RAID 1 over TCP/IP for Linux (user utilities)

Best Answer

DRBD is already Connected, but it's in a Diskless disk state. This means that there is either something wrong with the backing disk causing it to throw IO errors, or something else is preventing DRBD from accessing that disk.

/var/log/syslog should be full of messages surrounding the issue, but you can also try a, drbdadm attach r0 from the node that's Diskless to see what it tells you.

Related Topic