Puppet: DRBD Resource not promoting in Corosync/Pacemaker

corosyncdrbdpacemakerpuppet

I'm trying to recreate a complete cluster setup from blank using puppet. Everything up to this point works, but now the cluster doesn't promote the DRBD resource on either node.

This is the working cluster configuration that I'm trying to recreate:

primitive drbd_mysql ocf:linbit:drbd \
    params drbd_resource="r0" \
    op monitor interval="60s" role="Master" \
    op monitor interval="61s" role="Slave"
primitive fs_mysql ocf:heartbeat:Filesystem \
    params device="/dev/drbd/by-res/r0" directory="/var/lib/mysql/" fstype="ext4"
primitive ip_mysql ocf:heartbeat:IPaddr2 \
    params ip="10.0.7.20" cidr_netmask="24" nic="eth0" \
    op monitor interval="120s" timeout="60s"
primitive mysqld ocf:heartbeat:mysql \
    params binary="/usr/bin/mysqld_safe" \
    op start interval="0" timeout="120" \
    op stop interval="0" timeout="120" \
    op monitor interval="10" timeout="30" depth="0"
group mysql fs_mysql ip_mysql mysqld
ms ms_drbd_mysql drbd_mysql \
    meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" target-role="Master"
colocation mysql_on_drbd inf: mysql ms_drbd_mysql:Master
order mysql_after_drbd inf: ms_drbd_mysql:promote mysql:start

This is the result of the puppet recipe:

primitive drbd_mysql ocf:linbit:drbd \
    op monitor interval=60s role=Master \
    params drbd_resource=r0
primitive fs_mysql Filesystem \
    params device="/dev/drbd/by-res/r0" directory="/var/lib/mysql/" fstype=ext4
primitive ip_mysql IPaddr2 \
    op monitor interval=120s timeout=60s \
    params ip=10.0.7.20 cidr_netmask=24 nic=eth0
primitive mysqld mysql \
    op monitor interval=30 timeout=30 \
    op start interval=0 timout=120 \
    op stop interval=0 timout=120 \
    params binary="/usr/bin/mysqld_safe"
group mysql fs_mysql ip_mysql mysqld
ms ms_drbd_mysql drbd_mysql \
    meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true target-role=Master
order mysql_after_drbd inf: ms_drbd_mysql:promote mysql:start symmetrical=true
colocation mysql_on_drbd inf: ms_drbd_mysql:Master mysql

The only differences that I can spot are that

  • the drbd resource's slave monitor operation is missing
  • the colocation's resources are switched around
  • pupped added a default-value "symmetrical=true" in the order resource

According to my understanding the colo should not matter, since it just specifies that both resources MUST run on the same node, but does not mean they must run in any specific order. That's what the order resource is for.

The "symmetrical=true" just means that mysql must be stopped before the other node can be promoted in the ms resource. This should not matter either.

That leaves us with the missing monitor operation. This is the puppet config that leads to this result:

cs_primitive { 'drbd_mysql':
    primitive_class => 'ocf',
    provided_by    => 'linbit',
    primitive_type  => 'drbd',
    parameters      => { 'drbd_resource' => 'r0'  },
    promotable      =>  true,
    operations      => {  'monitor' =>  { 'interval'  =>  '60s', 'role' =>  'Master'},
#                         'monitor' =>  { 'interval'  =>  '61s', 'role' =>  'Slave' },
                        }, # end operations
    ms_metadata     =>  { 'master-max'      =>  '1',
                          'master-node-max' =>  '1',
                          'clone-max'       =>  '2',
                          'clone-node-max'  =>  '1',
                          'notify'          =>  'true',
                          'target-role'     =>  'Master',
                        },  # end MS_metadata
  }

Yes, in this config there is a second monitor op, and it's commented out. If I un-comment it, the slave monitor op with an interval of 61s replaces the master op.

(Sidenote: Corosync/Pacemaker refuses the config if both have the same value for the interval.)

How can I get both monitor ops from puppet into the cluster config?

Is something else wrong with the config that you can spot?

Best Answer

Turns out the answer was pretty simple, once I found it. You need to give the op in question an array of values:

operations      => {  'monitor' =>  
                        [ { 'interval'  =>  '60s', 'role' =>  'Master'},
                          { 'interval'  =>  '61s', 'role' =>  'Slave' },
                        ] 
                   },