Restart Galera Cluster – all nodes with same sequence number

galeramariadb

I have a 3 node Galera cluster running MariaDB on Ubuntu 18. I ran a few batch files to add data to the database, then shut down each node as this cluster is being relocated to a different subnet at a new location. After trying to bring the cluster back up, each node fails. Examining /var/lib/mysql/grastate.dat for ALL nodes show seqno = -1, safe_to_bootstrap = 0. I ran mysqld –wsrep-recover and ALL nodes report WSREP: Recovered position :100344, which is expected since the nodes were idle for sometime before shutdown.
The question is how to bring up the cluster again, I keep getting "Failed to start MariaDB 10.1.38 database server" when checking status. I set safe_to_bootstrap: 1 but that did not work.

Best Answer

The working fix for this was executing sudo galera_new_clusteron any node. I had the other pair shutdown when executing this, then started each in turn and the cluster was back up.