Mongo secondaries stuck at startup state

database-replicationmongodbreplication

I have a MongoDB replica set with secondaries that won't make it past the STARTUP state. The database is large but they should have moved to STARTUP2 by now. The correct ports are open as I can connect to the secondaries from the primary and vice-versa.

Best Answer

The next steps resolved the similar trouble:

  1. On the PRIMARY member:

    rs.status()
{
"set" : "ShardD",
"date" : ISODate("2015-08-28T17:01:40.647Z"),
"myState" : 1,
"members" : [
        {
                "_id" : 0,
                "name" : "host.example.com:27017",
                "health" : 1,
                "state" : 1,
                "stateStr" : "PRIMARY",
                "uptime" : 1167,
                "optime" : Timestamp(1440780623, 1),
                "optimeDate" : ISODate("2015-08-28T16:50:23Z"),
                "electionTime" : Timestamp(1440780252, 2),
                "electionDate" : ISODate("2015-08-28T16:44:12Z"),
                "configVersion" : 3,
                "self" : true
        },
        {
                "_id" : 1,
                "name" : "192.0.2.222:27017",
                "health" : 1,
                "state" : 0,
                "stateStr" : "STARTUP",
                "uptime" : 584,
                "optime" : Timestamp(0, 0),
                "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                "lastHeartbeat" : ISODate("2015-08-28T17:01:40.601Z"),
                "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                "pingMs" : 0,
                "configVersion" : -2
        },
        {
                "_id" : 2,
                "name" : "192.0.2.223:27017",
                "health" : 1,
                "state" : 0,
                "stateStr" : "STARTUP",
                "uptime" : 676,
                "lastHeartbeat" : ISODate("2015-08-28T17:01:40.643Z"),
                "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                "pingMs" : 0,
                "configVersion" : -2
        }
],
"ok" : 1

    }

  2. Check a name of the PRIMARY member (usually it is equal"_id" : 0)

  3. If the name has a domain name form, try rewrite the name to IP address. Change Hostnames in a Replica Set:

    cfg = rs.conf()
cfg.members[0].host = "192.0.2.221"
rs.reconfig(cfg)

After it replica set came to normal status.

P.S. host.example.com was resolved to IP address on all hosts.

Related Topic