Docker container unable to connect to MongoDB Replica set

dockerdocker-swarmmongodb

I'm using Docker Swarm to create a cluster of web applications and databases.

I've four containers running, three of them are a single mongodb replica set and the last is a asp.net core application which connects to the replica set.

All the four containers are publicaly accessible and are connected to the same docker network.

The replica set can internally connect to each other which hostnames such as: mongo1, mongo2 and mongo3.

The asp.net application however is unable to connect to the replica set.

  • I can connect to the mongo db through the mongo shell from the asp.net docker container just fine.
  • I'm able to connect to each node individually, just not as a replica set from outside.

Exception

Connection id "0HLBA78J6QB27", Request id "0HLBA78J6QB27:00000001": An unhandled exception was thrown by the application.
System.TimeoutException: A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = MongoDB.Driver.MongoClient+AreSessionsSupportedServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 } }. Client view of cluster state is { ClusterId : "1", ConnectionMode : "Automatic", Type : "Unknown", State : "Disconnected", Servers : [{ ServerId: "{ ClusterId : 1, EndPoint : "Unspecified/mongo1;mongo2;mongo3:27017" }", EndPoint: "Unspecified/mongo1;mongo2;mongo3:27017", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: No such device or address

Docker services:

Web app:
Published port: 80->80

Mongo1:
Published port: 1234->27017

Mongo2:
Published port: 1235->27017

Mongo3:
Published port: 1236->27017

Connection string looks like the following:

"mongodb://admin:password@mongo1,mongo2,mongo3/replicaSet=example&authSource=admin"

Best Answer

In order to support replica set failover and reconfiguration, clients with a replica set connection use the hostnames as configured in the replica set as required by MongoDB's Server Discovery and Monitoring (SDAM) specification. This ensures that the client view of the replica set is consistent with the replica set's configuration and status.

I'm able to connect to each node individually, just not as a replica set from outside.

This is expected behaviour if the external host/port differs from the replica set configuration. If you connect to a Single server (i.e. not specifying a replica set connection), clients do not do any server discovery. This can be useful for connecting to a specific replica set member for backup or admin purposes, or avoiding server discovery when you are connecting through a forwarded hostname or port combination that does not match the replica set configuration.

With a replica set connection, clients connect and run the isMaster command to discover the current configuration and state of the replica set. Per the SDAM spec, the clients then set up replica set monitoring so any changes in configuration or status will automatically be detected by the client.

Mongo1: Published port: 1234->27017 Mongo2: Published port: 1235->27017 Mongo3: Published port: 1236->27017

To use a replica set connection with the exposed Docker ports you will need to:

  • Make sure the hostnames used in your replica set config (Mongo1, Mongo2,Mongo3) can be resolved in the container you are trying to connect from.
  • Make the ports in the Docker images unique, and expose the same ports in your host environment (for example: 27017, 27018, 27019).

It looks like you already have name resolution working correctly if you can connect from your app container, but the ports in your replica set configuration do not match what you have exposed via Docker.

Related Topic