Design – System Design: Scalable Chat Server

Architecturechatdesignscalability

Suppose you were asked to design a scalable chat server with the following requirements:

  1. The main use case is: player A sees B online, A sends a message to B, B receives it.
  2. The secondary use case is: player A sees B offline, A sends a message to B. When B comes back online, B receives the message. (No push notifications).
  3. The goal is to minimize latency. Speed matters.
  4. The messages should arrive in order. We cannot lose messages but receiving duplicates once in a while is fine.
  5. Just text data, no binary data.
  6. No need to store chat history. Once sent, the messages can be destroyed.

I've been reading this article: How League Of Legends Scaled Chat To 70 Million Players and I think I missed the core architecture they used in the game. But anyway here is my "thought process". Can someone have a look at it?

  • If the secondary use case didn't exist, I wouldn't have to store anything. I think I could use a p2p network, wherein a user regularly sends a ping message "i'm online" to all his friends to notify of presence.
  • But since I have to store messages to be able to deliver them later, I need my own servers that store user presence, user friendship lists, and messages.
  • The goal of minimized latency can be achieved by placing servers close to the users. This means that there will be more than one server so they need to stay in sync. Also, we need to load balance them so that one server does not store everything.
  • I've read somewhere on the Internet that a way to load balance the servers is to assign a server to each user. So for example server 1 gets assigned everything related to user A, and server 2 gets assigned everything related to user B. We could decide this by proximity.
  • When A sends something to B, there has to be a way to dispatch the message to server 2. Maybe use a service bus to communicate servers.

  • So the flow would be something like this:

    1. A writes message "Hi B!"
    2. Server 1 receives message and B. Since it does not find B in his user base, he forwards the message to the service bus. He stores a copy of the message.
    3. The service bus requests all servers to look for user B.
    4. Server 2 replies that he has B in his user base.
    5. Server 2 receives the message and stores it.
    6. Server 2 sends message to user B.
    7. Server 2 signals to the service bus that the message was sent. He destroys the message.
    8. Server 1 destroys his copy of the message.

  • If B were offline, everything up to step 5 would stay the same. The difference is that server 1 can destroy his copy of the message but server 2 cannot.

  • Now, storage… My guess is that each server should have their own persistent storage, but I've no idea what should be optimized here (speed of reads? speed of writes?). Also I'm not sure if a MySQL store or a NoSQL store would be better. Since NoSQL is optimized to be partitioned and there's no need here I guess MySQL would be enough.
  • If a server crashes we need a way to failover quickly. I suppose we could place like a "primary" and "secondary" server in each location, the primary would be connected to primary storage and the secondary to replicated data.

So the overall architecture would look like this:

architecture

I realize I am missing many many things here, did I miss something obvious? Is there any part of my thought process just plain wrong?

Best Answer

You can use a P2P network, but it's architecturally interesting.

Using something like Kademlia as a DHT for peer discovery means talking to a limited number of nodes before reaching your target. If you stored your message at each of these hops, you'd have redundancy for your message store that may be reliable enough for your requirements. Offline delivery would mean periodic forwarding attempts for each buffered message. That would guarantee fairly low latency in terms of peer discovery, which is probably the most costly part of the problem.

Once a direct P2P connection is established, you're clearly in online mode and can skip offline storage (or not).

You can also run nodes that persistently store messages, but otherwise act as regular DHT participants. That would give more reliability, at the cost of only running a handful of nodes.

As @aridlehoover writes, though, there are so many possible answers that you can't really provide a final one.

Related Topic