Java – Web application Architecture for real time app

apache-kafkaArchitecturejavareal timespring

Usecase: End user places an order on the company website, the order needs to be pushed to company admin's screen with small latency.The company has multiple branches(identified by compId and branchId combination) so the order must be pushed to correct admin.

What I have so far: A REST API that takes Order, persists it to MongoDB and writes the message to Messaging gateway. The messaging gateway decouples my email sender service from my web app.

What I am struggling with: How to push orders to browser based admin client where delivery is guaranteed and in real time? I am thinking about using long polling for admin client, as the volume of orders is not big enough to justify WebSockets(Is this correct?). And Apache Kafka Or Redis for guaranteed delivery and real time order streaming on the server side.

Solution I have come up with: When the admin logs-in SseEmitter is created on the server and stored in a Map where the key is compID and BranchId combination. When an order comes in it is added to Kafka topic or Redis. A listener(kafka or redis)listens for these events and tries to get the SSE for this order from the Map if found write the order to the SSE. And remove the order from the topic or Redis DB.

Questions:

  1. What is better suited for this Kafka or Redis? I have no prior
    experience for both.
  2. What could be possible problems with this approach? long polling,
    messaging, etc. Is there something I am over looking?
  3. Is my assumption regarding web sockets correct?
  4. Any other insight you may have.

Best Answer

  1. Kafka and redis are not used for the same purpose. Redis is a key-value storage engine and Kafka is "streaming platform". I have never used Kafka but it seems to act like a message broker optimized for streaming

We have come across a similar demand at the company I work for. We had to mark seats as occupied on a seating plan along with people having their tickets scanned at the entrance of the theater. Here is a simplified version of our solution :

  • you need a message broker (rabbitMQ), a nodeJS daemon, a websocket client.
  • whenever a ticket is scanned, the corresponding data is sent to rabbitMQ thus creating a new message in a queue
  • the nodeJS daemon constantly checks the rabbitMQ queue for new messages.
  • whenever a supervisor connects to the display interface, a websocket is open with the nodeJS daemon. This websocket will be used to notify the client that some new data has to be fetched on the server.
  • whenever a message arrives in the queue, it is read (and destroyed to prevent it from being processed twice) by the nodeJs daemon. The daemon notifies every connected client through the websockets
  • whenever a client receives a websocket notification, it polls the fresh data from the webserver.
Related Topic