In one of our projects, we've built a Publisher – Subscriber Engine on Oracle Service Bus. The functionality being a series of events are published and subscribers (JMS queues) receive these whenever a new event is published.
We are facing some technical issues now, performance-wise and hence an architectural review is underway.
Now for my questions:
Architecturally the ESB has to publish events into a DB and read from the DB which users wish to be notified, then push the event onto their respective queues.
There is a high amount of DB interaction and the question is whether ESB should be having such high amount of interaction with the DB in the first place? Or should there have been some alternate component responsible for doing this.
Alternately is there any non-DB approach in which we can store the events and subscribers? Where else can this application data be held within the ESB context?
Best Answer
Have you considered event-driven systems, such as Complex Event Processing engines? These are tools designed to handle high volume streaming events. Some support integration with a database for historical correlation and data storage.
A prime example of an open-source CEP engine is Esper, although I believe database integration is part of the commercial Esper Enterprise solution. I haven't used this particular tool before, but have researched and developed with similar applications. I'm not sure what the performance hit of adding the database is, but they are designed for situations with a lot of events being processed quickly and maintaining a high event throughput.
At the very least, perhaps event-driven architectural models might be something you can look at and adapt as necessary to your particular problem.