Which Java blocking queue is best for multiple producer and single or multiple consumers scenarios?
I am testing with LinkedBlockingQueue but I am getting OutOfMemoryError exception.
I am trying to achieve following things.
- producer creates a object & put in a queue.
- consumer grabs the data from queue & insert into database. There would be 400 producers and I can adjust consumers as I wish.
Let me know any idea.
Update
Producer : It should listen to Server Socket. It reads the data from socket & construct the object (Domain objects) and put in a queue.
Consumer : Take the object from queue & insert into DB (Hiberante & connection pooling supported)
This my real environment. Process should be able to process at least 200 records/sec.I am testing the scalability of process and how to improve that. I hope this will give better idea.
Helpful links :
Best Answer
The problem is not which
Queue
implementation you use but rather how to approach the problem of throttling your producers if your consumers cannot keep up. One possible solution is to create aLinkedBlockingQueue
with a fixed capacity, and have your producers calloffer(E e)
, which will returnfalse
if the queue is full.Another possible solution is to tailor the number of producers and consumers accordiingly.