I am trying to write a Java application that works similar to MapReduce. There is a server and several workers. Workers may come and go as they please and the membership to the group has a soft-state. To become a part of the group, the worker must send a UDP datagram to the server, but to continue to be part of the group, the worker must send the UDP datagram to the server every 5 minutes. In order to accommodate temporary errors, a worker is allowed to miss as many as two consecutive periodic UDP datagrams. So, the server must keep track of the current set of workers as well as the last time each worker had sent a UDP datagram.
I've implemented a class called WorkerListener that implements Runnable and listens to UDP datagrams on a particular UDP port. Now, to keep track of active workers, this class may maintain a HashSet (or HashMap). When a datagram is received, the server may query the HashSet to check if it is a new member. If so, it can add the new worker to the group by adding an entry into the HashSet. If not, it must reset a "timer" for the worker, noting that it has just heard from the corresponding worker. I'm using the word timer in a generic sense. It doesn't have to be a clock of sorts. Perhaps this could also be implemented using int or long variables.
Also, the server must run a thread that continuously monitors the timers for the workers to see that a client that times out on two consecutive datagram intervals, it is removed from the HashSet. I don't want to do this in the WorkerListener thread because it would be blocking on the UDP datagram receive() function. If I create a separate thread to monitor the worker HashSet, it would need to be a different class, perhaps WorkerRegistrar. I must share the HashSet with that thread. Mutual exclusion must also be implemented, then.
My question is, what is the best way to do this? Pointers to some sample implementation would be great. I want to use the barebones JDK implementation, and not some fancy state maintenance API that takes care of everything, because I want this to be a useful demonstration for a class that I am teaching.
Thanks
Best Answer
The best sample implementation that I can think of is ZooKeeper.
There's a lot of architecture documentation on the ZooKeeper site that will guide you through the problems that you'll hit if you try to implement similar functionality. But I have to ask: are you really helping your students if you don't fully understand what you're teaching them?