You can wait()
for beanstalkd's pid; if it exits (cleanly or otherwise), wait()
will return you the exit code, and you will be able to restart the process imemdiately.
Beanstalkd persists its queue (if you specity -b
), so beanstalkd process crashing time to time (if ever) probably is not an issue. But your postgress trigger will not be able to push the data to beanstalkd queue at that moment. For this reason, I'd use a separate queue table in postgres. Transactions append records to this table. A periodic (say, once a second) process checks this table, pushes the data to beanstalkd and only removes it from queue table if beanstalkd reliably accepted the data.
With this setup, the worst case you'll have is data not being sent entirely timely to the system that beanstalkd supplies it to. Other parts of system will not need to actually pause, because once everything is in palce again, the backlog of messages will be cleared eventually.
Uncontrolled forking is definitely an anti-pattern. As you have discovered, it is relatively easy to consume all available resources, and then continue to consume them to the point where everything grinds to a complete standstill.
Just ask the Sorcerer's apprentice.
Like any problem involving recursion, you have to clearly understand your terminating condition. If your algorithm potentially never terminates, recursion is completely unsuitable since you have no control of how deep your stack can become.
Of course, some tasks may "discover" other tasks, so it may also never be possible to launch everything directly from the parent process.
One strategy often used to simplify a solution is to create some kind of task manager process, and have it consume a queue of pending tasks. The master task, or any children, can add tasks to the queue as they find them. The task manager then forks as many processes as it needs to in order to process the tasks.
In this way, the task manager can keep a count of the total number of running processes, and only fork a new process if it thinks it necessary and the hardware can cope with it.
There is a risk of blocking with this, of course: some task needs the result of another task before it can continue, yet that other task is in the queue and will never be run because all the currently executing tasks are blocking waiting for tasks that haven't been started yet...
You can fix this by having separate tasks whose job is to consume results, and managing these through a separate task manager. That way, the calculations can never become completely blocked: something is always going to be making progress.
Best Answer
Please be careful to understand that priority is not the same as order of running. As a rule, when you fork a process it runs at the same priority, unless you ask for it to be lower. The order in which parent and child run is separate.
The answer is that you cannot and should not depend on the order. In many cases after forking (or using any comparable mechanism in any other operating system), the parent will continue to run before the child starts. Or not. It all depends.
If it is important, you will have to include synchronising mechanisms to control which process runs and which waits.