How to Monitor Processes in Python on Linux

To my knowledge, the only standard for specific technologies is "whatever works for your needs."

That said, you might want to look into Redis. I've used it before to communicate between disparate systems, and it works well.

Basically, what it is is an in-memory datastore. You set the service to watch for a specific key, and when your system does something with that watched key, the service reacts.

In a more general sense, what you're asking for is often known as a "pub-sub" (publisher-subscriber) setup. So if Redis doesn't work for you, you might be able to use the more general term to find a solution that does work. And really, if you wanted to, you could leverage Twitter to be the intermediary, if you really wanted to (the system would post a tweet, and the service would watch the twitter feed for updates). If I recall correctly, GitHub uses Hubot and Campfire in a similar manner (Hubot sits in a Campfire chat room, waiting for someone to send commands, to which Hubot responds).

I have a similar system but have a different concept:

• I use cron to auto-start processes
• Check if related process is running
• If it is not running, then start the process.

The module uses psutil package to get a list of running processes, search for the related process and returns whether is it in the list or not. That may sound practical or not according to your use case though:

import psutil

class ProcessControl(object):
    """
    This program checks whether the given python file is running or not. If check_params flag is set as True, then
    it will be checked if an instance of the python file is running with all of the given paramters. If check_params
    flag sets as False, then the check will be made with only using the python file name and instances of the file
    running with different parameters will also be count.
    """
    def __init__(self, filename, *args):
        self.filename = filename
        self.args = args
        self.arg_num = len(args)

    def process_count(self, check_params):
        process_count = 0
        # Examine the process list to check if given process is running or not...
        for _prs in psutil.process_iter():
            try:
                _cmdline = _prs.cmdline()
            except TypeError:
                _cmdline = _prs.cmdline
            if len(_cmdline) >= 2 and "python" in _cmdline[0] and self.filename in _cmdline[1] and _prs.is_running():
                # We found an instance of the process that is running. Since This control function is triggered when
                # we run the python code, There would be at least one process (this one) which is running. Counting 2
                # or more processes means there is another instance which is still running when we trigger the python
                # code file
                if check_params:
                    # We also will check the parameters for the complete similarity
                    if len(_cmdline) == 2 + self.arg_num and all(str(_arg) in _cmdline[2:] for _arg in self.args):
                        process_count += 1
                    else:
                        # This is no match...
                        pass
                else:
                    process_count += 1
            else:
                # This is no match....
                pass
        return process_count

    def is_running(self, check_params=True):
        return self.process_count(check_params) > 1

I have little python files which have similar code as below:

Ex:

myFile.py

class MyCodeClass:
    def run_code(self):
         ...


if __name__ == "__main__":
    my_code = MyCodeClass()
    try:
        if ProcessControl(__file__).is_running():
            print "Already Running"
        else:
            my_code.run_code()
    except Exception as e:
        print e

And finally I have lines that trigger this file in my cron:

* * * * * python myFile.py

Logic:

cron triggers this file every x minutes. ProcessControl checks whether file is running or not. When calling is_running method, you can pass a bool value so the controller will ignore the parameters passed while running the python file. Like if check_params is True following two commands will be accepted as different and both will be triggered:

python myOtherFile.py 127.0.0.1 220
python myOtherFile.py 127.0.0.1 250

and disabling check_params will evaluate the second call as the same program and will not trigger it