I need some advice on my idea to write a controller file in Python, and not like the C in MVC type controller but a more simpler idea. Just a Python script that controls the operation of some other Python scripts.
I will outline the idea, and any advice or ideas would be greatly appreciated.
I have to write a system that does some background tasks for a web application. The web app is in PHP and separate from the Python system I am writing, the Python will do some automated tasks and calculate and save some analytical data for the web app to use and display.
I have experience with basic Python but I have not pushed forward to learn much about class/objective Python. So far what I have written for the system is a bunch of separate py scripts that do the jobs they have to do, written totally procedurally. They all work fine, but it's not really a system yet. My initial krux was that I had to keep them all alive and running infinitely in the background. After some research I decided that the best way to do this was to have a process manager like Supervisord look after each script, I would start up the scripts and allow them to run for a certain time with a finite loop (in each script) and then they would exit, then the supervisor would restart them every time they exit and keep them alive. I wanted to avoid using infinite loops because of the possibility of memory leaks (I believe allowing the process to exit totally frees up memory associated with that process and prevents leaks). I have not tested any of this, it's just a concept at the moment.
Something I would prefer to do would be to have one script be the controller of the rest of the scripts and have the supervisor only look after the controller. This saves me having to update the supervisors config file every time I add to the Python system.
The basic idea for the controller would be to have a loop fork new threads for each separate script/process and then have the controller wait for all the other threads to complete before allowing the loop to do another iteration (starting everything again).
I'm guessing what I need to do for this would be to wrap all the code in each script in a function, then import them as modules into the controller and call each function on a new thread inside the 'main' loop of the controller. Then the loop just runs, calls each function on a new thread and waits for each thread to finish before starting again.
How does this sound as a concept?
The only direct coding question I have is about turning my scripts into modules, how do I go from totally procedural code (no functions, no classes) into an importable module? can I wrap the procedures in just a function, or do I need to have them in a class also?
Best Answer
I have a similar system but have a different concept:
cron
to auto-start processesThe module uses
psutil
package to get a list of running processes, search for the related process and returns whether is it in the list or not. That may sound practical or not according to your use case though:I have little python files which have similar code as below:
Ex:
And finally I have lines that trigger this file in my
cron
:Logic:
cron
triggers this file everyx
minutes.ProcessControl
checks whether file is running or not. When callingis_running
method, you can pass a bool value so the controller will ignore the parameters passed while running the python file. Like ifcheck_params
isTrue
following two commands will be accepted as different and both will be triggered:and disabling
check_params
will evaluate the second call as the same program and will not trigger it