Windows – How to avoid “in use by another process” errors when piping output through a program

command-line-interfacepipewindows

In a Windows environment, are executable programs that are used to process piped output guaranteed to be unique tasks if called from different CMD environments?

For example, suppose script1.cmd contains this line:

echo "some logging information" | my_pipe_program >> logfile1.txt

Also there is script2.cmd that contains this line:

echo "some other information" | my_pipe_program >> logfile2.txt

If script1 and script2 happen to run at the same time, is Windows going to call separate instances of my_pipe_program.exe?

I'm getting sporadic errors where a command script fails with the error "The process cannot access the file because it is being used by another process"
This error always occurs right after a line that pipes through my_pipe_program.exe.
It causes the CMD script to stop right at that spot, leaving the CMD window open.

The conflict could be with my_pipe_program.exe or it could be logfile2.txt. The program my_pipe_program.exe is used in multiple CMD files, which could potentially run at the same time. The logfile is unique to the CMD file, so it seems less likely to be the culprit, unless Windows is failing to close it after the CMD file exists. Is there any approach to diagnose the problem and fix the error?

Related question – is there any way to trap and handle the "…being used by another process" error and prevent the CMD file from aborting?

Best Answer

It sounds like the issue may be with that EXE and how it handles being called on the same machine running concurrently in memory perhaps. Enabling advanced logging at this level to capture verbose detail or stack traces while emulating concurrent runs in memory from the same machine just like when the issue occurs may be best for the most accurate answer WHY it's doing this.

See my answer here (SuperUser) on SuperUser from the other day how to kill a process with batch if the CMD or BAT file is LIKE a certain name.

You could incorporate something like this to check and see if maybe something has been running for over a certain period of time with those batch file names and kill but you'd have to look at other options in WMIC to see if there's a timer, etc. if that'd work in your case. Fixing the EXE logic at that level and catching the error and ending may be best though if you control at that level.

ADDITIONAL IDEAS

A simple solution may be to create a lock file when the batch CMD file kicks off so all scripts that launch that same EXE from the same machine first check to see if the lock file exists, if it does, then end the script (or perhaps pause for so many minutes and then check again, etc.).

You could just end the script [GOTO locked] or pause it for so many seconds, then go to that routine [GOTO lockedrecheck] and check again if the other process is done and deleted the file if that'd suffice in your case.

If the lock file doesn't exist, it'd create the file, then run the process and when it's done, delete the lock file so other processes can run (example below).

You can also add the 2<&1 to the end of your log file to get more detail if the EXE allows for it to be captured that way where it wouldn't otherwise (example script logic below).

LOCK FILE EXAMPLE (end it or pause it and recheck)

<ABOVE LOGIC>
<ABOVE LOGIC>
<ABOVE LOGIC>
<ABOVE LOGIC>

:SetLockFile
::SET check for, and create lock file to ensure concurrent runs aren't possible
SET lockfile=\\server\share\lockfiles\my_pipe_program.lck
IF EXIST %lockfile% GOTO locked
::IF EXIST %lockfile% GOTO lockedrecheck 
ECHO This file is locked until current %~nx0 script ends or finishes using the EXE >> %lockfile%

ECHO "some logging information" | my_pipe_program >> %logfile% 2<&1

<More Logic>
<More Logic>
<More Logic>
<More Logic>

:end
ECHO ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> %logfile%
ECHO End: %date% at %time%                       >> %logfile%
ECHO ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> %logfile%
DEL /Q /F "%lockfile%"
ENDLOCAL
GOTO EOF

:locked
:: Come here to end the script since the lock file does exist
ECHO + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +>> %logfile%
ECHO LOCKED FILE: %~nx0 - %date% at %time%: FILE IS IN USE AND LOCKED                             >> %logfile%
ECHO + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +>> %logfile%
ENDLOCAL
GOTO EOF

:lockedrecheck
:: Come here to Pause the script with a loopback ping to nul for 3 minutes or roughly 180 seconds
PING 127.0.0.1 -n 180 > nul
GOTO :SetLockFile

2<&1 Example

echo "some logging information" | my_pipe_program >> logfile1.txt 2<&1
Related Topic