Python – Broken Pipe from subprocess.Popen.communciate() with stdin

python

I'm having a strange issue when using subprocess.Popen.communicate(). For background, I want to execute an application from my python script. When I run the program from the command line, I do it like this (UNIX):

$ echo "input text" | /path/to/myapp

From my script, I also want to pipe the input into the application. So, I tried the following. But I get a "broken pipe" error when I try to send the input with communicate():

>>> cmd = ['/path/to/myapp']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
>>> out,err = p.communicate('input text')
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.5/subprocess.py", line 670, in communicate
    return self._communicate(input)
  File "/usr/lib/python2.5/subprocess.py", line 1223, in _communicate
    bytes_written = self._write_no_intr(self.stdin.fileno(), buffer(input, input_offset, 512))
  File "/usr/lib/python2.5/subprocess.py", line 1003, in _write_no_intr
    return os.write(fd, s)
OSError: [Errno 32] Broken pipe

To make matters stranger, if I leave out the input data, I don't get any errors. However, this isn't really a good workaround because the application needs input to work.

>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
>>> out,err = p.communicate()
>>> print out
[error from myapp regarding lack of input]

Any idea what I'm missing?

Best Answer

Your observation suggests that myapp is terminating without reading (all of the) input. Not knowing anything about myapp, that's hard to confirm, but consider for example

$ echo 'hello world' | tr 'l' 'L'
heLLo worLd

now...:

>>> cmd = ['/usr/bin/tr']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
>>> out,err = p.communicate('hello world')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/subprocess.py", line 668, in communicate
    return self._communicate(input)
  File "/usr/lib/python2.5/subprocess.py", line 1218, in _communicate
    bytes_written = self._write_no_intr(self.stdin.fileno(), buffer(input, input_offset, 512))
  File "/usr/lib/python2.5/subprocess.py", line 997, in _write_no_intr
    return os.write(fd, s)
OSError: [Errno 32] Broken pipe

because...:

>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
>>> /usr/bin/tr: missing operand
Try `/usr/bin/tr --help' for more information.

and if we fix the bug:

>>> cmd = ['/usr/bin/tr', 'l', 'L']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
>>> out,err = p.communicate('hello world')>>> print out
heLLo worLd
>>> print err
None

...it fixes everything. What happens if you omit the stderr redirection -- do you perchance see any error messages from myapp...?

Related Topic