[Twisted-Python] Question on using processes

[Tim Allen]

On 6 Aug 2004, at 06:53, Matt Feifarek wrote:
> I'm using twisted.internet.utils.getProcessOutput() to run commands 
> from a reactor.
>
> I've pretty much got this working okay (with a few false starts 
> related to how args are put together... you can't send in a big long 
> string; you literally have to make a list of argument words... what a 
> PITA).

This is due to Python's (and C's) Unix heritage. The kernel call to 
spawn a new process takes the full path to the executable, and an array 
of null-terminated strings which are the arguments. This can be quite 
tedious to set up in C, but shouldn't be too painful under Python. 
Contrast this behaviour with MS-DOS and Win32, whose 'new process' API 
takes a single string of command-line parameters, and passes that 
single string directly to the new process.

When you type a command with some parameters at the command line, how 
does the program wind up with a sequence of command-line parameters?

Under Unix, the shell (usually /bin/sh or compatible) reads through the 
command-line and does all its special parsing - adding environment 
variables, constructing pipes, breaking up the string into 
space-delimited chunks, doing backslash-escaping and so forth, it 
searches the PATH environment variable for the location of the named 
program, and winds up with a full executable path and an array of 
string parameters which it hands off to the kernel, and your process is 
started. Because most shells have a well-designed meta-character set, 
you can spawn pretty much any program you like with pretty much any 
command-line you like (except for embedded nulls, of course) grouped in 
any way you like.

Under DOS/Win32, the shell (usually CMD.EXE) takes the command-line you 
give, searches $PATH for the executable, and hands the executable's 
path and the command-line string off to the kernel. The program that's 
started is responsible for parsing the given string into a list of 
parameters, and of course they all do it slightly differently. Some 
don't let you use filenames with spaces, some let you use filenames 
with spaces but don't let you use filenames with quotes, and so forth.

For example, under Unix, I can use 'python -c' to make instant Python 
scripts:

	python -c 'print "Hello, \"World\""'

Under Win32, I don't believe I can do the above step because CMD.EXE 
always treats double-quotes as delimiting a filename with spaces. I 
can't make a single commandline parameter that includes embedded 
quotes.

> Now, I've got a command that needs to pipe something through another 
> command, something like this:
>
> /bin/cat foo.txt | /usr/bin/tail

The reason for this is that piping is set up and executed by the shell, 
not the kernel. What you really need (to solve both these problems) is 
a way to get the shell to do all the dirty work for you - and luckily, 
such a feature exists. For the above example, instead of executing 
/bin/cat and handing it the other parameters try executing /bin/sh and 
giving it the parameters ['-c', '/bin/cat foo.txt | /usr/bin/tail']