[Twisted-Python] Need replacement for subprocess.call within a pyAMF method - any suggestions?

[exarkun at twistedmatrix.com]

On 3 Nov, 11:55 pm, rroberts at adobe.com wrote:
>I am trying to use the twisted reactor as a basis for a Python server 
>for a
>Flex app, using pyAMF.  My problem is that one  of the server methods
>requires me to run a series of command-line tools to complete parsing 
>data
>from a file into an XML object, with lots of Python processing on the
>results of one command-line tool to provide the input to the next one. 
>I
>have been using  subprocess.call  to invoke the command-line tools, but
>these intermittently - and frequently -  fail with an error "IOError: 
>[Errno
>4] Interrupted system call".
>
>I do know that this was supposed to be fixed in twisted for Python 2.6, 
>but
>it doesn't seem to be.

I'm not sure why you know this.  I sure would be great if that were the 
case, but it's not.  More changes in Twisted are necessary to let things 
like subprocess.call work properly alongside Twisted's process API.
>I have python 2.6.3 under Mac OSX 10.5.8, and twisted
>8.2.0. Is there a work-around for this problem, or a better way to do 
>this
>with twisted? I do know that I can make a command-line call with
>reactor.callLater, and that I can return a deferred from my server 
>method,
>but I don't see how to easily accomplish the chain of calls that I 
>need,
>short of breaking up my serverMethod1 into the several sequential 
>parts, and
>calling each from the result handler of the previous one. I'd rather 
>not go
>that route, as the logic of which command-line call gets made when is
>actually quite complicated.

Are you aware that this means your server will only be able to handle 
one request at a time?  If so and that's what you're shooting for, then 
you can fix this easily, by passing False to reactor.run.  This 
disable's Twisted's own process support and lets things like 
subprocess.call work (blockingly, of course).
>What I have now looks like (enormously simplified):
>class MyPythonServer:
>
>    def serverMethod1(self, dummyPost, inputFilePath):
>        # Open inputFilePath, do stuff, save results to file2
>        subprocess.call(["command1", "file2", "file3"])
>        # Open file3, do stuff to the data, save results to file4
>        subprocess.call(["command2", "file4", "file5"])
>        # Open file5, convert data to a large string containing XML.
>        return xmlString

Of course, you can still have methods that look like this if you use 
things like inlineCallbacks (which you can, since you said you're using 
Python 2.6).  That way you preserve the style you seem to prefer and 
avoid blocking your server any time you need to launch a child process.

Jean-Paul