[Twisted-Python] spawnProcess help needed

[Phil Christensen]

Hello All-

I'm running into an issue with a project I'm working on that involves  
coordinating a large number of external processes to encode audio into  
different formats, and do other bookkeeping.

I seem to be having a problem when I use spawnProcess to call the  
md5sum binary. The processes spawned seem to stick around forever,  
until I kill the server. This doesn't appear to happen with any other  
binary.

I start with the following abstract class:

     class DefaultProcessProtocol(protocol.ProcessProtocol):
         def __init__(self, preserveOut=True, preserveErr=True):
             self.deferred = defer.Deferred()
             self.debug = False
             self.preserveOut = preserveOut
             self.out = ''
             self.preserveErr = preserveErr
             self.err = ''

         def connectionMade(self):
             if(self.debug):
                 print '%s connection made' % self.__class__.__name__

         def outReceived(self, data):
             if(self.debug):
                 print '%s stdout: %s' % (self.__class__.__name__, data)
             if(self.preserveOut):
                 self.out += data

         def errReceived(self, data):
             if(self.debug):
                 print '%s stderr: %s' % (self.__class__.__name__, data)
             if(self.preserveErr):
                 self.err += data

         def processEnded(self, status):
             if(isinstance(status.value, error.ProcessDone)):
                 self.deferred.callback(self)
             else:
                 if(self.out):
                     print self.out
                 if(self.err):
                     print self.err
                 self.deferred.errback(status.value)

Then I made a subclass as follows:

     class MD5Checksum(DefaultProcessProtocol):
         @staticmethod
         def run(target, md5_path='/usr/bin/md5sum'):
             hasher = MD5Checksum(preserveOut=True)
             hasher.debug = True
             reactor.spawnProcess(hasher, md5_path, [md5_path, target])
             return hasher.deferred

         def get_checksum(self):
             filehash = self.out
             if(filehash.find('=') == -1):
                 filehash = [output.strip() for output in  
filehash.split(' ')][0]
             else:
                 filehash = [output.strip() for output in  
filehash.split('=')][1]
             return filehash

With the plan of executing it inside an inlineCallbacks-decorated  
function, like this:

         proto = yield process.MD5Checksum.run(filename)
         checksum = proto.get_checksum()

Everything is basically working, except for the fact that the md5sum  
processes never go away. It does seem that processEnded is being  
called, since that's what issues the callback on the deferred returned  
by run(), but the process sticks around, as a non-zombie, non-defunct  
process that still appears to be using small amounts of CPU time.

Also, once the checksum has been retrieved, killing the server doesn't  
cause any errors, unlike when a process is legitimately terminated by  
killing the server, which displays an error (since right now i'm not  
actually catching ProcessTerminated scenarios):

     2008-12-09 15:17:48-0500 [-] Unhandled error in Deferred:
     2008-12-09 15:17:48-0500 [-] Unhandled Error
         Traceback (most recent call last):
         Failure: twisted.internet.error.ProcessTerminated: A process  
has ended with a probable
     error condition: process ended by signal 2.

Any help in this matter would be appreciated...

-phil