[Twisted-Python] git clone with spawnProcess() can prevent socket data flow, any clue why?

[Martin Nordholts]

Hi

I am getting a weird problem where git cloning with 
reactor.spawnProcess() fails for some repositories. It appears as if 
Twisted blocks reading from the server socket that git packs should come on.

I originally got the problem with buildbot, but I have has isolated the 
problem further to Twisted. The following program can be used to 
reproduce the problem:

  #!/usr/bin/python

  from twisted.internet import reactor, protocol

  class ProcessPrinter(protocol.ProcessProtocol):
      def connectionMade(self):
          print "::connectionMade"

      def outReceived(self, data):
          print "::outReceived"
          print data

      def errReceived(self, data):
          print "::errReceived"
          print data

      def processEnded(self, status_object):
          print "::processEnded"
          print "exit code %d" % status_object.value.exitCode
          reactor.stop()

  def spawn_after_run():
      argv = ['/usr/bin/git', 'clone', 'git://git.gnome.org/gimp']
      reactor.spawnProcess(ProcessPrinter(), argv[0], argv)

  reactor.callLater(0, spawn_after_run);
  reactor.run();

If a debugger is attached to the spawned git process, it can be seen 
that the process blocks on read() on the socket that is supposed to be 
fed with git packs from the server.

I have done a tcpdump of the traffic between my computer and the git 
server, and there *is* successful communication going on. More 
specifically, the git client receives the refs the server has, and sends 
the refs it wants, which is acknowledged by the server, but for some 
reason it still eventually blocks on read().

Curiously enough, if the program is changed to clone e.g. 
'git://git.gnome.org/gegl' instead of 'git://git.gnome.org/gimp', it 
works! The GIMP repo is much bigger than the GEGL one, which might be 
the trigger for the bug, but I don't understand why it would matter.

I see this problem on Fedora 13 x86_64, both with the distro version 
Twisted 0.8.2, and with the latest stable release Twisted 10.1.0. My 
personal guess is that it has something to do with how spawnProcess() 
manages the fds or the child process.

Does anyone have any guess of what is going on?
Can you reproduce it on other systems?

Thanks in advance for any input.

Regards,
Martin