[Twisted-Python] threads

[Andrew MacIntyre]

I'm trying to get Twisted 1.2.0 up on the OS/2 EMX port of Python I
maintain.

Most of it appears to be no great drama, but when I run the test suite,
and it hits a threads related test things get awfully out of shape - the
process goes into a runaway state that is unkillable, using all available
CPU, although the system remains generally usable.  A reboot is required
to restore order.

When I started the testing, I had an OS/2 port of SQLite+PySQLite
installed, and the SQLite test triggered the above behaviour.  On the
assumption that my SQLite port could be implicated, I deinstalled SQLite
etc and tried again.

This uncovered a typo in twisted/flow/threads.py:
*** threads.py.orig     Sun Aug 10 13:00:48 2003
--- threads.py  Mon Mar 15 17:12:16 2004
***************
*** 122,128 ****
              except StopIteration:
                  reactor.callFromThread(self._stopping)
              except:
!                 self.failure = Faliure()
                  reactor.callFromThread(self._cooperate)
              self._cooperate.immediate = True

--- 122,128 ----
              except StopIteration:
                  reactor.callFromThread(self._stopping)
              except:
!                 self.failure = Failure()
                  reactor.callFromThread(self._cooperate)
              self._cooperate.immediate = True


With that fixed, I'm now back to the original behaviour.

The test log contains the following:

...
2004/03/15 17:13 est [-] --> twisted.test.test_flow.FlowTest.testProtocolLocalhost <--
2004/03/15 17:13 est [-] twisted.internet.protocol.ServerFactory starting on 0
2004/03/15 17:13 est [-] Starting factory <twisted.internet.protocol.ServerFactory instance at 0x2de70c>
2004/03/15 17:13 est [-] Starting factory <twisted.internet.protocol.ClientFactory instance at 0xb4780c>
2004/03/15 17:13 est [_Protocol,client] Stopping factory <twisted.internet.protocol.ClientFactory instance at 0xb4780c>
2004/03/15 17:13 est [-] --> twisted.test.test_flow.FlowTest.testQueryIterator <--
2004/03/15 17:13 est [-] --> twisted.test.test_flow.FlowTest.testThreaded <--
2004/03/15 17:13 est [-] Traceback (most recent call last):
	  File "Lib/threading.py", line 436, in __bootstrap
	    self.run()
	  File "Lib/threading.py", line 416, in run
	    self.__target(*self.__args, **self.__kwargs)
	--- <exception caught here> ---
	  File "F:/dev/Twisted-1.2.0/twisted/python/threadpool.py", line 157, in _worker
	    context.call(ctx, function, *args, **kwargs)
	  File "F:/dev/Twisted-1.2.0/twisted/python/context.py", line 53, in callWithContext
	    return self.currentContext().callWithContext(ctx, func, *args, **kw)
	  File "F:/dev/Twisted-1.2.0/twisted/python/context.py", line 32, in callWithContext
	    return func(*args,**kw)
	  File "F:/dev/Twisted-1.2.0/twisted/flow/threads.py", line 126, in _process
	    reactor.callFromThread(self._cooperate)
	  File "F:/dev/Twisted-1.2.0/twisted/internet/base.py", line 200, in callFromThread
	    self.threadCallQueue.append((f, args, kw))
	exceptions.AttributeError: 'NoneType' object has no attribute 'append'

2004/03/15 17:13 est [twisted.internet.protocol.Factory] (Port 0 Closed)
2004/03/15 17:13 est [twisted.internet.protocol.Factory] Stopping factory <twisted.internet.protocol.Factory instance at 0xaad8cc>
2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] (Port 0 Closed)
2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] Stopping factory <twisted.internet.protocol.ServerFactory instance at 0x2de70c>
2004/03/15 17:13 est [Foo,client] Stopping factory <twisted.internet.protocol.ClientFactory instance at 0xabc0ac>
2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] (Port 0 Closed)
2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] Stopping factory <twisted.internet.protocol.ServerFactory instance at 0xabc26c>
2004/03/15 17:13 est [Foo,client] Stopping factory <twisted.internet.protocol.ClientFactory instance at 0xabcb0c>
2004/03/15 17:13 est [Foo,client] Stopping factory <twisted.internet.protocol.ClientFactory instance at 0xabcd0c>
2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] (Port 0 Closed)
2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] Stopping factory <twisted.internet.protocol.ServerFactory instance at 0xabc14c>
<<<log ends>>>

The traceback is identical to that in the log after the SQLite test
failure.

While the threads support in the EMX port of Python passes the Python test
suite, and people are successfully running Zope on OS/2 with this Python
port, I don't know of any other widespread usage which might have
exercised it.  So I won't be surprised if it has problems.

Can anyone give me any hints about what might be missing or working
incorrectly to trigger the above?

I should note that I've never tried to use Twisted before.  The above is
with Python 2.3.3.  I don't have permanent internet connectivity, and the
test was being run without an internet connection.

Regards,
Andrew.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac at pcug.org.au             (alt) |        Belconnen  ACT  2616
Web:    http://www.andymac.org/               |        Australia