[Twisted-Python] Thoughts about testing

[Andrew Bennetts]

On Tue, Oct 25, 2005 at 12:59:24PM -0400, Jean-Paul Calderone wrote:
> On Tue, 25 Oct 2005 05:19:43 -0700 (PDT), Antony Kummel 
[...]
> 
> Running tests in a child process is an interesting idea.  It 
> provides a much greater degree of isolation between tests than 
> essentially any other approach, which is great.  Isolation is 
> great for unit tests.  Unfortunately, it comes with a lot of 
> overhead.  While there are techniques for optimizing the 
> implementation of such a feature, running each test method in a
> different process would probably add at least 4 minutes to 
> Twisted's test suite.  This is basically unacceptable (Twisted's 
> suite takes way too long to run already).

Robert Collins' subunit and testresources libraries for pyunit go a long way
to solving this, I believe: http://www.robertcollins.net/unittest/

> >The second thought is this: there seem to be popping
> >up different testing toolkits each with their own very
> >nice extensions and features
> >(http://testoob.sourceforge.net/,
> >http://codespeak.net/py/current/doc/test.html). Trial
> >cannot benefit from this, having branched away at the
> >pyUnit level. I think Trial's special features can be
> >relatively easily formulated as plugins to a
> >plugin-oriented testing framework (especially if the
> >clean reactor requirement is relieved), and so can the
> >other testing packages. What this means, is that the
> >best thing anyone who wants the world of unit testing
> >to develop, and to benefit from it, is to push for a
> >plugin-oriented pyUnit, and for an implementation of
> >Trial (and the other tools) as a plugin for that
> >framework. I think.

Trial is moving very rapidly towards pyunit compatibility once again, thanks
to Jonathan Lange's efforts.  It should be possible to run trial test suites
from an ordinary unittest.py runner in the near future, if it's not already
possible with SVN Trial.  I'm not sure what you mean by "push for a
plugin-oriented pyUnit" -- it already is extensible.

I was under the impression that py.test wasn't at all pyunit compatible,
though?  The docs I've seen tell me that they explicitly aren't interested
in being able to interoperate with unittest or unittest extensions.

> Rather than hearing about the plethora of new testing libraries 
> appearing, I'd like to hear about features they provide that are
> valuable for writing tests.  I would certainly like to borrow 
> py.test's magical assert.  What other features are test authors 
> finding useful in some of these projects?

FWIW, I don't like py.test's magical assert ;)

-Andrew.