[Twisted-Python] retrial, the new-and-improved unittesting framework for twisted
Jonathan Simms
slyphon at twistedmatrix.com
Sun Sep 19 14:08:14 MDT 2004
Hello twisted people,
Many of you know that I've been working on rewriting the
twisted-framework's unittesting program "trial".
This message is for those of you who don't ;)
I've nearly completely redesigned trial in terms of both it's
user-facing features, and it's internal workings. In initial testing,
it's shown a 4x speed increase over regular trial, running twisted's
tests in roughly 220s on a p3-1Ghz machine (as compared to 800+ seconds
on the same box).
here are some of the features you can look forward to (from the
twisted.trial.unittest docstring, so please excuse the epydoc markup):
1. B{Trial now understands deferreds!}
- There is no reason to use L{twisted.trial.util.wait} or
L{twisted.trial.util.deferredResult}. Write your deferred handling
code as you normally would, make your assertions in your callbacks
and errbacks, then I{return the deferred from your test method}.
Trial will spin the reactor (correctly) and will wait for the
results before running the next test. This will allow developers to
write more natural-looking tests for their asynchronous code.
- there is a new attribute that has been introduced, C{.timeout}.
Trial will wait a default 4 seconds for a result from a deferred
that is returned from a test method. If you wish to make this value
smaller or larger:
>>> from twisted.trial.unittest import TestCase
>>> from twisted.internet import defer
>>> class MyTestCase(TestCase):
... def testThatReturnsADeferred(self):
... return defer.success('Hooray!')
... testThatReturnsADeferred.timeout = 2.8
...
>>>
This would cause trial to wait up to 2.8 seconds (quite needlessly in
this case) for the deferred to either callback or errback
2. B{Trial is now 100% compatible with new-style classes and zope
interfaces}
- Some people (the maintainer included), have been bitten in the past
by trial's mediocre support for new-style classes (classes which
inherit from object). In v2.0, nearly all of the classes that
comprise the framework inherit from object, so support is built-in.
Whereas before the TestCase finding machinery used a test for
inheritance from L{twisted.trial.unittest.TestCase}, the new
mechanism tests that L{twisted.trial.interfaces.ITestCaseFactory}
is supplied by your class B{type}. You can write a custom TestCase,
and trial will detect it and use it as a class to test, if you do:
>>> import zope.interface as zi
>>> from twisted.trial.interfaces import ITestCaseFactory
>>> from twisted.trial.interfaces import ITestCase
>>> class MyTestCase(object):
... zi.classProvides(ITestCaseFactory)
... zi.implements(ITestCase)
>>>
Naturally, the class should actually provide an implementation of
L{twisted.trial.interfaces.ITestCase}.
- To avoid any possible conflicts (and to provide component
de-registration), trial uses it's own private adapter registry, see
L{twisted.trial.__init__} for details.
- Trial makes use of zope.interface.Interfaces to allow flexibility
and adaptation. All objects implement interfaces, and those
interfaces are centralized and documented in
L{twisted.trial.interfaces}.
3. B{All assert* and fail* methods are now top-level functions of the
unittest module}
- Previously they were only available as instance-methods on the
TestCase. You can now import all of the assert* and fail* variants
and use them as functions. This will allow you to use this
functionality in helper classes and functions that aren't part of
your TestCase (plus less typing ;])
- Note: these methods are no longer part of the ITestCase API, but
are provided as a backwards-compatability to classes written to use
the original TestCase class.
4. B{The trial script now accepts a --reporter option}
- This is to allow for custom reporter classes. If you want to run a
trial process remotely, and gain access to the output, or if you
would just like to have your reporting formatted differently, you
can supply the fully-qualified class name (of a class that
implements L{twisted.trial.interfaces.IReporter}) to --reporter,
and trial will report results to your class.
- The Reporter is now (almost) totally stateless. All stats on the
test run are held in the TestSuite and are reported as necessary
using the ITestStats interface. This allows for greatly simplified
design in the Reporter implementation.
- The Reporter API has been greatly simplified by changing the method
signatures so that methods are called with a single object that can
easily be adapted to provide all information necessary about a
given test phase.
5. B{Compatibility for PyUnit tests}
- Trial now supports stdlib unittest.TestCase classes transparently.
This functionality is unstable, and has not been heavily tested.
- Note: Trial accomplishes this by monkey-patching unittest.TestCase
in L{twisted.trial.__init__}.
- Please report any bugs you find with this feature to the
twisted-python mailing list
6. B{Experimental support for doctests}
- The trial script now supports a --doctest=[module] option. The
argument is a fully-qualified module name, and trial will use a
modified version of DocTestSuite to run the doctests it finds.
- My support for doctests is broken when using Python 2.4-alpha3,
hopefully, i'll get this fixed by the time the first major-release
comes out.
- Note: you cannot use .skip or .todo attributes with doctests, all
tests will be reported as pass/fail
- Please report any bugs you find with this feature to the
twisted-python mailing list
7. B{expectedAssertions} is no longer supported
- it was just too difficult to make radix's clever
deferred-doublecheck feature work with this code revision. With his
permisison, this feature has been removed.
Trial's 'special' attributes:
1. .todo attributes can either be set on the TestCase or on an
individual test* method, and indicate that the test is expected to
fail. New tests (for which the underlying functionality has not yet
been added) should set this flag while the code is being written. Once
the feature is added and the test starts to pass, the flag should be
removed.
2. Tests of highly-unstable in-development code should consider using
.skip to turn off the tests until the code has reached a point where
the success rate is expected to be monotonically increasing.
3. Tests that return deferreds may alter the default timeout period of
4.0 seconds by adding a method attribute C{.timeout} which is the
number of seconds as a float that trial should wait for a result.
I am currently developing this in a branch in the twisted repository. I
would like to encourage anyone who is curious to check it out and play
with it and its new features. I'd like to make it as bug free as
possible before I merge it back into trunk, and the more eyes I can get
on it, the better.
you can check it out by doing
"svn co svn://svn.twistedmatrix.com/svn/Twisted/branches/slyphon/retrial-2"
and thank you for your support....
-Jonathan "slyphon" Simms
More information about the Twisted-Python
mailing list