[Twisted-Python] Thoughts about testing

[glyph at divmod.com]

On Wed, 26 Oct 2005 19:48:32 +0200, Moof <moof at metamoof.net> wrote:
>Glyph Lefkowitz wrote:

>I do despise the unittest (and hence trial) camelcaseness. I don't know
>enough low level python to be able to answer whether a test() function would
>have the same drawback as assert, if -O testing is really an issue.

Nope.  A test() function could do all the same stack introspection and everything as assert, but would not create problems with -O.  So it's a stealable idea.

>>>3. Tests execute in predictable order.

>There is a use case for this, though. py.test has a -x option that will stop
>executing after one test returns an error. This is very much a time saver
>when you're running a bunch of lengthy tests on a function where one initial
>test failing may inevitably mean that a bunch of more sophisticated tests
>involving the same item will fail.

The correct way to deal with this is to print errors immediately when they happen, like trial -e, or to run just the test that you want, like twisted-dev.el does.  If your unit test spits out the error right away, you can fix it while the rest of the tests are running, then kill and restart them immediately, for maximum user/computer parallelism.

>One thing that I did like out of py.test was the ability to name a test or
>test object that you wanted to run and it would do magic pattern matching on
>the test's __name__ based on what you typed in. It still printed out
>non-matching tests as skipped, but it only ran what you wanted. It allows
>you to, again, focus on small functionality within a larger test file.

Hmm, wildcards might be handy, but in practice I've never wanted more than trial's hierarchical naming.  One thing I wish trial had was a reporter mode that looked like 'make' output, eg:

% trial --gcc -e foo
trial --gcc -e foo.test
trial --gcc -e foo.test.test_whatever
trial --gcc -e foo.test.test_whatever.TestWhatever.test1
    OK
trial --gcc -e foo.test.test_whatever.TestWhatever.test2
    OK
trial --gcc -e foo.test.test_whatever.TestWhatever.test3
    FAILED: FooError
foo/bar.py:1: ...

so I could easily jump to a failed test, then copy the exact trial line needed to run it, so that I could paste that into my buffer's test line, or run trial manually on the command line to 'zoom in' to the test or suite I want.

I bet mumak has cleaned up the reporter API so this is now sane to do, I should have a look.

>Serving up my ego over at <http://metamoof.net/>

BTW, this site seems to be down.