Hilarious dream-logic (was Re: [Twisted-Python] [patch] (etc)

[Justin Warren]

On Fri, 2006-05-19 at 01:42 +0200, Andrea Arcangeli wrote:

[snip. Not overly interested in the flamewar.]

> 2) verifying a bugfix correctness doesn't require unittests in most
> cases, anybody with a basic CS knowledge perfectly knows that
> no matter how much testing you do you can never be guaranteed that the
> code is correct by just testing it. Infact if the test is buggy too, it
> may generate false positives. The same way you _have_ verify any unit-test
> without using yet another unittest (or you enter an infinite recursion),
> you can also verify that the bugfix is correct in the first place. The
> idea that without an unittest isn't possible to verify the correctness
> of a bugfix is totally wrong. An unit-test is sure useful, but it
> shouldn't be mandatory for quick bugfixes. And the time that it would
> take to write a new unittest for a new bug, before deploying the fix in
> production, is definitely "wasted".

I don't believe you understand the purpose of unit testing. The approach
you appear to be advocating leads to unmaintainable, buggy code. The
unit test is developed to verify that the bug you believe exists does,
in fact, exist. Buggy code will cause the test to fail, while correct
code will cause the test to pass. If the behaviour you observe is
actually caused by something else, your test will pass, but the
behaviour will stay the same.

In this way, you can correctly identify the cause of the problem. Then
you can develop a patch. Importantly, once you have developed a patch,
you can verify that it does, in fact, fix the problem. If the code
passes your new unit test after you apply your patch, the patch is
correct. If the test still fails, your patch doesn't, in fact, fix the
problem. 

Additionally, when other changes are made in the future, the new unit
test can be used to verify that those changes (like your patch) don't
break existing code, ie: regression testing. I'm personally a big fan of
this feature of test suites.

Incidentally, those with slightly more CS knowledge know that it is
possible, though by no means easy, to build a system that is provably
correct. An investigation of the Z specification language may prove
enlightening. I think you mean that a test suite proves that the system
passes all the tests, not that it is bug free.

> 5) about the technical things that I said about nevow, axiom and epsilon
> previously, I'm sorry if the divmod guys didn't like it, but if they're
> not true, then they should not get angry and insult me, but they should
> post benchmarks that invalidate my claims. They are very welcome to
> start to invalidate my bold claim that for rendering web2+cheetah is an
> order of magnitude faster (and in turn more scalable) than web+nevow.

I think you have this backwards: those who make bold claims are the ones
with the burden of proof. Simply claiming that something is so does not
make it so. If you have benchmarks that demonstrate your claim, and
importantly, the method used to generate them, then by all means share
them with others. If your claims are true, this information will help in
fixing the problem. Fixing these bugs is what we all want, after all.

-- 
Justin Warren <daedalus at eigenmagic.com>