Hilarious dream-logic (was Re: [Twisted-Python] [patch] (etc)

[Andrea Arcangeli]

On Thu, May 18, 2006 at 09:25:07PM +0200, Andrea Arcangeli wrote:
> 	http://twistedmatrix.com/pipermail/twisted-python/2006-May/013137.html
> 	http://www.mail-archive.com/twisted-web%40twistedmatrix.com/msg00352.html
> 
> (sadly there are many more)

Within a few hours here one more that can be added to the list:

	http://twistedmatrix.com/pipermail/twisted-python/2006-May/013158.html

The guy is kind enough not to blame me for not speaking english as my
first native language, but yet he's trying to teach me english by saying
that unittests are a "waste of time" 

---quote---
This from a man who believes that unit tests are a "waste of time".
---quote---

means the same as "I won't have to waste time to fix bugs".

In my minimal english language understananding, it was clear that with
"wasted time to fix bugs" I meant "the time between fixing the bug and
deploying the bugfix in production (or to the userbase)". I never said,
believed, thought or meant that writing an unit test is a waste of time
at large, as proven by the CPUShare-Twisted homepage that explicitly
says for the last 3 months "unittest are welcome". Why should they be
welcome if they are "a waste of time"?

But then the fact the guy puts words in my mouth that I never said in
order to make me look like an idiot is after all a minor thing compared
to the rest he says.

But let's ingore the usual politcally correct behaviour I receive from
some element of this community and let's stick to technical facts:

1) callFromThread should be renamed callFromReactor (no need to make up
long names when it actually means callFromReactor)
2) verifying a bugfix correctness doesn't require unittests in most
cases, anybody with a basic CS knowledge perfectly knows that
no matter how much testing you do you can never be guaranteed that the
code is correct by just testing it. Infact if the test is buggy too, it
may generate false positives. The same way you _have_ verify any unit-test
without using yet another unittest (or you enter an infinite recursion),
you can also verify that the bugfix is correct in the first place. The
idea that without an unittest isn't possible to verify the correctness
of a bugfix is totally wrong. An unit-test is sure useful, but it
shouldn't be mandatory for quick bugfixes. And the time that it would
take to write a new unittest for a new bug, before deploying the fix in
production, is definitely "wasted".
3) when I posted the ticket (again I received no answer with that one
because of the trac hidden feature) about POLLHUP/ERR/NVAL core reactor
troubles, I thought it wasn't strictly needed. But that was a few weeks ago
if I remember well. In the last few weeks things changed and now it
looks strictly needed and worst of all to be fully reliable I would need
it in the client too, and on the client I still allow my users to use
the standard twisted from twistedmatrix.com. I will probably have to
workaround it with a timer but it's still not a desiderable solution
since I will have to rearm the poll syscall for a little way in
unblocking mode which is theoretically unsafe.
4) his belief that the core reactor limitation of point 3 "can be
affected even without changing any external APIs" (btw, I guess with his
native english he actually means "fixed" and not "affected") implies the
external API won't change but I can guarantee you that the semantics
will change in a not-backwards-compatible way. I think the users should
know if you're going to make not-backwards compatible semantical changes
in the core reactor without changing the external API (i.e.
transport.pauseProducing and friends) so they can check their apps won't
break.
5) about the technical things that I said about nevow, axiom and epsilon
previously, I'm sorry if the divmod guys didn't like it, but if they're
not true, then they should not get angry and insult me, but they should
post benchmarks that invalidate my claims. They are very welcome to
start to invalidate my bold claim that for rendering web2+cheetah is an
order of magnitude faster (and in turn more scalable) than web+nevow.
6) about the performance of web2 vs apache/cherrypy (note: this is
different from point 5, this is _not_ web2+cheetah vs web+nevow), there
is no reason to worry about the channel of web2. No reason to change the
API. I already told foom and drier privately. The benchmark I've seen
posted to the list had a tiny size per page. Unfortunately the majority
of the cpu time is spent in the twisted core (i.e. basically the
reactor). I measured it. There's little or nothing you can do inside
web2 to speedup the per-connection overhead. Infact if you increase the
size of the page, the connections per second remain the same, only the
bandwidth increases. So effectively web2 has no immediate performance
issues because most pages are larger than a dozen bytes. When you do the
test on a normal page size, things are ok. Most of the overhead happens
in the handling of the socket outside web2, and it's very similar for
twisted.web and it's a fixed cost outside web2. Again I will be totally
happy to be proven wrong and see that the divmod folks can make web2 go
much faster with a tiny page size with a change of API inside
twisted/web2 directory and without touching the reactor code.