[Twisted-Python] documentation / kqueue / feedback

[Dr aio]

Glyph

Front-up, first thing: As far as I'm concerned, we can talk friendly.
Probably we both "punished" each other for things we did at least not
mean (evil). :-)

>    "You guys really need to do X, Y, and Z to make your software usable."

Just to avoid further misunderstandings: Nope. I showed up and said
"Yuck ! TM seems to be friggin great. Why do you make it
useless~unuseable~difficult and confusing to use ?"

That was a decalaration of love, not war. I remember once telling my
son (then 7) sth along the line "Listen, this is a very, very nice
essay and it damn deserves to be written in a way your teacher can
read it. It deserves not be written barely readable and sloppy".

> > One problem was that a lot of docs and tutorials (on the web, too) are
> > simply very old and also using different approaches.
> > Example: I'm just trying to understand how resources work. Suddenly
> > they talk about "application" and tap. That is confusing.
> >
>
>  Definitely a problem.  This is probably worth filing a ticket about, but
> can you provide a little more detail, and some links to the relevant
> documents and the order you discovered them in?  The more detail the easier
> the documentation fixer's job will be.

I remember that in the section
http://twistedmatrix.com/projects/core/documentation/howto/ there are
sime links to the api (I suppose) that are broken/changed, e.g. in
http://twistedmatrix.com/projects/core/documentation/howto/basics.html
there is a link to "twisted.application.service.Application"
(http://twistedmatrix.com/projects/core/documentation/howto/twisted.application.service.Application)
that's broken.

This immensily disturb when one just discovers a complex thing like TM.


>  We don't know exactly how newbies
> approach the documentation because we can never approach it without knowing
> it.

OK, I'll try.

First I read about TM (here and there) being rudely complex. But then,
it also seemed to be the one that offered what I was looking for.
The bad reputation seemed to be mostly based in deferreds, so I looked
at those first.
Good: There is good documentation and I think anyone with some
experience in IT can understand it.
Bad: That documentation is spread all over the place, sometimes even
just a sentence or two somewhere.
Tip: Put relevant stuff in "getting started" and also offer links to
papers like the one about reactors.

After feeling assured enough about deferreds and actually liking them
and the was TM implements them I wanted to start up by writing a small
webserver.
What disturbed me the most was that whole resource thing and, more
generally, all that "stuff" like application, resource, Iresource tap,
etc.
It felt like having gotten the basic concept but having a thick fat
layer of "sugar", comfort in between the core and myself. Things
didn't have a place, they didn't fit - there was no overview but it
rather felt like trial and error.
There are plenty of snippets and examples but I didn't get the overall picture.

Tip: Publish some graphics, possibly a class tree and have some
introductory essay in "getting started". (e.g. Why would I want and
use "application" ? What does it do an how does it do that ?)

I noticed sth. similar with the API doc.
Sure, you guys know it by heart but I as a newbie can be completely
lost. Be generous and have a sentence or two with each function.

> ...  I do not
> mean to suggest that you should write documentation _immediately_. Merely
> that once you have learned enough about Twisted, you will be a good person
> to do these improvements.

OK, sounds reasonable. Will do once I come along with TM myself.

> "kqueue
>  This is of course my point.  To you, perhaps.  Not to Twisted :).  But,
> just because it's not "vital" doesn't mean it's not important.  We do need
> kqueue support.  It is a selling point of Twisted to support multiple
> different multiplexing mechanisms and kqueue is an important mechanism.

Maybe it's useful to you to consider my perspective. Actually, without
kqueue I would probably not have looked at TM. Python offers asyncore
and httpd so, somewhat more basic, of course, I would have had what I
needed if I were looking for _any_ select/poll based solution.
funny, btw. that for me personally one of the big plus-point of TM,
the whole array of protocolls, is by far less attractive. My approach
was rather "Ah, cool, they offer what plain good Python doesn't have;
a solid high-performance AIO core with usefull stuff on top".
It's probaly not the audience you targetted but I'd bet there are more
like myself looking at TM.

>  *PLEASE* do not switch to linux to use Twisted.  We have, like, a million
> developers who use Ubuntu.  Hence, we have very good support for linux.  We
> need more Twisted developers who use FreeBSD (and Windows and Solaris and
> AIX and even distros like Red Hat and SuSE).  Please, please stay around and
> help us with that.

No worry. I wouldn't switch to Linux even if it came with naked women along * g
Seriously, I'd rather do the whole stuff in C than switch away from BSD.

>  The problem is that FreeBSD people often show up, complain that Twisted has
> problems on FreeBSD, and then go away.  Either that, or they decide to give
> up FreeBSD and just use Linux on their servers, which is great for them, but
> doesn't help us at all.

I'll do neither. As long as I can use TM and get some help when needed
I 'll be around. On FreeBSD :)
I don't worry about the occasional hickup. I only worry about getting
into TM, about getting a good start and sth. done with it.

Again: It's just a very small thing but what I can offer right now is
a patchfile incl. the patches by ITS for the FreeBSD port. I tested it
both on 6.3 and 7.0 and it builds fine.

> (FreeBSD) ... so you
> have only to worry about kqueue issues (which therve and others have already
> done a ton of work on).

Would someone listen, please ?

Actually py-kqueue 1.4 IS WORKING on my FreeBSD.
I stressed it a little bit on my devel box, a P4, 1.8GHz, 768MB box
and it delivered nicely - around 300 connections/s with a payload of
around 2.5 KB delivered by a simple rpy. (with system load of ca.
0.75)

The problem is only, when it's daemonized. then it breaks (see my
earlier post for detailled output)


>  Well, let's move forward trying to make that happen.

Yes, Glyph ;)