[Twisted-Python] documentation / kqueue / feedback

[glyph at divmod.com]

On 05:06 pm, dr.pythoniac at googlemail.com wrote:
>Sorry. I simply assumed that you know your site and that those broken
>links seem "natural" to you as everything is in flux.
> From now on I will report them.

Thanks.
>>  You can do putChild to any resource, as long as you understand its 
>>lifecycle.  A "resource tree" is simply a tree ( 
>>http://en.wikipedia.org/wiki/Tree_structure ) of resources ( 
>>http://twistedmatrix.com/documents/current/api/twisted.web.resource.IResource.html 
>>).
>>
>>  Our documentation assumes, and will continue to assume, a certain 
>>level of familiarity with basic computer-sciency jargon;
>
>This sounds as if you were to say "You don't fully understand TM ?
>Dumbhead ! Learn the basics before opening your mouth."
>
>Thank you, sir.

If I thought you were a "dumbhead", and didn't know this stuff, I would 
not have provided API links.  If I thought you were ignorant I would 
have provided an explanation, and if I thought you were stupid I would 
not have bothered to reply.  I assume that you understand the concepts 
involved and I was just trying to be as specific as possible, since 
apparently the language that you found in the documentation was 
ambiguous.  (Normally, to communicate this concept, I would just say 
"resource tree" in conversation as well.)

Please note that I never insulted you.  Let me be more specific: I don't 
think you're stupid.  Believe me, if I were calling you an idiot, you'd 
know.  If you look around for rants that I've written, you can find some 
unkind things I have said about other people :).  (Links withheld to 
protect the guilty.)
>Short: I didn't mean to insult or belittle TM but rather to express
>that imho it could be more useable.

I understand.  Perhaps I should have been more specific with my 
comments.  I didn't mean to say that the problems that you were 
encountering were not real, or they should not be fixed.  Merely to 
remind our developers that their work is not "unusable" because of the 
problems you've mentioned, and to ask you to phrase your requests 
differently.

You showed up and said:

    "You guys really need to do X, Y, and Z to make your software 
usable."

The implication being that Twisted is useless to everyone, and therefore 
mostly worthless.  Also, it suggests that you expect us to do additional 
work, which may be relevant only to you, for free.  You can say almost 
exactly the same thing, but phrasing it slightly differently makes all 
the difference.  A better approach:

    "I tried to use Twisted but I can't figure out X, Y, and Z.  Can 
someone help me with that?"

It would also help if you would sprinkle in something about how you 
can't help out with anything until you've learned X, Y, and Z.  If you 
are asking for help so that your team can build a product, then you are 
asking for professional training; but if you're asking for help so that 
you can contribute an improved newbie experience to twisted (perhaps 
after your team builds a product...) then you are asking for the 
community to work together.

You've done that in this message, and I appreciate it.
>One problem was that a lot of docs and tutorials (on the web, too) are
>simply very old and also using different approaches.
>Example: I'm just trying to understand how resources work. Suddenly
>they talk about "application" and tap. That is confusing.

Definitely a problem.  This is probably worth filing a ticket about, but 
can you provide a little more detail, and some links to the relevant 
documents and the order you discovered them in?  The more detail the 
easier the documentation fixer's job will be.  We don't know exactly how 
newbies approach the documentation because we can never approach it 
without knowing it.
>>If you were truly sorry, you would be contributing patches which fix 
>>the broken links and updates the documentation to be more recent.
>
>Uhum, sir ... Me ? contributing doc. ? Frankly, this does not seem to
>be your smartest idea so far * g
>I'll gladly do as soon as I can walk more or less alone in TM, OK.

This is exactly the difference in tone I was talking about :-).  I do 
not mean to suggest that you should write documentation _immediately_. 
Merely that once you have learned enough about Twisted, you will be a 
good person to do these improvements.  For example, I've never had a bad 
Twisted documentation experience; it would be very hard for me to.
>>  Kqueue seems to be vital to an event driven approach like TM, yet 
>>there are ...
>
>>Kqueue is most definitely _not_ "vital" to something like Twisted.  It 
>> > is an optimization of a very specific part of a Twisted application 
>>on > a very specific platform.  *Most* realistic Twisted applications 
>>will
>>be bottlenecked on application code long before something like
>>kqueue (or epoll) will help.
>
>Yes, to me it is vital. Modern AIO implementations haven't been
>researched and worked on for thegeeky  fun of it. select() is just not
>the answer to many of todays needs.

This is of course my point.  To you, perhaps.  Not to Twisted :).  But, 
just because it's not "vital" doesn't mean it's not important.  We do 
need kqueue support.  It is a selling point of Twisted to support 
multiple different multiplexing mechanisms and kqueue is an important 
mechanism.
>>Working from the amount of support
>>from the community, most Twisted applications also don't run on
>>FreeBSD.
>
>Uh ? Means: "Use linux !" ? Irritating to read that.

Let me expand on this.

*PLEASE* do not switch to linux to use Twisted.  We have, like, a 
million developers who use Ubuntu.  Hence, we have very good support for 
linux.  We need more Twisted developers who use FreeBSD (and Windows and 
Solaris and AIX and even distros like Red Hat and SuSE).  Please, please 
stay around and help us with that.

The problem is that FreeBSD people often show up, complain that Twisted 
has problems on FreeBSD, and then go away.  Either that, or they decide 
to give up FreeBSD and just use Linux on their servers, which is great 
for them, but doesn't help us at all.

The amount of work required to fix these problems is relatively small, 
especially compared with what these people sometimes do (write their own 
event loop).  It's even smaller now that some of our mostly-Linux 
developers have now made at least select-on-FreeBSD a fully supported 
platform, so you have only to worry about kqueue issues (which therve 
and others have already done a ton of work on).
>I apologize if this response seems too harsh, but I am annoyed by
>people who deciding for me what's important, what's the OS worth to be
>supported and doubting my professional and intellectual capacity
>because I dare to offer constructive criticism.

I am sorry that you read this as a criticism of your intellectual and 
professional abilities.  I take issue with your tone but, this cannot be 
emphasized enough, I *do not* think you are stupid or even necessarily 
wrong.  Just a little rude :).
>Glyph, I subscribed to this list, I spent hours and hours trying to
>wrap my mind around TM, I read documents, tickets and annotations even
>on quite remote sites - short, I invested a lot in TM.
>That *IS* an acknowledgement of your work. That *IS* a whole lot of
>trust in your project.
>And you tell me more or less bluntly that I'm stupid, unprofessional
>and using a minority OS ?
>At the same time you suggest that I, the newbie to TM contribute to
>docs and even code ? Uhum. Sure.

You are making the same mistake here that you're accusing me of.

I ignore a lot of messages to this list.  Writing a detailed, point-by- 
point reaction to your criticism is also a sign of respect.  Not all of 
what I have said is even disagreement, for example I acknowledged in my 
first reply that we did a very bad job of describing the implementation 
status of the kqueue reactor.
>Possibly, but I won't because I'd feel strange doing so, knowing quite
>well that kqueue is by many considered to be _the_ AIO implementation
>and knowing that kqueue is used - with excellent results - e.g. by
>some servers (lighttpd, nginx, ...).

FWIW, lighttpd's kqueue support does not appear to be perfect either: 
http://trac.lighttpd.net/trac/ticket/66 ;)
>>  - How about getting 1 version of pykqueue properly running and into 
>>TM ?
>
>I'll do as soon as I find some spare time. right now I need to get
>some product made. Hopefully with TM.

Awesome!
>>Twisted is very, very stable.
>
>Great. Love to hear that.

Hooray :)
>>>  - How about writing some complete docs and tutorials? ...
>
>>Yes, how about that?  You can write documentation on your own
>>site, and we will link to it.  Or, you could contribute to the core
>>documentation.
>
>IF I can do my project with twisted I'll gladly give back.

Again, good.
>a) Before fixing sth I should understand it pretty well. As a newbie
>with TM I'm not in the position to fix things but to learn them -
>ideally with some help.

We are _always_ willing to help out someone who is interested in 
contributing to the project.
>b) I'm afraid humans tend to care most about the tools they work with.
>Frankly, I'm not sure about my involvement with TM if it's big shots
>treat newbies like this.

I hope you can see now why I think this is a little unfair.  Your 
initial post was quite blunt, and I didn't do much but respond in kind. 
I did not intend to insult you, as apparently, you did not intend to 
insult me (or Twisted).  But, you started it ;-).
>Which is not to say TM has been a _complete_ waste of time.  If I find
>my way into it and get it working in time (Again: This is not my hobby
>but my job);  then it's probably worth it.

Well, let's move forward trying to make that happen.

Thank you for your feedback.  Overall it has been very helpful, even the 
parts that I didn't necessarily like.