[Twisted-Python] Refactoring Documentation

[Tom Davis]

On Fri, Jan 21, 2011 at 10:00 PM, Glyph Lefkowitz
<glyph at twistedmatrix.com>wrote:

> On Jan 20, 2011, at 11:20 PM, Tom Davis wrote:
>
> I definitely agree that resolving the low-hanging fruit first is a good
> idea. For finishing "docs branch X" to make sense, my personal belief is
> that X should:
>
>    - Still be relevant in terms of best practices and simply what's
>    available
>    - If project documentation, have outstanding issues that a passing
>    familiarity with the project [right now] will be sufficient to close them (I
>    could spend time learning one project, or the same time improving *all* the
>    documentation)
>    - Adhere to whatever documentation standards we agree upon, if much is
>    to left to do.
>
> I guess my overall opinion here is that, yes, if relatively minor edits can
> bring a branch close enough to completion that we can get it out there to
> help newcomers *now*, let's do that. If a branch is more of a draft and
> requires a good deal of fleshing out (or is simply stale), it's probably
> worth nailing down the structure and previously mentioned docs standards
> before I just create more work for myself (or others) down the road.
>
>
> If branches that are out there *don't* meet these standards, commenting on
> their tickets and getting them deleted or closed as invalid (as appropriate)
> would be a big help too.  Lots of languishing tickets that nobody knows what
> to do with is not a good thing, and there's plenty of opportunities for
> interested parties to reopen tickets, attach new patches, and object in
> various ways, so you shouldn't be too concerned about stepping on toes.
>  Focus is a valuable commodity.
>

I was wondering to what extent it would be helpful to actually reply to all
the tickets, or just the ones that seem to have actionable next steps. I
will try to find something to ask or opine on in each of the documentation
tickets so we can get them moving along or removed.

Part of my comment about low-hanging fruit was to help you get familiar with
> and integrated into the development process.  Going through the process of
> getting patches reviewed and accepted will be _much_ easier if you go
> through the motions of doing a few trivial things first.  In fact you may
> want to just pick up a couple of trivial non-docs patches as well, which
> might help you on documenting the development process :). <
> bit.ly/easy-twisted-tickets> might help you there.
>

I will find an easy ticket or few to bang out. I just replied here (
http://twistedmatrix.com/trac/ticket/2491) in an attempt to get started on
one that doesn't already have a long history.

After spending about an hour going over the "easy" tickets, it seems many of
them are in odd states. Either they're done and waiting on something
undefined or there is an incomplete debate in the comments or the owner
disappeared or... well, there are lots of examples. Maybe this is just me
being dense or whatever, but I think (at least as a newcomer) I could
mass-update all these tickets with "Guidance!" and it would more often than
not be a relevant comment given the state of the ticket.

Here's a great example of what I'm talking about (and I apologize for the
mid-message digression, but I think it's relevant...):
http://twistedmatrix.com/trac/ticket/4636. This seems totally trivial, but
five months later __all__ was never changed in t.i.main and JP's buildbot
link is broken. Whether the offending class should be removed from __all__
or imported instead was never even mentioned. Anyway, I took a stab at it
and attached a trivial patch so this isn't just another complaint (and it
looks like JP closed it while I was drafting this email, so that's great!).
But I do agree that working tickets would really help document the process!


> Mostly, I really don't want you to write a gigantic pile of new
> documentation and then find, when you're "done", that you missed some nuance
> of the coding standard, or the patch is too big to be reasonably reviewed,
> and that now you have three months of additional work to do before it's all
> *really* done.  Experience with the process will mitigate that problem
> significantly.  (And in fact I hope that you don't actually have a gigantic
> pile of stuff to commit all at once at any point, and can continue this work
> incrementally as a series of small tickets, but I realize that later on some
> of the index reorganization stuff may need to be big.  This is mostly just
> restating what Kevin already said in his message, but it bears repeating.)
>

The more I think about this, the more I agree with you. My initial
inclination was just to start from scratch and move over existing docs that
I found I could use. This doesn't conflict with the reuse I've been
supportive of (and for good reason: there's a lot of decent documentation
already there) but it does conflict with the Twisted development policies.

At the end of the day, I can't really submit patches to existing
documentation until said documentation is in Sphinx form. I guess one way
this could work to the happiness of all involved is:

   1. Get re-org nailed down; basic samples for whatever pages I think
   should have standardized format / elements.
   2. Mixed in here, find existing docs and non-docs tickets to work on
   trunk.
   3. When the re-org structure is ready, create a new branch from whatever
   the current Sphinx conversion branch is (hopefully it'll be "done" at this
   point)
   4. Migrate new structure to branch, moving the existing docs to
   semi-logical-but-possibly-temporary places in the new structure.
   5. Submit tickets / patches against existing documentation in said branch
   to move docs to new(er) locations, submit new TDD-style code samples,
   general edits, new docs, etc.
   6. When everything is complete, submit a final ticket to merge to trunk

Does that make sense or am I still cognitively off the mark in terms of
existing development practices?


> Finally, my biggest hurdle right now is not knowing how to *find* said
> branches. I don't see "documentation" as a category in Trac and common
> keyword searches didn't show up much for me. I'm sure this is an easy
> question to answer, though.
>
>
> <
> http://twistedmatrix.com/trac/query?status=assigned&status=new&status=reopened&order=priority&col=id&col=summary&col=status&col=owner&col=type&col=priority&col=milestone&keywords=%7Edocumentation
> >
>
> There is, unfortunately, no "has branch" column in that report, but it will
> at least give you some data to work with.
>
>
>  What *should* a newcomer who reads this document know by the end of it?
>
>
> I'm not sure because I can't see how a practical guide to creating
> something so generic really fits in the grand scheme of things. I think if
> you want to create a TCP server from scratch you must first create the
> Universe! In this case, that means learning how Twisted addresses the *
> concept* of a server before ever bothering to write one so generic. My
> general beef is that many documents seem to make an attempt to appeal to
> everybody and in doing so don't sufficiently help anybody. Maybe I can
> justify that claim better with examples of "better" (at least more targeted)
> documents.
>
>
> I don't think you need a clear definition of "better" for this particular
> document, if it doesn't really fit into your scheme; I don't mind if it
> eventually becomes irrelevant.  I just want a clear statement of goals for
> the documentation _in advance_ of writing that new documentation, so that we
> can discuss whether it's actually better for the intended audience or
> whether it's just more suited to a new author's tastes.
>

I will be fleshing out those goals more in the coming days, but some of it
is implicit within the new documentation structure that's already in my
repo. As for a statement of goals when replacing/removing specific
documentation, sure, I can do that as necessary. It's impossible to
completely remove the element of taste but I will certainly avoid the
"because I like this better" argument for making changes.


>
> You will probably have to press us core developers on this one, and you may
>> spark some debates. These tend to sputter out with no clear resolution as
>> everyone is frustrated that nobody's solution (not even their own) is ideal,
>> but you would be doing us all a great service if you really forced us to
>> develop a consensus about certain things (like "what's the best way to build
>> a twisted command-line program", for example) and agree to agree on the
>> current documented "best practice" for those things.
>
>
> Debates are great!
>
>
> Debates *that reach some kind of conclusion* are great :).  Debates that
> just go in circles until everybody feels crappy about the topic aren't.  So
> I'm really just asking you to help us make these debates into the great
> kind.  (The rest of your reply seems to agree with that, I just wanted to be
> clear.)
>

Yeah, as I've already mentioned in this reply, the "limbo syndrome" of many
tickets drives me nuts. I'm sure you guys hate it too, but having somebody
new around who isn't afraid to ruffle some feathers for the greater good and
isn't yet at some "acceptance" level with it should be helpful! That being
said, I will do my best not to be a jerk about it; you guys have been doing
this for much, much longer and I respect that.


>
> The biggest problem with this is that you will find that a very small group
>> of people have created the vast majority of this stuff and don't have time
>> to maintain it all any more :).  We certainly don't have a separate
>> dedicated maintainer for each project (although I really wish we could get
>> to that point).
>
>
> Kevin touched on this already, but I really think if we make maintaining
> and growing a Project something that is both honorable *and* accessible,
> more people will want to do it.
>
>
> Okay, so, this is really a tangent, but maybe you could let me know what
> you think of <http://tm.tl/2372>.
>

Commented in ticket.


>
> If you like Python and DNS or SMTP or whatever the hell else, what has the
> potential to be a more awesome implementation than its Twisted one? Let's
> help people find out for themselves that the answer is *nothing*. Then
> they'll want it to be *their* project.
>
>
> If you do this, you will be my hero forever.
>

Twisted is a naturally-superior Python choice for most supported protocols
by virtue of the fact that most network "stuff" benefits from event-based
solutions. Like, find me a Python DNS server that *isn't* implemented in
Twisted. People have these weird misconceptions about Twisted; it's really
hard to grasp, ugly, unmaintained, etc. Ultimately, I want to correct these
misconceptions, have *twisted.names* show up first on Google for "Python DNS
Server", and all the other shit that really should be the case but isn't.
Resolving these issues should ultimately resolve the problem of maintainers,
community involvement, etc.

I can't promise that in the course of this crusade I won't cause grief by
trying to change things, but I will always strive to have a rational reason
for wanting to do so!


>
> OK, I'm going to skip the rest of the reply so that I don't write another
> book-length thing here, and I agree with most of the rest of what you've
> said.  So get to it!
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20110122/2c08432c/attachment.html>