[Twisted-Python] Proposal (almost) to switch from Lore to Sphinx

[Kevin Horn]

On Wed, Nov 11, 2009 at 5:56 PM, Glyph Lefkowitz <glyph at twistedmatrix.com>wrote:

>
> On Nov 11, 2009, at 5:52 PM, Kevin Horn wrote:
>
> > Please let me know what you think.
>
> Initial reaction: awesome!
>
>
Well, that's encouraging. :)


> I don't really care about the documentation format or tools, I'm just
> thrilled that someone is willing to step forward and maintain them in _any_
> form.
>
> Secondary reaction: automated testing of code and examples is great.
>  Necessary, even, especially for a new tool.  But doctests are not so much a
> testing plan as a disease that comes to infest your code.
>
> (You are free to disagree with me about the utility or quality of doctest
> as a tool, but doing so on this mailing list is counterproductive, as my
> mind is pretty well made up.  Despite the fact that I have fond feelings
> towards lore's design, this is the *only* point I'd veto a migration over.)
>

I never use doctests in my own projects...but the functionality comes with
Sphinx (as a "built-in extension"...which such a fantastic oxymoron it makes
me smile).

There's lot's of other work being done in the Sphinx testing arena, though,
notably:

Sphinxtest:
http://groups.google.com/group/sphinx-dev/browse_thread/thread/fd9fc42648fc306c#
Manuel: http://packages.python.org/manuel/

and something-I-read-about-the-other-day-but-can't-find-now.


> Automated tests for documentation should look roughly like this:
>
>
> http://divmod.org/trac/browser/trunk/Nevow/nevow/test/test_howtolistings.py?rev=17299
>
> but the stuff that does "sys.modules.clear()" and so on should be a lot
> further away from the actual test cases.
>
> The only thing that is necessary for testing like that to work is the
> ability to reference an external .py file as a code listing.  This is also a
> good thing because then users can just run the Python files straight in the
> examples directory, rather than copying some stuff from a web browser into a
> text editor, mucking around with path variables for a while, trying to make
> sure they got the indentation right etc.
>
> Can sphinx do that out of the box?
>

Yes.  The "literalinclude" directive pulls in an external file and sticks it
in a literal block (the ReST equivalent of a <pre> tag).  So you could use
whatever testing framework you wanted and just reference those files.


>
> Tertiary reaction, since I had to go to divmod.org to dig up that example:
> a migration plan should include the Divmod projects, especially Nevow, since
> a lot of people use that.  But, if there's an automated tool that can handle
> the Twisted conversion, there really shouldn't be any additional effort to
> just run it over the Divmod codebase and just check in the result.  The only
> real work will be in adjusting the buildbot.  This is really just an
> argument for an automated, repeatable conversion process rather than manual
> editing.
>
>
Hmmm...I hadn't considered the Divmod stuff, but you're right.

I think the automated conversion plan is the way to go...up to a point.  As
I laid out in the proposal, I think at some point, the Sphinx docs (and the
automated conversion process) will become "good enough", and then edits
should be done manually to the ReST/Sphinx source files, rather than the
Lore files.  Otherwise you just end up with another tool to maintain
forever.  And in my opinion, Twisted needs to get away from having it's own
collection of self-maintained tools as much as possible. :)

I have some basic code for the conversion process, but it's in a VERY
preliminary state, and needs a fair bit of work.  There's still several tags
it doesn't handle at all (including tables, which _might_ be a real beast),
and it needs to be refactored some.  I mostly wrote it as a proof-of-concept
for myself to prove (to myself) that automated conversion was feasible.
 That said, it _will_ generate a buildable Sphinx project (though it's
currently VERY ugly, and generates warning in the hundreds).

If there's interest, I can put up what I can generate to this point, though
it's awfully early days yet.


Also:
Were my claims/assumption about Lore pretty much correct?  Did I miss
anything?

Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20091111/887bc52c/attachment.html>