[Twisted-Python] Lore to Sphinx Conversion Progress Report 6

[Glyph Lefkowitz]

On Jul 10, 2010, at 4:17 PM, Kevin Horn wrote:

> On Sat, Jul 10, 2010 at 2:40 AM, Tim Allen <screwtape at froup.com> wrote:
> On Sat, Jul 10, 2010 at 02:58:40AM -0000, exarkun at twistedmatrix.com wrote:
> > At last I've got a buildbot set up generating the sphinx docs.  The
> > build results can be seen here:
> >
> >   http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/
> >
> > (or with a different revision number for different revisions; the
> > containing directory is browseable).
> >
> > It would be great if everyone could go look around and report any
> > problems you see.
> 
>    - In the Documentation Table Of Contents page, "Historical
>      Documents" is listed between "Twisted Core" and "Twisted Lore"
>      (presumably because of alphabetical order) but this seems
>      a strange place for it. Could it be moved to the bottom, or an
>      appendix, or something?
> 
> They're in alphabetical order.  The TOC page is generated dynamically by looking at the file structure of the docs and that's the order file-globbing lists them in.  We can fix this manually post-conversion.

It's worth noting that even in this peculiar order, the existence of this document is an upgrade.  It did not previously exist.  Or, rather, it did, but I think the analogous page in the current system is this: <http://twistedmatrix.com/documents/10.1.0/>, which is certainly not as nice.

(If you were expecting to see something like <http://twistedmatrix.com/documents/10.1.0/core/howto/index.html>, that's here: <http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/core/howto/index.html>.)

>  
>    - The bottom of every page has a "Download in other formats: Plain
>      Text" link that doesn't work; presumably scooped up from the Trac
>      templates. It should be removed (the Sphinx "Show Source" link is
>      in the right-hand sidebar anyway).
> 
> Yeah, this is a flaw in the trac theme.  Looks like I forgot to remove it.

This seems like a simple fix - if you could do it quickly so we can see if fixes get propagated to the buildbot, that would be great (so we can get the build logistics out of the way for future, more complex issues).

>    - When the breadcrumb navigation at the top of the page gets too
>      long[1], it wraps to a second line, which looks kind of odd
>      because the nice gradient background image loops. Changing the CSS
>      background colour to be the same as the bottom of the gradient
>      image would probably help.

The breadcrumb names are also kinda weird, and very wordy.

<http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/conch/howto/conch_client.html> says "Twisted >> Twisted Documentation Table of Contents >> Twisted Conch Documentation >> Twisted Documentation".  This is altogether too many "Twisted"s ;-).

It should really be more like

'Twisted >> Documentation >> Conch >> "Writing a client with Twisted.Conch"'

Again: this is a new feature, so I'm not too concerned if we defer fixing this; go ahead if it's easy, but if it's hard, it's still better than the status quo, which is to say, not having any navigation at all.

>    - Chromium's error console reports:
>        - A 404 error for "/trac/chrome/common/js/jquery.js"
>        - A 404 error for "/trac/chrome/common/js/trac.js"
>        - A 404 error for "/trac/chrome/common/js/search.js"
>        - A 404 error for "/builds/sphinx-html-15615/_static/dots.gif"
> 
> Ah! This explains why the search quit working.  Not sure why these files are missing,  Probably something to do with the custom theme.

They're missing because this code is hosted on buildbot.twistedmatrix.com, not twistedmatrix.com.  It's probably most expedient to just make those links include the netloc for now.

>    - The Twisted.Conch tutorial[1] has a number of things surrounded
>      with double-backticks like "``Deferred``" or
>      "``ClientTransport``". I'm not sure if that's deliberate or if
>      it's markup gone horribly wrong.
> 
> This is probably due to missing spaces before/after the markup in the rst source.  One of those things that will need to be fixed manually.  I spent a lot of time trying to fix this in lore2sphinx and when you fix it in one place, it breaks somewhere else.  Right now the automated conversion is about as good as it is likely to get with a sane amount of effort.

Pardon my ignorance, but why not just always add the spaces when generating this type of markup?  It doesn't seem like it should be that hard; docutils claims to be able to treat ReST as structured, right?  Isn't there an API to construct some objects that will do the quoting for you?

>    - The Conch code examples (as linked from the code examples page[2])
>      are sent with a Python mime-type (which Firefox tries to download)
>      while the .tac files are sent as text/html; this may be an
>      artifact of the docs being hosted on the buildbot machine rather
>      than Sphinx, but I think it would be nice if by default they were
>      displayed in the browser, syntax-highlighted, with an option to
>      download the original files (much like Trac does).
> 
> This is a web server configuration thing.  The files are actually .py and .tac files (and probably need to remain so, if we ever want to get automated example code tests going).

> Maybe we can do some web server magic to get them nicely displayed in the browser, but I see that as a secondary issue for the moment.  Anyone should feel free to give a shout if they disagree, though.

I agree completely.

We should fix it so the .tac files are text/x-python as well, so your browser can DTRT with them; when I click on an example, I often want to download it.

Perhaps the lore markup should be adjusted to format them inline as well, though?

>    - The Twisted Split FAQ[3] has a heading "Why are arr the packages
>      still named twisted.*subproject*?", but it's not obvious whether
>      the author was trying to use italics (in which case it's
>      a lore2sphinx bug) or globbing (in which case it's not).
> 
> Pretty sure it was supposed to be italics.  Spacing again.  There are ways to make this display properly, but it'll need to be done manually.
> 
> Also, this is pretty outdated stuff...we might consider just removing some of this.  Or perhaps moving some of it into a more obvious place.

Let's try to have those discussions in separate threads so they can be resolved individually.  I don't have an opinion at the moment and would not like to develop one in the middle of a monster thread about the whole conversion :).
 
>    - The Twisted Coding Standard[6] mentions "If you modify, or write
>      a new, HOWTO, please read the Lore documentation to learn how to
>      format the docs"; if there's not already a follow-up ticket for
>      the lore-sphinx conversion titled "Remove references to Lore from
>      Twisted's non-Lore-related documentation", there should be, and
>      this should be in it.
> 
> There isn't, but there will be.  See the transition plan here: http://twistedsphinx.funsize.net/transition_plan.html

FWIW, you should really start linking to the ticket <http://twistedmatrix.com/trac/ticket/4500>, rather than the "transition plan" page, so that folks can comment, link to other tickets, bookmark it for review when it comes up, etc.

> [...] All bad spacing issues. [...] Almost all of the remaining problems will need to be fixed manually.

It really seems like there's a lot of these.  I'm grateful to Tim for such a thorough job finding them, but I'm sure he missed some - maybe it would be worthwhile to get someone else to help make the tool be correct more often?

> Thanks for the fantastic (and nicely detailed) feedback, Tim!

+1.

(I had some thoughts about manual pages but those deserved their own message, so I'll send them separately.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20100710/e0b6f17a/attachment.html>