[Twisted-Python] Is it just me ....

[Andrew Bennetts]

On Thu, Jul 10, 2003 at 02:02:47AM +0200, Thomas Weholt wrote:
> or is Twisted ... well, complicated. I've been programming for several years
> and I don't consider myself to be dumb, allthough not a genius either ;-) --
> but Twisted is hard to understand. Perhaps you guys find the docs simple
> enough, but I just don't get a lot of the key concepts.

I think Twisted isn't so much complex as large.  It has many pieces, and can
do many things.  Some of those pieces can be complex (e.g. PB or Woven), but
different people use Twisted for a lot of different things.  It's only the
last month or so I've seriously looked a twisted.web; I've found plenty of
use for Twisted in things like implementing simple (and not so simple)
protocols.  In particular, simple multi-protocol support in one process was
a big drawcard for me.  Twisted made that stuff *much* simpler than any of
the alternatives I saw.

> The fact that there are so few examples of running ( and uptodate as
> mentioned by P. Lafoucriere ) apps available might be a clue. Just look at
> the whole RPYs-thing we had going some time ago. Web-development, or to be
> precise, running a full-scale webserver is why a lot of people look into
> Twisted in the first place. The lack of example-apps and obvious
> misunderstandings on how Twisted is meant to be used is what's holding
> Twisted back, IMHO.

This is an interesting point.  Most of the Twisted developers, I think,
didn't come to Twisted for the web stuff, at least not primarily.  Thus the
sudden interest in Twisted purely for its web capabilities has caught the
documentation a little off-guard.  I think people expect to see
documentation a little more geared towards web developers, and instead they
get a bunch of docs that barely mentions web until the sixth chapter.  

Also, Woven has been stable and well-documented for a really short period of
time.  It's not like it's been a year since it stabilised and was
documented and we have nothing to show for it -- rather, it's been perhaps 3
months, and we're just starting to see examples like Issues emerge.

Perhaps the confusion is that to fully use twisted web, you need to
understand:
   - Twisted basics, e.g. Deferreds and reactor.callLater
   - Twisted Web basics, e.g. Resources
   - Woven

But you don't necessarily need full mastery of all that to be productive --
I've seen someone start with the PicturePile tutorial, and extend it to have
captions, nicer layout, and other things without major difficulty.  They
didn't need Deferreds, they didn't need the reactor, and they didn't need to
use twisted.web.resoure.Resource directly, and so they had no problems
building upon the parts of Woven that PicturePile demonstrated.

Let me emphasise that: Twisted is actually pretty good at being modular.  If
you don't need something, you don't need to care about it.  So in a sense,
what we need isn't bigger and better omni-examples that do everything at
once (although they might still be useful) -- we need to help new users
filter out the bits of Twisted they don't need, so they can concentrate on
working with learning a manageable subset of it.

I think the PicturePile tutorial is a great example because of this -- you
can do an awful lot with very little.

So ideally we don't need a "how to use a database in a web page", we should
just need "how to use a database in Twisted" and "how to use deferred
results with Twisted Web".  [I think both of these topics are actually
answered pretty poorly at the moment; I think I can feel another bout of doc
hacking coming on...]

> I really hope the new Twisted-community page can bring some simple, well
> documented examples on how to do things. By the way things are going we'll
> probably have very good docs in a couple of months or so, but I want to
> implement a feature-rich webserver based on Twisted now, as I've described
> and nagged about in my postings on this list several times.

I don't see why we need a new "community" page to achieve this.  We have a
perfectly good mailing-list and website -- if someone has a great example,
we'll gladly use it!

> Anyway, I feel there is too much promise in the Twisted platform to let a
> few good examples scare alot of potential users away. I'm still working on a
> test-implementation of a webserver which hopefully will be easy enough to
> understand and deploy, extend and customize, written in a Twisted-way.

What do you mean by "implementation"?  Have you seen Moshe Zadka's
twisted-web Debian package that's being developed?

> Anyway, I'm going to ask again; any example, small and simple, showing how
> to implement a webserver serving static content *and* dynamic pages, not
> static pages like Issues or Bannerfish, using woven-forms and
> user-authentication, in a Twisted-compatible manner would be appreciated. If
> I'm the only one feeling this way, I'll shut up, dive into the docs again,
> try some more and keep silent while doing so. Hopefully I'll be able to
> learn Twisted and contribute something, not just nag, to a dedicated group
> of developers.

There's no need to shut up -- if you're confused or unsure how to do
something, then ask on the list and/or IRC.  I'd like to think we do a
fairly good job at helping, and conversations on the list can and do lead to
improvements in the docs.

-Andrew.