[Twisted-Python] Five Crazy Ideas to Start Your Week

[Glyph Lefkowitz]

Too much of the vision for Twisted lives in my head, because I'm concerned
about releasing unfinished visions and having lots of potentially contradictory
documents live in the repository.  So, I'm going to try now every once in a
while to vent some of these ideas that have been playing around inside my head.

This is my first attempt at doing this, so feel free to ask questions.

These are all features that I intend to implement personally, but if anyone
wants to contribute a patch for me to review, I'd love to see it.


1. Perspective Broker URLS
==========================

The current (huge constellation of) connection APIs that PB ships with are all
subtly broken in one way or another.  They all walk the line between "too
verbose" and "too limited".

In essence what PB needs is a mini-language for specifying these connection
conversations.  URLs provide a very useful basis for this sort of
mini-language.  I haven't worked out a lot of the details of this connection
scheme yet (otherwise I would have implemented it) but it should be pretty
straightforward extension of the ideas in HTTP URLs.

These URLs need to be able to represent transport mechanism (TCP? UNIX socket?
SSL? Weird firewall-tunneling stuff?) plus everything about the object that
you're trying to access (location? authentication information? authentication
format?)

The idea is that you pass in a string, you get a Deferred PB "endpoint", which
should be the object that you're really wanting to interact with.  Here are
some off-the-cuff examples:

    pb://pyramid/service#perspective

Log in to Pyramid, with the given service and perspective.  The part of the
login conversation which attempts to get the "child" object from the service
will prompt for interactive authentication.

    pb://pyramid/identity:password@service#perspective

Normal login/password to service/perspective pair.  Challenge-response
authentication implied.

    pb://pyramid/md5-cr$identity:password@service#perspective

Same thing, normal challenge-response explicitly selected.

    pb-ssh://pyramid/sshkey$identity@service#perspective

Public-key authentication mechanism, using the same identity name.  Connection
over SSH transport (sshkey auth only available this way).

    pb-ssl://pyramid/cert$identity@service#perspective

PB over SSL, using certificate auth.

    pb://pyramid/this/that/the-other/my-service#perspective

A service located within a tree of services.  Again the my-service bit of the
path will prompt for interactive authentication.

    pb://pyramid/user@sekrit/otheruser@double-sekrit/super-sekrit-service#perspective

Two objects in the path that require separate authentication.  An unusual, but
still possible situation.


2. Automatic XSS input scrubbing for Woven
==========================================

Form input that is intended to be used as HTML can be scanned for potentially
abusive input so that the web administration interface for a bulletin board
written using Twisted doesn't fall prey to the usual "cross site scripting"
vulnerabilities that plague feedback-capable web sites.

The problem formulation, generally, is that an attacker will place some
malicious javascript in a web form.  When some article is viewed by an admin,
that malicious javascript executes in their browser, taking advantage of the
fact that they have an admin cookie for that website in their browser.  It will
take advantage of a security flaw in the browser and perform administrative
tasks with their authority on the website without the admin's approval.

There are pretty much only a few ways they can do this, and I believe each one
has a simple solution.

    - <script /> tags

Don't allow script tags in input, ever.  Force input that uses tags into a DOM
tree before it is even made to be preview-able.  (DOM parsing can be relaxed,
but before the tree is output, it is converted to well-formed XML).

    - <a href="..." /> tags

Stick a little warning after any "off-site" link, and turn it into a redirect.
Any admin clicking on such a link will first be bounced through a URL which
revokes their admin cookie so the attacking site won't have the ability to call
back.

    - attributes on other HTML tags, such as onmouseover, that execute js
      expressions

Catalogue these and scrub them all off.

    - <img /> or <object /> tags that automatically download other items, which
      may be webpages containing javascript.

Again, don't allow these tags.  Or, have the server-side site download the
images and verify that they are PNGs (alternately also doing PIL stuff to
remove steganographic information, if you're _really_ paranoid).


3. widgets.Form-like Abstraction for Woven (and others) 
=======================================================

(This has been alternately called "form methods" or "actions".)  The idea is
that we have a meta-data format for publishing methods to have user interfaces,
which would most likely look like forms.  This metadata should be
platform-neutral and have a Woven front-end widget (controller?) that formats
it prettily, perhaps in conjunction with a template for the form itself.

This interface should also extend to other UIs.  Having a GTK interface would
be particularly useful for building GUI equivalents of tools like COIL, which
should be able to have a GUI hook into it as easily as a web page.

Additionally there should be some support for remote methods.  A PB reference
should be able to present at least a trivial version of this meta-data for its
objects.  That way you could present user-interfaces to arbitrary PB objects
(and, with a little magic, format those interfaces through a local web-proxy,
for total platform independence.  *OR* have a GTK client!)

This really sounds like black magic, but there is some precedent for it in
widgets.Form.  The general idea is that, for a method like:

  def formattedMath(op, a, b):
      result = op(a, b)
      return "%s %s %s = %s" % (a, op.__name__, b, result)

You'd present a list that looked like:

   import form
   [["op", form.Menu(["Multiply", operator.mul],
                     ["Add", operator.add],
                     ["Subtract", operator.sub],
                     ["Divide", operator.div])],
    ["a", form.Number()],
    ["b", form.Number()]]

That would provide a user-interface that could provide nice input constraints
for invoking that method.

The biggest problem I've hit so far with this idea is how to provide a
reasonable default appearance of various data-types while still allowing for
formattable customization in the template.  I've got some vague ideas of how to
fix this by allowing embedding of certain specifically-named elements in a
template and providing a default template that has all elements provided.

Even if I managed to get this right for the web, though, trying to avoid lock
in to the web's way of thinking about forms is very tricky!  I have yet to
seriously think about how GTK, PyUI, or Tkinter would work with this.  I have
spent a little while thinking about how a text-based virtual world interface in
twisted reality, though, and that should be pretty easy (building a machine
with dials, knobs, etc, and a big button for "submit").

4. Generalized Tree Structure for Twisted Applications
======================================================

I want to have it so that Twisted apps are easily navigable and arbitrarily
manipulable, almost like a virtual filesystem.  The most obvious application of
this is to make something like 'mktap --append' generally _work_ when you add
stuff to a .tap.

The current hierarchical nature of services is a step in this direction, though
it clearly doesn't go high enough.  Right now, mktap --append could be checking
for the existence of services, allowing you to add new services at given
'paths' inside the application without binding ports... though it isn't, and
this requires a fair amount of logical restructuring of the tap plugin
metaphor.

In keeping with the idea of URLs for PB, it would be nice if each object, as it
was inserted into this hierarchy, would get a notification that it was now
available at a new URL, or that it had been removed from a URL.
(availableAtURL) These URLs would include information about how it was
accessible from within the .tap (twisted://tcp/80) and how it was accessible to
the outside world (http://64.123.27.105/you/are/here/,
telnet://user@64.123.27.105/obj).

Objects could much more reliably provide hyperlinks to themselves in this case,
and that would be a very neat feature.  In addition, the sysadmin's view of
Twisted would be greatly simplified.


5. Future directions for Words
==============================

I already explained this to some extent in Words's massive docstring update, so
I'll be brief.  Twisted words ought to be a distributed, consensus directory
service for chat groups, not a centralized (though hierarchical and
geographically dispersed) message relaying service like IRC.

Chat groups really ought to have a lot of metadata associated with them,
including a customizable summary homepage and an RSS feed.  Ideally once we're
sure that Woven is not terribly exploitable via the web (see idea 2), we can
allow users to upload templates directly.

Also, group membership should be persistent.  The ability to join the group as
a "regular" is an interesting function and I'm wondering how it can be done.
First-come-first-serve options for grabbing names seems like they'd encourage
squatting.  I think that by having a distributed naming system (where different
people call the "python" group different things, but python at twistedmatrix.com
will always get you to the one you're looking for) might circumvent squatting.

-------------------

Anyway, that's all for tonight.  I hope that this is useful to some of you out
there who are wondering what I'm thinking about such things :)

-- 
 |    <`'>    |  Glyph Lefkowitz: Traveling Sorcerer   |
 |   < _/ >   |  Lead Developer,  the Twisted project  |
 |  < ___/ >  |      http://www.twistedmatrix.com      |
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: </pipermail/twisted-python/attachments/20021008/fe6cbb05/attachment.sig>