[Twisted-Python] Configuration options for twisted code

[exarkun at twistedmatrix.com]

On 12:41 pm, adi at roiban.ro wrote:
>>A configuration system would take a while to design
>
>This reads like: A configuration system will never be implemented.
>
>I agree that for the time being we can implement current tickets using
>__init__ arguments,
>but why not try to implement something simple?
>
>Example:
>
>TWISTED_WEB_DEFAULT_CONFIGURATIO = {
>  'request.headers.count': 500,
>  'request.headers.size': 16384,
>  'request.session.name' :  'TWISTED_SESSION',
>  }
>
># We can have a helper method to get default configuration
>configInstance = TWISTED_WEB_DEFAULT_CONFIGURATION.copy()
>configInstance[''request.headers.count'] = 100
>t.w.s.Site(configuration=configInstance)
>
>Or a bit more structured:
>
>class RequestConfiguration(object):
>   def __init__(self):
>     self.max_headers_count = 500
>     self.request_max_header_size = 16384
>     self.session_cookie_name = 'TWISTED_SESSION'
>     self.session_cookie_secure = True
>     self.session_cookie_http_only = True
>
>class TwistedWebConfiguration(object):
>  def __init__(self):
>       self.request = RequestConfiguration()
>
>configInstance = TwistedWebConfiguration()
>configInstance.request.max_headers_count = 100
>
>t.w.s.Site(configuration=configInstance)
>
>Or a mix:
>
>configInstance = TwistedWebConfiguration()
>configInstance.set('request.headers.count', 100)
>t.w.s.Site(configuration=configInstance)
>
>and support some sort of inheritance.
>
>configInstance.set('timeout', 100)
>assert configInstance.get('request.timeout') == 100
>configInstance.set('request.timeout', 200)
>assert configInstance.get('request.timeout') == 200
>
>Or some design used in other project... or some other crazy idea.
>
>-----
>
>In Twisted web I found both configuration overridden in sub-classes
>(t.w.s.Site) and defined via __init__ (t.w.s.File)
>
>twisted.web.static.File has 5 arguments, but the following
>configuration are missing : indexNames, childNotFoundResource,
>forbiddenResource, directoryListing resource
>
>As Jean-Paul commented, when there are too many arguments and they are
>all related, they could be represented by a single argument.
>For me, this single argument could be a configuration object.

The big downside of a "configuration object" is that "configuration" 
isn't any kind of abstraction.  "Configuration" approximately means 
"stuff".  Where does your "configuration" end?  By just saying this is 
where "configuration" goes you've defined its scope as infinite.  Before 
you know it, there's 80,000 items in your "configuration" object and 
you're really much worse off than if you'd just added a couple more 
arguments to `__init__`.

I'd like to see this discussion going in the other direction so to give 
it a little push that way, here's a specific suggestion. :)  Most or all 
of this may be recycled from discussion that's already taken place - I'm 
not taking credit, just repeating it. :)

Give `Request.__init__` some new optional arguments to control the size 
of the header.  This could be several arguments like `maxHeaderLines` 
and `maxHeaderValueLength` and so on... Or perhaps it should just be one 
argument that can completely describe the header size limitation 
(stretching this idea further, perhaps the one argument is a callable 
that takes a `Headers` instance and determines if it has gotten to big).

Then, perhaps, also give `HTTPFactory.__init__` (and therefore 
`Site.__init__` a `requestFactory` argument so that the request class 
can be replaced more easily (although setting the `requestFactory` 
attribute after creating an instance isn't *too* bad).

These two things combined make the usage pretty flexible while still 
being pretty simple and without involving any more complicated 
"configuration" system:

    def requestFactory(*args, **kwargs):
        kwargs["headerSizeLimit"] = HeaderSizeLimit(
            maxLines=23, maxTotalBytes=57, ...)
        return Site.requestFactory(*args, **kwargs)

    site = Site(..., requestFactory=requestFactory)

If anyone is really interested in building (or adopting) a more general 
"configuration" system for Twisted then I suggest you start by looking 
up some of Twisted's past attempts including the the "Coil" library 
Itamar started (he'll have to provide a link, I've lost track of where 
it lives now).

Jean-Paul