[Twisted-Python] twisted.ftp status, and metadatafs

[Benjamin Bruheim]

Well, I ought to add some words were I think twisted.ftp is heading too. The first things to do, is to add proper authentication, and filling in some holes (eg, resume and hash). A better path-handling, since os.path is silly under windows, and os.path.isabs('\..'). Distributed is the next big spot. I'm not entirely safe with it yet, but I think it should be easy to nail. One pending 'TODO' is to make it deployable as a systemwide ftp (without requiring each and every user to setup their own ftp-server).

So, if I jump forward in time, and extend the view to storing eg. media-files generally, I think the need of a vfs of some kind arises. What follows is some random notes on it, with varying value, and passing the scope of twisted.ftp with lightyears.
There's two properties it requires: it has to store metadata [1], and it has to support different 'filesystems'. 
The direct effect of the storing the metadata is that the differences between OSes and platforms can be compensated and metadata don't have to be lost due to the filesystems. Though, I think this can be a burden too (should it keep all the metadata in dicts and stored in fragile taps?). 
There's only one filesystem I know which can easily store metadata, and that is a database :) twisted.enterprise could turn handy here, though it seems to require an external SQL-server. Correct me if I'm wrong.
Other filesystems? The OS's filesystem, FTP, Source-Code (hm), 
Some other requirements, is the ability to refer to a file independent of its physical location, pass the reference over PB, do basic operations, and of course, version-checking etc. 

This issue is probably related to the "File Transfer"-layer which is mentioned in the TODO too:

008:
    File Transfer layer for PB.  This would be especially nice for
    twisted.words; having standard a way to transfer "large" (100MB+) packets
    across or in tandem with a PB connection without breaking anything would be
    very good.

[1] http://arstechnica.com/reviews/01q3/metadata/metadata-1.html