[Twisted-Python] ldaptor feedback

[Tommi Virtanen]

On Wed, Jul 02, 2003 at 03:27:44AM +0000, Clark C. Evans wrote:
> 1.  I would like to see ldap in twisted proper, it is really a
>     core protocol (like imap, etc.).   From what I heared there are
>     two arguments against, size and licensing.
> 
>     As for size, I think that LDAP is so important that it really
>     belongs in twisted.   It is at the same level as IMAP.   I do not
>     want to have to go to an external module, hence LDAP is in-scope
>     so the size argument does not matter to me.

	IMAP is <4200 lines with tests, and I believe it is expected
	to be pretty complete protocol, including client and server
	sides. Ldaptor is already almost three times that big,
	client-only, and is still only scratching the surface.

	LDAP is not really used commonly enough to bloat the twisted
	core with it. (Quick! Name three end user applications using
	LDAP!)

	If you think installing two libraries instead of one is too
	hard, you need to get a better OS ;)

>     As for licensing, I understand your concerns.  However, it is very
>     important for enforcement of copyright and for the ability to 'fix'
>     the license that a single entity have the ability to license the
>     and enforce the work as needed.    Long term I'd like to see Glyph
>     form some sort of non-profit to hold the copyrights, or at the
>     very least present a 'will' of sorts describing what happens to
>     the copyrights in the event that he is incapacitated or gets 
>     hit by a MAC truck.    For now, I'm not too worried about this.

	I think we as the community don't need anything better than
	LGPL.

>     As for Glyph selling the software and making a profit... well, 
>     as a bunch of individual packages this just is not an option at
>     all, so better someone than no one.

	I don't feel that way. And most of all, I am not willing to
	delegate that power when it comes to my creation.

	Besides, glyph's major point on that was the game industry.
	I'm betting they will not be interested in LDAP.

>     So, Glyph approving, would you consider contributing ldaptor
>     to twisted?

	No, you are missing the point. That was the starting point;
	if I had been happy with that I would never have moved the
	code from Twisted to Ldaptor.

> 2.  Your package hierarchy needs help... lots of help.

	Agreed.

>     Let's start with this example... 
> 
>     from ldaptor.protocols.ldap.ldapsyntax import LDAPJournalOperation_Replace
>
>     Arg!

	You never do that.

	1. s/from foo.bar import OneClass/from foo import bar/

	2. users of the API never need to touch LDAPJournalOperations,
	   or even the journal itself.

	The current situation looks more like

	from ldaptor.protocol.ldap import ldapsyntax

	o = ldapsyntax.LDAPObject(..)

	which is within the limits of what my emacs is willing to
	dabbrev-expand. Most of the time you just do things like

	d = o.search(...)
	d.addCallback(foo)

>     First, ldaptor has one protocol... ldap.  So "protocols.ldap" is
>     quite redundant.

	There's likely to be LDAPv4. LDAP protocol versions aren't really
	compatible.

	I think LDIF-as-a-replication format is a protocol.

	There's LDUP, the LDAP replication protocol, but that's probably
	just a superset of LDAPv3.

	I think I may end up supporting LDAP-over-UDP, if the standard
	is stable.

	What's really making you grit your teeth is that the mechanism
	an LDAP client uses to open a connection to an LDAP server should
	be in ldaptor.client; and it will be, once I have a clear idea
	what the API should look like. For now, you need to make your own
	subclass of ldapclient.LDAPClient, a Protocol, even if you
	only override *1* method of it! And, remember what I said, everything
	in ldapclient is the old, ugly, API.

	In the future, it'll look more like

	from ldaptor import client

	d=client.openDN('dc=example,dc=com')
	d.addCallback(lambda o: o.search('cn=foo'))

	and you won't need to import the low-level things.

>     Second, ldapsyntax is describing a ldap proxy 
>     object with the ability to journal its changes and save.  It has
>     nothing to do with "syntax" let alone "ldapsyntax", how about 
>     'object' or 'proxy'?.

	Yes, the word syntax is bad. It dates back to the time I tried to
	make the data in LDAP be manipulatable as Python objects, from the
	days before Twisted.

	I refuse to name a class "Object". I refuse to name a class
	"Proxy".

	Besides, I'm a TCP/IP person. To me, a proxy is a thing that
	intermediates between two (different or the same) protocols,
	not something that is really a client-side state for a
	networking API.

>     Third, I count 3 occurances of LDAP... Uncle!

	You purposefully imported the lowest-level thing you could find.
	I could care less how ugly that seems to you. You are not supposed
	to touch that. Internal consistency matters more than shortness
	of the names.

>     Fourth, the class name is *huge* and using prefixes this way sucks, 
>     JournalOperations should be in a file called 'journal'.    In short, 
>     what I'd want to see...
> 
>        from ldaptor import ldap
>        op = ldap.journal.Replace()

	Class named Replace? That way leads to insanity, IMHO.
	What does it replace? With what? Where? Remember, you are
	talking about an implementation detail users of the library
	should never see.

>     The 'low-level' pureber and pureldap are really not protocol
>     items so much as they are parsers/emitters.  You could stuff
>     these, as well as the filter, ldif, and schema all into 
>     another part of ldaptor, ldaptor.parse.ldif, etc.

	There are two things in a protocol:

	1. the wire representation of things
		ldaptor.protocols.pureber
		ldaptor.protocols.pureldap

	2. the state machine
		ldaptor.protocols.ldap.ldapclient
		ldaptor.protocols.ldap.ldapsyntax

	(often #1 is so simple it's not implemented separately)

	The wire representation of the LDAP messages is never
	written to a file, so there's no use for it outside of
	the protocol scope.

	Filter and ldif need to move some day. They started life
	as residing inside the neighbouring .py files, and were
	split out when they grew too big; moving them should be
	reasonably trivial.

	schema needs to be split into pure-ASCII-parser
	(ldaptor.schema?) and the state machine to fetch the used
	schema from the server over LDAP (which should be refactored
	to use LDAPObjects).

>     Anyway, for ldaptor to be understandable it really needs a 
>     refactor like this.   Just look at your commands as to how
>     ugly the packaging is.   Good packaging and naming takes quite
>     a bit of effort and thought.

	Agreed.

> 3.  I can help convert your public interfaces to use Deferreds, and I 
>     can rewrite your examples to use flow (so that they are more
>     readable).    In particular, I was playing with a flow version
>     of your ldap client and it makes things quite clean; to use
>     the client you have a generator which yield ldap commands...

	Thank you for that.

	Please, no flow rewrites for Ldaptor as of now; when flow has
	been in use in Twisted core protocols for ~ 3 months, we can
	talk again.

>     If I'm to help, I'll need commit access and you can't be 
>     completely opposed to me restructuring/commenting things as
>     I move along.   If you need an example of my work, see flow.py

	I think we got a good start on that in IRC.

	I hope to make ldaptor cleaner and clearer. Any help from
	you is welcome.

	Of course, I still get to review your changes and bitch about
	them ;)

> Anyway, I'm curious how open you are to these items.   Adding this to
> Twisted (with alot of work) is probably a good bargin; you get more
> eyes helping your project along, documentation, etc.  

	I have long hoped for some other developers to share the work.

	But no, I will not donate >10000 lines of my work to glyph.
	I'm too much a free software bigot; I want to force him to
	the rules of LGPL. Just like everyone else. In fact, that's
	why I still consider him to have a partial copyright on Ldaptor,
	even though he wrote none of it; that makes _me_ faithful to
	the LGPL, too. And that's a good thing.

	Oh, by the way, you are horribly naive for thinking that
	merely having the code located in the Twisted CVS makes
	everyone else maintain it. That just isn't the way things
	have happened. LDAP is not a core protocol for most people.

-- 
:(){ :|:&};: