Introduction
This is the second part of the Twisted tutorial Twisted from Scratch, or The Evolution of Finger.
In this section of the tutorial, our finger server will continue to sprout features: the ability for users to set finger announces, and using our finger service to send those announcements on the web, on IRC and over XML-RPC.
Setting Message By Local Users
Now that port 1079 is free, maybe we can run on it a different server, one which will let people set their messages. It does no access control, so anyone who can login to the machine can set any message. We assume this is the desired behavior in our case. Testing it can be done by simply:
% nc localhost 1079 # or telnet localhost 1079 moshez Giving a tutorial now, sorry! ^D
# But let's try and fix setting away messages, shall we? from twisted.application import internet, service from twisted.internet import protocol, reactor, defer from twisted.protocols import basic class FingerProtocol(basic.LineReceiver): def lineReceived(self, user): self.factory.getUser(user ).addErrback(lambda _: "Internal error in server" ).addCallback(lambda m: (self.transport.write(m+"\r\n"), self.transport.loseConnection())) class FingerFactory(protocol.ServerFactory): protocol = FingerProtocol def __init__(self, **kwargs): self.users = kwargs def getUser(self, user): return defer.succeed(self.users.get(user, "No such user")) class FingerSetterProtocol(basic.LineReceiver): def connectionMade(self): self.lines = [] def lineReceived(self, line): self.lines.append(line) def connectionLost(self, reason): self.factory.setUser(*self.lines[:2]) # first line: user second line: status class FingerSetterFactory(protocol.ServerFactory): protocol = FingerSetterProtocol def __init__(self, ff): self.setUser = ff.users.__setitem__ ff = FingerFactory(moshez='Happy and well') fsf = FingerSetterFactory(ff) application = service.Application('finger', uid=1, gid=1) serviceCollection = service.IServiceCollection(application) internet.TCPServer(79,ff).setServiceParent(serviceCollection) internet.TCPServer(1079,fsf).setServiceParent(serviceCollection)
Use Services to Make Dependencies Sane
The previous version had the setter poke at the innards of the finger factory. It's usually not a good idea: this version makes both factories symmetric by making them both look at a single object. Services are useful for when an object is needed which is not related to a specific network server. Here, we moved all responsibility for manufacturing factories into the service. Note that we stopped subclassing: the service simply puts useful methods and attributes inside the factories. We are getting better at protocol design: none of our protocol classes had to be changed, and neither will have to change until the end of the tutorial.
# Fix asymmetry from twisted.application import internet, service from twisted.internet import protocol, reactor, defer from twisted.protocols import basic class FingerProtocol(basic.LineReceiver): def lineReceived(self, user): self.factory.getUser(user ).addErrback(lambda _: "Internal error in server" ).addCallback(lambda m: (self.transport.write(m+"\r\n"), self.transport.loseConnection())) class FingerSetterProtocol(basic.LineReceiver): def connectionMade(self): self.lines = [] def lineReceived(self, line): self.lines.append(line) def connectionLost(self,reason): self.factory.setUser(*self.lines[:2]) # first line: user second line: status class FingerService(service.Service): def __init__(self, **kwargs): self.users = kwargs def getUser(self, user): return defer.succeed(self.users.get(user, "No such user")) def getFingerFactory(self): f = protocol.ServerFactory() f.protocol, f.getUser = FingerProtocol, self.getUser return f def getFingerSetterFactory(self): f = protocol.ServerFactory() f.protocol, f.setUser = FingerSetterProtocol, self.users.__setitem__ return f application = service.Application('finger', uid=1, gid=1) f = FingerService(moshez='Happy and well') serviceCollection = service.IServiceCollection(application) internet.TCPServer(79,f.getFingerFactory() ).setServiceParent(serviceCollection) internet.TCPServer(1079,f.getFingerSetterFactory() ).setServiceParent(serviceCollection)
Read Status File
This version shows how, instead of just letting users set their messages, we can read those from a centrally managed file. We cache results, and every 30 seconds we refresh it. Services are useful for such scheduled tasks.
moshez: happy and well shawn: alive
# Read from file from twisted.application import internet, service from twisted.internet import protocol, reactor, defer from twisted.protocols import basic class FingerProtocol(basic.LineReceiver): def lineReceived(self, user): self.factory.getUser(user ).addErrback(lambda _: "Internal error in server" ).addCallback(lambda m: (self.transport.write(m+"\r\n"), self.transport.loseConnection())) class FingerService(service.Service): def __init__(self, filename): self.users = {} self.filename = filename def _read(self): for line in file(self.filename): user, status = line.split(':', 1) user = user.strip() status = status.strip() self.users[user] = status self.call = reactor.callLater(30, self._read) def startService(self): self._read() service.Service.startService(self) def stopService(self): service.Service.stopService(self) self.call.cancel() def getUser(self, user): return defer.succeed(self.users.get(user, "No such user")) def getFingerFactory(self): f = protocol.ServerFactory() f.protocol, f.getUser = FingerProtocol, self.getUser return f application = service.Application('finger', uid=1, gid=1) f = FingerService('/etc/users') finger = internet.TCPServer(79, f.getFingerFactory()) finger.setServiceParent(service.IServiceCollection(application)) f.setServiceParent(service.IServiceCollection(application))
Announce on Web, Too
The same kind of service can also produce things useful for other protocols. For example, in twisted.web, the factory itself (the site) is almost never subclassed -- instead, it is given a resource, which represents the tree of resources available via URLs. That hierarchy is navigated by site, and overriding it dynamically is possible with getChild.
# Read from file, announce on the web! from twisted.application import internet, service from twisted.internet import protocol, reactor, defer from twisted.protocols import basic from twisted.web import resource, server, static import cgi class FingerProtocol(basic.LineReceiver): def lineReceived(self, user): self.factory.getUser(user ).addErrback(lambda _: "Internal error in server" ).addCallback(lambda m: (self.transport.write(m+"\r\n"), self.transport.loseConnection())) class MotdResource(resource.Resource): def __init__(self, users): self.users = users resource.Resource.__init__(self) # we treat the path as the username def getChild(self, username, request): motd = self.users.get(username) username = cgi.escape(username) if motd is not None: motd = cgi.escape(motd) text = '<h1>%s</h1><p>%s</p>' % (username,motd) else: text = '<h1>%s</h1><p>No such user</p>' % username return static.Data(text, 'text/html') class FingerService(service.Service): def __init__(self, filename): self.filename = filename self._read() def _read(self): self.users = {} for line in file(self.filename): user, status = line.split(':', 1) user = user.strip() status = status.strip() self.users[user] = status self.call = reactor.callLater(30, self._read) def getUser(self, user): return defer.succeed(self.users.get(user, "No such user")) def getFingerFactory(self): f = protocol.ServerFactory() f.protocol, f.getUser = FingerProtocol, self.getUser f.startService = self.startService return f def getResource(self): r = MotdResource(self.users) return r application = service.Application('finger', uid=1, gid=1) f = FingerService('/etc/users') serviceCollection = service.IServiceCollection(application) internet.TCPServer(79, f.getFingerFactory() ).setServiceParent(serviceCollection) internet.TCPServer(8000, server.Site(f.getResource()) ).setServiceParent(serviceCollection)
Announce on IRC, Too
This is the first time there is client code. IRC clients often act a lot like servers: responding to events from the network. The reconnecting client factory will make sure that severed links will get re-established, with intelligent tweaked exponential back-off algorithms. The IRC client itself is simple: the only real hack is getting the nickname from the factory in connectionMade.
# Read from file, announce on the web, irc from twisted.application import internet, service from twisted.internet import protocol, reactor, defer from twisted.words.protocols import irc from twisted.protocols import basic from twisted.web import resource, server, static import cgi class FingerProtocol(basic.LineReceiver): def lineReceived(self, user): self.factory.getUser(user ).addErrback(lambda _: "Internal error in server" ).addCallback(lambda m: (self.transport.write(m+"\r\n"), self.transport.loseConnection())) class FingerSetterProtocol(basic.LineReceiver): def connectionMade(self): self.lines = [] def lineReceived(self, line): self.lines.append(line) def connectionLost(self,reason): self.factory.setUser(*self.lines[:2]) class IRCReplyBot(irc.IRCClient): def connectionMade(self): self.nickname = self.factory.nickname irc.IRCClient.connectionMade(self) def privmsg(self, user, channel, msg): user = user.split('!')[0] if self.nickname.lower() == channel.lower(): self.factory.getUser(msg ).addErrback(lambda _: "Internal error in server" ).addCallback(lambda m: irc.IRCClient.msg(self, user, msg+': '+m)) class FingerService(service.Service): def __init__(self, filename): self.filename = filename self._read() def _read(self): self.users = {} for line in file(self.filename): user, status = line.split(':', 1) user = user.strip() status = status.strip() self.users[user] = status self.call = reactor.callLater(30, self._read) def getUser(self, user): return defer.succeed(self.users.get(user, "No such user")) def getFingerFactory(self): f = protocol.ServerFactory() f.protocol, f.getUser = FingerProtocol, self.getUser return f def getResource(self): r = resource.Resource() r.getChild = (lambda path, request: static.Data('<h1>%s</h1><p>%s</p>' % tuple(map(cgi.escape, [path,self.users.get(path, "No such user <p/> usage: site/user")])), 'text/html')) return r def getIRCBot(self, nickname): f = protocol.ReconnectingClientFactory() f.protocol,f.nickname,f.getUser = IRCReplyBot,nickname,self.getUser return f application = service.Application('finger', uid=1, gid=1) f = FingerService('/etc/users') serviceCollection = service.IServiceCollection(application) internet.TCPServer(79, f.getFingerFactory() ).setServiceParent(serviceCollection) internet.TCPServer(8000, server.Site(f.getResource()) ).setServiceParent(serviceCollection) internet.TCPClient('irc.freenode.org', 6667, f.getIRCBot('fingerbot') ).setServiceParent(serviceCollection)
Add XML-RPC Support
In Twisted, XML-RPC support is handled just as though it was another resource. That resource will still support GET calls normally through render(), but that is usually left unimplemented. Note that it is possible to return deferreds from XML-RPC methods. The client, of course, will not get the answer until the deferred is triggered.
# Read from file, announce on the web, irc, xml-rpc from twisted.application import internet, service from twisted.internet import protocol, reactor, defer from twisted.words.protocols import irc from twisted.protocols import basic from twisted.web import resource, server, static, xmlrpc import cgi class FingerProtocol(basic.LineReceiver): def lineReceived(self, user): self.factory.getUser(user ).addErrback(lambda _: "Internal error in server" ).addCallback(lambda m: (self.transport.write(m+"\r\n"), self.transport.loseConnection())) class FingerSetterProtocol(basic.LineReceiver): def connectionMade(self): self.lines = [] def lineReceived(self, line): self.lines.append(line) def connectionLost(self,reason): self.factory.setUser(*self.lines[:2]) class IRCReplyBot(irc.IRCClient): def connectionMade(self): self.nickname = self.factory.nickname irc.IRCClient.connectionMade(self) def privmsg(self, user, channel, msg): user = user.split('!')[0] if self.nickname.lower() == channel.lower(): self.factory.getUser(msg ).addErrback(lambda _: "Internal error in server" ).addCallback(lambda m: irc.IRCClient.msg(self, user, msg+': '+m)) class FingerService(service.Service): def __init__(self, filename): self.filename = filename self._read() def _read(self): self.users = {} for line in file(self.filename): user, status = line.split(':', 1) user = user.strip() status = status.strip() self.users[user] = status self.call = reactor.callLater(30, self._read) def getUser(self, user): return defer.succeed(self.users.get(user, "No such user")) def getFingerFactory(self): f = protocol.ServerFactory() f.protocol, f.getUser = FingerProtocol, self.getUser return f def getResource(self): r = resource.Resource() r.getChild = (lambda path, request: static.Data('<h1>%s</h1><p>%s</p>' % tuple(map(cgi.escape, [path,self.users.get(path, "No such user")])), 'text/html')) x = xmlrpc.XMLRPC() x.xmlrpc_getUser = self.getUser r.putChild('RPC2', x) return r def getIRCBot(self, nickname): f = protocol.ReconnectingClientFactory() f.protocol,f.nickname,f.getUser = IRCReplyBot,nickname,self.getUser return f application = service.Application('finger', uid=1, gid=1) f = FingerService('/etc/users') serviceCollection = service.IServiceCollection(application) internet.TCPServer(79, f.getFingerFactory() ).setServiceParent(serviceCollection) internet.TCPServer(8000, server.Site(f.getResource()) ).setServiceParent(serviceCollection) internet.TCPClient('irc.freenode.org', 6667, f.getIRCBot('fingerbot') ).setServiceParent(serviceCollection)
A simple client to test the XMLRPC finger:
# testing xmlrpc finger import xmlrpclib server = xmlrpclib.Server('http://127.0.0.1:8000/RPC2') print server.getUser('moshez')