Creating XML-RPC Servers and Clients with Twisted

Introduction

XML-RPC is a simple request/reply protocol that runs over HTTP. It is simple, easy to implement and supported by most programming languages. Twisted's XML-RPC support is implemented using the xmlrpclib library that is included with Python 2.2 and later.

Creating a XML-RPC server

Making a server is very easy - all you need to do is inherit from twisted.web.xmlrpc.XMLRPC. You then create methods beginning with xmlrpc_. The methods' arguments determine what arguments it will accept from XML-RPC clients. The result is what will be returned to the clients.

Methods published via XML-RPC can return all the basic XML-RPC types, such as strings, lists and so on (just return a regular python integer, etc). They can also raise exceptions or return Failure instances to indicate an error has occurred, or Binary, Boolean or DateTime instances (all of these are the same as the respective classes in xmlrpclib. In addition, XML-RPC published methods can return Deferred instances whose results are one of the above. This allows you to return results that can't be calculated immediately, such as database queries. See the Deferred documentation for more details.

XMLRPC instances are Resource objects, and they can thus be published using a Site. The following example has two methods published via XML-RPC, add(a, b) and echo(x).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
from twisted.web import xmlrpc, server

class Example(xmlrpc.XMLRPC):
    """
    An example object to be published.
    """

    def xmlrpc_echo(self, x):
        """
        Return all passed args.
        """
        return x

    def xmlrpc_add(self, a, b):
        """
        Return sum of arguments.
        """
        return a + b

    def xmlrpc_fault(self):
        """
        Raise a Fault indicating that the procedure should not be used.
        """
        raise xmlrpc.Fault(123, "The fault procedure is faulty.")

if __name__ == '__main__':
    from twisted.internet import reactor
    r = Example()
    reactor.listenTCP(7080, server.Site(r))
    reactor.run()

After we run this command, we can connect with a client and send commands to the server:

>>> import xmlrpclib
>>> s = xmlrpclib.Server('http://localhost:7080/')
>>> s.echo("lala")
'lala'
>>> s.add(1, 2)
3
>>> s.fault()
Traceback (most recent call last):
...
xmlrpclib.Fault: <Fault 123: 'The fault procedure is faulty.'>
>>>

If the Request object is needed by an xmlrpc_* method, it can be made available using the twisted.web.xmlrpc.withRequest decorator. When using this decorator, the method will be passed the request object as the first argument, before any XML-RPC parameters. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
from twisted.web.xmlrpc import XMLRPC, withRequest
from twisted.web.server import Site

class Example(XMLRPC):
    @withRequest
    def xmlrpc_headerValue(self, request, headerName):
        return request.requestHeaders.getRawHeaders(headerName)

if __name__ == '__main__':
    from twisted.internet import reactor
    reactor.listenTCP(7080, Site(Example()))
    reactor.run()

XML-RPC resources can also be part of a normal Twisted web server, using resource scripts. The following is an example of such a resource script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
from twisted.web import xmlrpc
import os

def getQuote():
    return "What are you talking about, William?"

class Quoter(xmlrpc.XMLRPC):

    def xmlrpc_quote(self):
        return getQuote()

resource = Quoter()

Using XML-RPC sub-handlers

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
import time
from twisted.web import xmlrpc, server

class Example(xmlrpc.XMLRPC):
    """
    An example object to be published.
    """

    def xmlrpc_echo(self, x):
        """
        Return all passed args.
        """
        return x

    def xmlrpc_add(self, a, b):
        """
        Return sum of arguments.
        """
        return a + b

class Date(xmlrpc.XMLRPC):
    """
    Serve the XML-RPC 'time' method.
    """

    def xmlrpc_time(self):
        """
        Return UNIX time.
        """
        return time.time()

if __name__ == '__main__':
    from twisted.internet import reactor
    r = Example()
    date = Date()
    r.putSubHandler('date', date)
    reactor.listenTCP(7080, server.Site(r))
    reactor.run()

Using your own procedure getter

Sometimes, you want to implement your own policy of getting the end implementation. E.g. just like sub-handlers you want to divide the implementations into separate classes but may not want to introduce XMLRPC.separator in the procedure name. In such cases just override the lookupProcedure(self, procedurePath) method and return the correct callable. Raise twisted.web.xmlrpc.NoSuchFunction otherwise.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
from twisted.web import xmlrpc, server

class EchoHandler:

    def echo(self, x):
        """
        Return all passed args
        """
        return x



class AddHandler:

    def add(self, a, b):
        """
        Return sum of arguments.
        """
        return a + b



class Example(xmlrpc.XMLRPC):
    """
    An example of using you own policy to fetch the handler
    """

    def __init__(self):
        xmlrpc.XMLRPC.__init__(self)
        self._addHandler = AddHandler()
        self._echoHandler = EchoHandler()

        #We keep a dict of all relevant
        #procedure names and callable.
        self._procedureToCallable = {
            'add':self._addHandler.add,
            'echo':self._echoHandler.echo
        }

    def lookupProcedure(self, procedurePath):
        try:
            return self._procedureToCallable[procedurePath]
        except KeyError, e:
            raise xmlrpc.NoSuchFunction(self.NOT_FOUND,
                        "procedure %s not found" % procedurePath)

    def listProcedures(self):
        """
        Since we override lookupProcedure, its suggested to override
        listProcedures too.
        """
        return ['add', 'echo']



if __name__ == '__main__':
    from twisted.internet import reactor
    r = Example()
    reactor.listenTCP(7080, server.Site(r))
    reactor.run()

Adding XML-RPC Introspection support

XML-RPC has an informal Introspection API that specifies three methods in a system sub-handler which allow a client to query a server about the server's API. Adding Introspection support to the Example class is easy using the XMLRPCIntrospection class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
from twisted.web import xmlrpc, server

class Example(xmlrpc.XMLRPC):
    """An example object to be published."""

    def xmlrpc_echo(self, x):
        """Return all passed args."""
        return x

    xmlrpc_echo.signature = [['string', 'string'],
                             ['int', 'int'],
                             ['double', 'double'],
                             ['array', 'array'],
                             ['struct', 'struct']]

    def xmlrpc_add(self, a, b):
        """Return sum of arguments."""
        return a + b

    xmlrpc_add.signature = [['int', 'int', 'int'],
                            ['double', 'double', 'double']]
    xmlrpc_add.help = "Add the arguments and return the sum."

if __name__ == '__main__':
    from twisted.internet import reactor
    r = Example()
    xmlrpc.addIntrospection(r)
    reactor.listenTCP(7080, server.Site(r))
    reactor.run()

Note the method attributes help and signature which are used by the Introspection API methods system.methodHelp and system.methodSignature respectively. If no help attribute is specified, the method's documentation string is used instead.

SOAP Support

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
from twisted.web import soap
import os

def getQuote():
    return "That beverage, sir, is off the hizzy."

class Quoter(soap.SOAPPublisher):
    """Publish one method, 'quote'."""

    def soap_quote(self):
        return getQuote()

resource = Quoter()

Creating an XML-RPC Client

The interface Twisted presents to XML-RPC client is that of a proxy object: twisted.web.xmlrpc.Proxy. The constructor for the object receives a URL: it must be an HTTP or HTTPS URL. When an XML-RPC service is described, the URL to that service will be given there.

Having a proxy object, one can just call the callRemote method, which accepts a method name and a variable argument list (but no named arguments, as these are not supported by XML-RPC). It returns a deferred, which will be called back with the result. If there is any error, at any level, the errback will be called. The exception will be the relevant Twisted error in the case of a problem with the underlying connection (for example, a timeout), IOError containing the status and message in the case of a non-200 status or a xmlrpclib.Fault in the case of an XML-RPC level problem.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
from twisted.web.xmlrpc import Proxy
from twisted.internet import reactor

def printValue(value):
    print repr(value)
    reactor.stop()

def printError(error):
    print 'error', error
    reactor.stop()

proxy = Proxy('http://advogato.org/XMLRPC')
proxy.callRemote('test.sumprod', 3, 5).addCallbacks(printValue, printError)
reactor.run()

prints:

[8, 15]

Serving SOAP and XML-RPC simultaneously

twisted.web.xmlrpc.XMLRPC and twisted.web.soap.SOAPPublisher are both Resources. So, to serve both XML-RPC and SOAP in the one web server, you can use the putChild method of Resource.

The following example uses an empty resource.Resource as the root resource for a Site, and then adds /RPC2 and /SOAP paths to it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
from twisted.web import soap, xmlrpc, resource, server
import os

def getQuote():
    return "Victory to the burgeois, you capitalist swine!"

class XMLRPCQuoter(xmlrpc.XMLRPC):
    def xmlrpc_quote(self):
        return getQuote()

class SOAPQuoter(soap.SOAPPublisher):
    def soap_quote(self):
        return getQuote()

def main():
    from twisted.internet import reactor
    root = resource.Resource()
    root.putChild('RPC2', XMLRPCQuoter())
    root.putChild('SOAP', SOAPQuoter())
    reactor.listenTCP(7080, server.Site(root))
    reactor.run()

if __name__ == '__main__':
    main()

Refer to Twisted Web Development for more details about Resources.