twisted.protocols.amp

This module implements AMP, the Asynchronous Messaging Protocol.

AMP is a protocol for sending multiple asynchronous request/response pairs over the same connection. Requests and responses are both collections of key/value pairs.

AMP is a very simple protocol which is not an application. This module is a "protocol construction kit" of sorts; it attempts to be the simplest wire-level implementation of Deferreds. AMP provides the following base-level features:

• Asynchronous request/response handling (hence the name)
• Requests and responses are both key/value pairs
• Binary transfer of all data: all data is length-prefixed. Your application will never need to worry about quoting.
• Command dispatching (like HTTP Verbs): the protocol is extensible, and multiple AMP sub-protocols can be grouped together easily.

The protocol implementation also provides a few additional features which are not part of the core wire protocol, but are nevertheless very useful:

• Tight TLS integration, with an included StartTLS command.
• Handshaking to other protocols: because AMP has well-defined message boundaries and maintains all incoming and outgoing requests for you, you can start a connection over AMP and then switch to another protocol. This makes it ideal for firewall-traversal applications where you may have only one forwarded port but multiple applications that want to use it.

Using AMP with Twisted is simple. Each message is a command, with a response. You begin by defining a command type. Commands specify their input and output in terms of the types that they expect to see in the request and response key-value pairs. Here's an example of a command that adds two integers, 'a' and 'b':

    class Sum(amp.Command):
        arguments = [('a', amp.Integer()),
                     ('b', amp.Integer())]
        response = [('total', amp.Integer())]

Once you have specified a command, you need to make it part of a protocol, and define a responder for it. Here's a 'JustSum' protocol that includes a responder for our 'Sum' command:

    class JustSum(amp.AMP):
        def sum(self, a, b):
            total = a + b
            print 'Did a sum: %d + %d = %d' % (a, b, total)
            return {'total': total}
        Sum.responder(sum)

Later, when you want to actually do a sum, the following expression will return a Deferred which will fire with the result:

    ClientCreator(reactor, amp.AMP).connectTCP(...).addCallback(
        lambda p: p.callRemote(Sum, a=13, b=81)).addCallback(
            lambda result: result['total'])

Command responders may also return Deferreds, causing the response to be sent only once the Deferred fires:

    class DelayedSum(amp.AMP):
        def slowSum(self, a, b):
            total = a + b
            result = defer.Deferred()
            reactor.callLater(3, result.callback, {'total': total})
            return result
        Sum.responder(slowSum)

This is transparent to the caller.

You can also define the propagation of specific errors in AMP. For example, for the slightly more complicated case of division, we might have to deal with division by zero:

    class Divide(amp.Command):
        arguments = [('numerator', amp.Integer()),
                     ('denominator', amp.Integer())]
        response = [('result', amp.Float())]
        errors = {ZeroDivisionError: 'ZERO_DIVISION'}

The 'errors' mapping here tells AMP that if a responder to Divide emits a ZeroDivisionError, then the other side should be informed that an error of the type 'ZERO_DIVISION' has occurred. Writing a responder which takes advantage of this is very simple - just raise your exception normally:

    class JustDivide(amp.AMP):
        def divide(self, numerator, denominator):
            result = numerator / denominator
            print 'Divided: %d / %d = %d' % (numerator, denominator, total)
            return {'result': result}
        Divide.responder(divide)

On the client side, the errors mapping will be used to determine what the 'ZERO_DIVISION' error means, and translated into an asynchronous exception, which can be handled normally as any Deferred would be:

    def trapZero(result):
        result.trap(ZeroDivisionError)
        print "Divided by zero: returning INF"
        return 1e1000
    ClientCreator(reactor, amp.AMP).connectTCP(...).addCallback(
        lambda p: p.callRemote(Divide, numerator=1234,
                               denominator=0)
        ).addErrback(trapZero)

For a complete, runnable example of both of these commands, see the files in the Twisted repository:

    doc/core/examples/ampserver.py
    doc/core/examples/ampclient.py

On the wire, AMP is a protocol which uses 2-byte lengths to prefix keys and values, and empty keys to separate messages:

    <2-byte length><key><2-byte length><value>
    <2-byte length><key><2-byte length><value>
    ...
    <2-byte length><key><2-byte length><value>
    <NUL><NUL>                  # Empty Key == End of Message

And so on. Because it's tedious to refer to lengths and NULs constantly, the documentation will refer to packets as if they were newline delimited, like so:

    C: _command: sum
    C: _ask: ef639e5c892ccb54
    C: a: 13
    C: b: 81

    S: _answer: ef639e5c892ccb54
    S: total: 94

Notes:

In general, the order of keys is arbitrary. Specific uses of AMP may impose an ordering requirement, but unless this is specified explicitly, any ordering may be generated and any ordering must be accepted. This applies to the command-related keys _command and _ask as well as any other keys.

Values are limited to the maximum encodable size in a 16-bit length, 65535 bytes.

Keys are limited to the maximum encodable size in a 8-bit length, 255 bytes. Note that we still use 2-byte lengths to encode keys. This small redundancy has several features:

• If an implementation becomes confused and starts emitting corrupt data, or gets keys confused with values, many common errors will be signalled immediately instead of delivering obviously corrupt packets.
• A single NUL will separate every key, and a double NUL separates messages. This provides some redundancy when debugging traffic dumps.
• NULs will be present at regular intervals along the protocol, providing some padding for otherwise braindead C implementations of the protocol, so that <stdio.h> string functions will see the NUL and stop.
• This makes it possible to run an AMP server on a port also used by a plain-text protocol, and easily distinguish between non-AMP clients (like web browsers) which issue non-NUL as the first byte, and AMP clients, which always issue NUL as the first byte.