t.p.amp : module documentation

This module implements AMP, the Asynchronous Messaging Protocol.

AMP is a protocol for sending multiple asynchronous request/response pairs over the same connection. Requests and responses are both collections of key/value pairs.

AMP is a very simple protocol which is not an application. This module is a "protocol construction kit" of sorts; it attempts to be the simplest wire-level implementation of Deferreds. AMP provides the following base-level features:

• Asynchronous request/response handling (hence the name)
• Requests and responses are both key/value pairs
• Binary transfer of all data: all data is length-prefixed. Your application will never need to worry about quoting.
• Command dispatching (like HTTP Verbs): the protocol is extensible, and multiple AMP sub-protocols can be grouped together easily.

The protocol implementation also provides a few additional features which are not part of the core wire protocol, but are nevertheless very useful:

• Tight TLS integration, with an included StartTLS command.
• Handshaking to other protocols: because AMP has well-defined message boundaries and maintains all incoming and outgoing requests for you, you can start a connection over AMP and then switch to another protocol. This makes it ideal for firewall-traversal applications where you may have only one forwarded port but multiple applications that want to use it.

Using AMP with Twisted is simple. Each message is a command, with a response. You begin by defining a command type. Commands specify their input and output in terms of the types that they expect to see in the request and response key-value pairs. Here's an example of a command that adds two integers, 'a' and 'b':

   class Sum(amp.Command):
       arguments = [('a', amp.Integer()),
                    ('b', amp.Integer())]
       response = [('total', amp.Integer())]

Once you have specified a command, you need to make it part of a protocol, and define a responder for it. Here's a 'JustSum' protocol that includes a responder for our 'Sum' command:

   class JustSum(amp.AMP):
       def sum(self, a, b):
           total = a + b
           print 'Did a sum: %d + %d = %d' % (a, b, total)
           return {'total': total}
       Sum.responder(sum)

Later, when you want to actually do a sum, the following expression will return a Deferred which will fire with the result:

   ClientCreator(reactor, amp.AMP).connectTCP(...).addCallback(
       lambda p: p.callRemote(Sum, a=13, b=81)).addCallback(
           lambda result: result['total'])

Command responders may also return Deferreds, causing the response to be sent only once the Deferred fires:

   class DelayedSum(amp.AMP):
       def slowSum(self, a, b):
           total = a + b
           result = defer.Deferred()
           reactor.callLater(3, result.callback, {'total': total})
           return result
       Sum.responder(slowSum)

This is transparent to the caller.

You can also define the propagation of specific errors in AMP. For example, for the slightly more complicated case of division, we might have to deal with division by zero:

   class Divide(amp.Command):
       arguments = [('numerator', amp.Integer()),
                    ('denominator', amp.Integer())]
       response = [('result', amp.Float())]
       errors = {ZeroDivisionError: 'ZERO_DIVISION'}

The 'errors' mapping here tells AMP that if a responder to Divide emits a ZeroDivisionError, then the other side should be informed that an error of the type 'ZERO_DIVISION' has occurred. Writing a responder which takes advantage of this is very simple - just raise your exception normally:

   class JustDivide(amp.AMP):
       def divide(self, numerator, denominator):
           result = numerator / denominator
           print 'Divided: %d / %d = %d' % (numerator, denominator, total)
           return {'result': result}
       Divide.responder(divide)

On the client side, the errors mapping will be used to determine what the 'ZERO_DIVISION' error means, and translated into an asynchronous exception, which can be handled normally as any Deferred would be:

   def trapZero(result):
       result.trap(ZeroDivisionError)
       print "Divided by zero: returning INF"
       return 1e1000
   ClientCreator(reactor, amp.AMP).connectTCP(...).addCallback(
       lambda p: p.callRemote(Divide, numerator=1234,
                              denominator=0)
       ).addErrback(trapZero)

For a complete, runnable example of both of these commands, see the files in the Twisted repository:

   doc/core/examples/ampserver.py
   doc/core/examples/ampclient.py

On the wire, AMP is a protocol which uses 2-byte lengths to prefix keys and values, and empty keys to separate messages:

   <2-byte length><key><2-byte length><value>
   <2-byte length><key><2-byte length><value>
   ...
   <2-byte length><key><2-byte length><value>
   <NUL><NUL>                  # Empty Key == End of Message

And so on. Because it's tedious to refer to lengths and NULs constantly, the documentation will refer to packets as if they were newline delimited, like so:

   C: _command: sum
   C: _ask: ef639e5c892ccb54
   C: a: 13
   C: b: 81

   S: _answer: ef639e5c892ccb54
   S: total: 94

Notes:

In general, the order of keys is arbitrary. Specific uses of AMP may impose an ordering requirement, but unless this is specified explicitly, any ordering may be generated and any ordering must be accepted. This applies to the command-related keys _command and _ask as well as any other keys.

Values are limited to the maximum encodable size in a 16-bit length, 65535 bytes.

Keys are limited to the maximum encodable size in a 8-bit length, 255 bytes. Note that we still use 2-byte lengths to encode keys. This small redundancy has several features:

• If an implementation becomes confused and starts emitting corrupt data, or gets keys confused with values, many common errors will be signalled immediately instead of delivering obviously corrupt packets.
• A single NUL will separate every key, and a double NUL separates messages. This provides some redundancy when debugging traffic dumps.
• NULs will be present at regular intervals along the protocol, providing some padding for otherwise braindead C implementations of the protocol, so that <stdio.h> string functions will see the NUL and stop.
• This makes it possible to run an AMP server on a port also used by a plain-text protocol, and easily distinguish between non-AMP clients (like web browsers) which issue non-NUL as the first byte, and AMP clients, which always issue NUL as the first byte.

Interface	IArgumentType	An IArgumentType can serialize a Python object into an AMP box and deserialize information from an AMP box back into a Python object.
Interface	IBoxSender	A transport which can send AmpBox objects.
Interface	IBoxReceiver	An application object which can receive AmpBox objects and dispatch them appropriately.
Interface	IResponderLocator	An application object which can look up appropriate responder methods for AMP commands.
Class	AmpError	Base class of all Amp-related exceptions.
Class	ProtocolSwitched	Connections which have been switched to other protocols can no longer accept traffic at the AMP level. This is raised when you try to send it.
Class	OnlyOneTLS	This is an implementation limitation; TLS may only be started once per connection.
Class	NoEmptyBoxes	You can't have empty boxes on the connection. This is raised when you receive or attempt to send one.
Class	InvalidSignature	You didn't pass all the required arguments.
Class	TooLong	One of the protocol's length limitations was violated.
Class	BadLocalReturn	A bad value was returned from a local command; we were unable to coerce it.
Class	RemoteAmpError	This error indicates that something went wrong on the remote end of the connection, and the error was serialized and transmitted to you.
Class	UnknownRemoteError	This means that an error whose type we can't identify was raised from the other side.
Class	MalformedAmpBox	This error indicates that the wire-level protocol was malformed.
Class	UnhandledCommand	A command received via amp could not be dispatched.
Class	IncompatibleVersions	It was impossible to negotiate a compatible version of the protocol with the other end of the connection.
Class	AmpBox	I am a packet in the AMP protocol, much like a regular str:str dictionary.
Class	QuitBox	I am an AmpBox that, upon being sent, terminates the connection.
Class	BoxDispatcher	A BoxDispatcher dispatches '_ask', '_answer', and '_error' AmpBoxes, both incoming and outgoing, to their appropriate destinations.
Class	CommandLocator	A CommandLocator is a collection of responders to AMP Commands, with the help of the Command.responder decorator.
Class	SimpleStringLocator	Implement the locateResponder method to do simple, string-based dispatch.
Class	Argument	Base-class of all objects that take values from Amp packets and convert them into objects for Python functions.
Class	Integer	Encode any integer values of any size on the wire as the string representation.
Class	String	Don't do any conversion at all; just pass through 'str'.
Class	Float	Encode floating-point values on the wire as their repr.
Class	Boolean	Encode True or False as "True" or "False" on the wire.
Class	Unicode	Encode a unicode string on the wire as UTF-8.
Class	Path	Encode and decode filepath.FilePath instances as paths on the wire.
Class	ListOf	Encode and decode lists of instances of a single other argument type.
Class	AmpList	Convert a list of dictionaries into a list of AMP boxes on the wire.
Class	Command	Subclass me to specify an AMP Command.
Class	StartTLS	Use, or subclass, me to implement a command that starts TLS.
Class	ProtocolSwitchCommand	No summary
Class	BinaryBoxProtocol	A protocol for receving Boxes - key/value pairs - via length-prefixed strings. A box is composed of:
Class	AMP	This protocol is an AMP connection. See the module docstring for protocol details.
Class	Decimal	Encodes decimal.Decimal instances.
Class	DateTime	Encodes datetime.datetime instances.
Class	_SwitchBox	Implementation detail of ProtocolSwitchCommand: I am a AmpBox which sets up state for the protocol to switch.
Function	_wireNameToPythonIdentifier	No summary
Class	_NoCertificate	No summary
Class	_TLSBox	I am an AmpBox that, upon being sent, initiates a TLS connection.
Class	_LocalArgument	Local arguments are never actually relayed across the wire. This is just a shim so that StartTLS can pretend to have some arguments: if arguments acquire documentation properties, replace this with something nicer later.
Class	_ParserHelper	A box receiver which records all boxes received.
Function	_stringsToObjects	Convert an AmpBox to a dictionary of python objects, converting through a given arglist.
Function	_objectsToStrings	Convert a dictionary of python objects to an AmpBox, converting through a given arglist.
Class	_FixedOffsetTZInfo	Represents a fixed timezone offset (without daylight saving time).