twisted.protocols.amp
module documentationtwisted.protocols
View Source
This module implements AMP, the Asynchronous Messaging Protocol.
AMP is a protocol for sending multiple asynchronous request/response pairs over the same connection. Requests and responses are both collections of key/value pairs.
AMP is a very simple protocol which is not an application. This module is a "protocol construction kit" of sorts; it attempts to be the simplest wire-level implementation of Deferreds. AMP provides the following base-level features:
The protocol implementation also provides a few additional features which are not part of the core wire protocol, but are nevertheless very useful:
Using AMP with Twisted is simple. Each message is a command, with a response. You begin by defining a command type. Commands specify their input and output in terms of the types that they expect to see in the request and response key-value pairs. Here's an example of a command that adds two integers, 'a' and 'b':
class Sum(amp.Command): arguments = [('a', amp.Integer()), ('b', amp.Integer())] response = [('total', amp.Integer())]
Once you have specified a command, you need to make it part of a protocol, and define a responder for it. Here's a 'JustSum' protocol that includes a responder for our 'Sum' command:
class JustSum(amp.AMP): def sum(self, a, b): total = a + b print 'Did a sum: %d + %d = %d' % (a, b, total) return {'total': total} Sum.responder(sum)
Later, when you want to actually do a sum, the following expression will
return a Deferred
which
will fire with the result:
ClientCreator(reactor, amp.AMP).connectTCP(...).addCallback( lambda p: p.callRemote(Sum, a=13, b=81)).addCallback( lambda result: result['total'])
Command responders may also return Deferreds, causing the response to be sent only once the Deferred fires:
class DelayedSum(amp.AMP): def slowSum(self, a, b): total = a + b result = defer.Deferred() reactor.callLater(3, result.callback, {'total': total}) return result Sum.responder(slowSum)
This is transparent to the caller.
You can also define the propagation of specific errors in AMP. For example, for the slightly more complicated case of division, we might have to deal with division by zero:
class Divide(amp.Command): arguments = [('numerator', amp.Integer()), ('denominator', amp.Integer())] response = [('result', amp.Float())] errors = {ZeroDivisionError: 'ZERO_DIVISION'}
The 'errors' mapping here tells AMP that if a responder to Divide emits
a ZeroDivisionError
,
then the other side should be informed that an error of the type
'ZERO_DIVISION' has occurred. Writing a responder which takes advantage of
this is very simple - just raise your exception normally:
class JustDivide(amp.AMP): def divide(self, numerator, denominator): result = numerator / denominator print 'Divided: %d / %d = %d' % (numerator, denominator, total) return {'result': result} Divide.responder(divide)
On the client side, the errors mapping will be used to determine what
the 'ZERO_DIVISION' error means, and translated into an asynchronous
exception, which can be handled normally as any Deferred
would
be:
def trapZero(result): result.trap(ZeroDivisionError) print "Divided by zero: returning INF" return 1e1000 ClientCreator(reactor, amp.AMP).connectTCP(...).addCallback( lambda p: p.callRemote(Divide, numerator=1234, denominator=0) ).addErrback(trapZero)
For a complete, runnable example of both of these commands, see the files in the Twisted repository:
doc/core/examples/ampserver.py doc/core/examples/ampclient.py
On the wire, AMP is a protocol which uses 2-byte lengths to prefix keys and values, and empty keys to separate messages:
<2-byte length><key><2-byte length><value> <2-byte length><key><2-byte length><value> ... <2-byte length><key><2-byte length><value> <NUL><NUL> # Empty Key == End of Message
And so on. Because it's tedious to refer to lengths and NULs constantly, the documentation will refer to packets as if they were newline delimited, like so:
C: _command: sum C: _ask: ef639e5c892ccb54 C: a: 13 C: b: 81 S: _answer: ef639e5c892ccb54 S: total: 94
Notes:
In general, the order of keys is arbitrary. Specific uses of AMP may impose an ordering requirement, but unless this is specified explicitly, any ordering may be generated and any ordering must be accepted. This applies to the command-related keys _command and _ask as well as any other keys.
Values are limited to the maximum encodable size in a 16-bit length, 65535 bytes.
Keys are limited to the maximum encodable size in a 8-bit length, 255 bytes. Note that we still use 2-byte lengths to encode keys. This small redundancy has several features:
Variable | MAX_VALUE_LENGTH | The maximum length of a message. (type: int ) |
Variable | ASK | Marker for an Ask packet. (type: bytes ) |
Variable | ANSWER | Marker for an Answer packet. (type: bytes ) |
Variable | COMMAND | Marker for a Command packet. (type: bytes ) |
Variable | ERROR | Marker for an AMP box of error type. (type: bytes ) |
Variable | ERROR_CODE | Marker for an AMP box containing the code of an error. (type: bytes ) |
Variable | ERROR_DESCRIPTION | Marker for an AMP box containing the description of the error. (type: bytes ) |
Interface | IArgumentType | An IArgumentType
can serialize a Python object into an AMP box and deserialize information
from an AMP box back into a Python object. |
Interface | IBoxSender | A transport which can send AmpBox
objects. |
Interface | IBoxReceiver | An application object which can receive AmpBox objects
and dispatch them appropriately. |
Interface | IResponderLocator | An application object which can look up appropriate responder methods for AMP commands. |
Class | AmpError | Base class of all Amp-related exceptions. |
Class | ProtocolSwitched | Connections which have been switched to other protocols can no longer accept traffic at the AMP level. This is raised when you try to send it. |
Class | OnlyOneTLS | This is an implementation limitation; TLS may only be started once per connection. |
Class | NoEmptyBoxes | You can't have empty boxes on the connection. This is raised when you receive or attempt to send one. |
Class | InvalidSignature | You didn't pass all the required arguments. |
Class | TooLong | One of the protocol's length limitations was violated. |
Class | BadLocalReturn | A bad value was returned from a local command; we were unable to coerce it. |
Class | RemoteAmpError | This error indicates that something went wrong on the remote end of the connection, and the error was serialized and transmitted to you. |
Class | UnknownRemoteError | This means that an error whose type we can't identify was raised from the other side. |
Class | MalformedAmpBox | This error indicates that the wire-level protocol was malformed. |
Class | UnhandledCommand | A command received via amp could not be dispatched. |
Class | IncompatibleVersions | It was impossible to negotiate a compatible version of the protocol with the other end of the connection. |
Class | AmpBox | I am a packet in the AMP protocol, much like a regular bytes:bytes dictionary. |
Class | QuitBox | I am an AmpBox that, upon being sent, terminates the connection. |
Class | BoxDispatcher | A BoxDispatcher
dispatches '_ask', '_answer', and '_error' AmpBox es, both
incoming and outgoing, to their appropriate destinations. |
Class | CommandLocator | A CommandLocator
is a collection of responders to AMP Command s, with
the help of the Command.responder
decorator. |
Class | SimpleStringLocator | Implement the AMP.locateResponder
method to do simple, string-based dispatch. |
Class | Argument | Base-class of all objects that take values from Amp packets and convert them into objects for Python functions. |
Class | Integer | Encode any integer values of any size on the wire as the string representation. |
Class | String | Don't do any conversion at all; just pass through 'str'. |
Class | Float | Encode floating-point values on the wire as their repr. |
Class | Boolean | Encode True or False as "True" or "False" on the wire. |
Class | Unicode | Encode a unicode string on the wire as UTF-8. |
Class | Path | Encode and decode filepath.FilePath
instances as paths on the wire. |
Class | ListOf | Encode and decode lists of instances of a single other argument type. |
Class | AmpList | Convert a list of dictionaries into a list of AMP boxes on the wire. |
Class | Descriptor | Encode and decode file descriptors for exchange over a UNIX domain socket. |
Class | Command | Subclass me to specify an AMP Command. |
Class | StartTLS | Use, or subclass, me to implement a command that starts TLS. |
Class | ProtocolSwitchCommand | No summary |
Class | BinaryBoxProtocol | A protocol for receiving AmpBox es -
key/value pairs - via length-prefixed strings. A box is composed of: |
Class | AMP | This protocol is an AMP connection. See the module docstring for protocol details. |
Class | Decimal | Encodes decimal.Decimal instances. |
Class | DateTime | Encodes datetime.datetime instances. |
Class | _SwitchBox | Implementation detail of ProtocolSwitchCommand: I am an AmpBox which sets up state for the protocol to switch. |
Function | _wireNameToPythonIdentifier | No summary |
Class | _NoCertificate | No summary |
Class | _TLSBox | I am an AmpBox that, upon being sent, initiates a TLS connection. |
Class | _LocalArgument | Local arguments are never actually relayed across the wire. This is just a shim so that StartTLS can pretend to have some arguments: if arguments acquire documentation properties, replace this with something nicer later. |
Class | _DescriptorExchanger | _DescriptorExchanger
is a mixin for BinaryBoxProtocol
which adds support for receiving file descriptors, a feature offered by IUNIXTransport . |
Class | _ParserHelper | A box receiver which records all boxes received. |
Function | _stringsToObjects | Convert an AmpBox to a dictionary of python objects, converting through a given arglist. |
Function | _objectsToStrings | Convert a dictionary of python objects to an AmpBox, converting through a given arglist. |
(Private) Normalize an argument name from the wire for use with Python code. If the return value is going to be a python keyword it will be capitalized. If it contains any dashes they will be replaced with underscores.
The rationale behind this method is that AMP should be an inherently multi-language protocol, so message keys may contain all manner of bizarre bytes. This is not a complete solution; there are still forms of arguments that this implementation will be unable to parse. However, Python identifiers share a huge raft of properties with identifiers from many other languages, so this is a 'good enough' effort for now. We deal explicitly with dashes because that is the most likely departure: Lisps commonly use dashes to separate method names, so protocols initially implemented in a lisp amp dialect may use dashes in argument or command names.
Parameters | key | a bytes , looking something like 'foo-bar-baz' or 'from' (type: bytes ) |
Returns | a native string which is a valid python identifier, looking something like 'foo_bar_baz' or 'From'. |
Convert an AmpBox to a dictionary of python objects, converting through a given arglist.
Parameters | strings | an AmpBox (or dict of strings) |
arglist | a list of 2-tuples of strings and Argument objects, as described in Command.arguments . | |
proto | an AMP instance. | |
Returns | the converted dictionary mapping names to argument objects. |
Convert a dictionary of python objects to an AmpBox, converting through a given arglist.
Parameters | objects | a dict mapping names to python objects |
arglist | a list of 2-tuples of strings and Argument objects, as described in Command.arguments . | |
strings | [OUT PARAMETER] An object providing the dict interface which
will be populated with serialized data. | |
proto | an AMP instance. | |
Returns | The converted dictionary mapping names to encoded argument strings
(identical to strings ). |