Twisted Documentation: Logging with twisted.python.log

Basic usage

Twisted provides a simple and flexible logging system in the twisted.python.log module. It has three commonly used functions:

msg

Logs a new message. For example:

from twisted.python import log
log.msg('Hello, world.')

err

Writes a failure to the log, including traceback information (if any). You can pass it a Failure or Exception instance, or nothing. If you pass something else, it will be converted to a string with repr and logged. If you pass nothing, it will construct a Failure from the currently active exception, which makes it convenient to use in an except clause:

try:
    x = 1 / 0
except:
    log.err()   # will log the ZeroDivisionError

startLogging

Starts logging to a given file-like object. For example:

log.startLogging(open('/var/log/foo.log', 'w'))

or:

log.startLogging(sys.stdout)

By default, startLogging will also redirect anything written to sys.stdout and sys.stderr to the log. You can disable this by passing setStdout=False to startLogging.

Before startLogging is called, log messages will be discarded and errors will be written to stderr.

Logging and twistd

If you are using twistd to run your daemon, it will take care of calling startLogging for you, and will also rotate log files. See twistd and tac and the twistd man page for details of using twistd.

Log files

The twisted.python.logfile module provides some standard classes suitable for use with startLogging, such as DailyLogFile, which will rotate the log to a new file once per day.

Using the Python logging module

If your application uses the logging module or you want to use its ease of configuration but don't want to lose twisted-produced messages, the observer PythonLoggingObserver should be useful to you

You just start it like any other observers:

observer = log.PythonLoggingObserver()
observer.start()

And then you'll just have to configure logging to do what you want: logging documentation.

This method allows you to customize the log level received by the logging module using the logLevel keyword:

log.msg("This is important!", logLevel=logging.CRITICAL)
log.msg("Don't mind", logLevel=logging.DEBUG)

Unless logLevel is provided, logging.INFO is used for log.msg and logging.ERROR is used for log.err.

One special care should be made when you use special configuration of the python logging module: some handlers (e.g. SMTP, HTTP) uses network so can block inside the reactor loop. Nothing in the bridge is done to prevent that.

Writing log observers

Log observers are the basis of the Twisted logging system. An example of a log observer in Twisted is the FileLogObserver used by startLogging that writes events to a log file. A log observer is just a callable that accepts a dictionary as its only argument. You can then register it to receive all log events (in addition to any other observers):

twisted.python.log.addObserver(yourCallable)

The dictionary will have at least two items:

message: The message (a list, usually of strings) for this log event, as passed to log.msg or the message in the failure passed to log.err.
isError: This is a boolean that will be true if this event came from a call to log.err. If this is set, there may be a failure item in the dictionary as will, with a Failure object in it.

Other items the built in logging functionality may add include:

printed: This message was captured from sys.stdout, i.e. this message came from a print statement. If isError is also true, it came from sys.stderr.

You can pass additional items to the event dictionary by passing keyword arguments to log.msg and log.err. The standard log observers will ignore dictionary items they don't use.

Important notes:

Never raise an exception from a log observer. If your log observer raises an exception, it will be removed.
Never block in a log observer, as it may run in main Twisted thread. This means you can't use socket or syslog Python-logging backends.
The observer needs to be thread safe if you anticipate using threads in your program.

Customizing `twistd` logging

The behavior of the logging that twistd does can be customized by setting the ILogObserver component on the application object. See the Application document for more information.

Logging with twisted.python.log