| A Logging System for Python | Home Download Copyright & License Recent Changes API Documentation |
| "Oh, I'm a lumberjack and I'm okay..." (Monty Python, The Lumberjack Song) |
Abstract
Motivation
Influences
A Simple Example
Control Flow
Levels
Loggers
Handlers
Formatters
Filters
Configuration
The GUI Configurator
Case Scenarios
Thread Safety
On-The-Fly Reconfiguration
Module-Level Convenience Functions
Performance
Implementation Status
Acknowledgements
Still To Do
Download and Installation
Change History
Copyright and License
There was a need for a standard logging system in Python, as comprehensively documented in PEP 282 and enthusiastically endorsed by the BDFL in the Parade of the PEPs. By a happy coincidence, the package described here was already in development and fairly close in intent and design to the description in the aforementioned PEP, borrowing as it did heavily from JSR-47 (now JDK 1.4's java.util.logging package) and log4j. This page describes it in more detail. As I have tweaked the package to meet comments on PEP 282, I have structured this page in the same way as the original PEP. This package is now part of Python 2.3, but if you have an earlier version of Python, you can download the package from here and use it with Python versions between 1.5.2 and 2.2.x.
The Python community has been incredibly helpful to me, a relative newcomer to the language. Python and its community has certainly saved me much time and effort, and it seems appropriate to give something back to the community by offering up this package for people to try. Any feedback will be gratefully accepted.
This package owes its greatest debt to Apache log4j. Due notice was also taken of log4j's comprehensive critique (no longer online) of JSR47. This package bears a close resemblance to log4j, but is not a close translation. I have attempted to be more minimalist (and hopefully more Pythonic) in my approach. You be the judge!
Using the package doesn't get much simpler. It is packaged as a standard Python package
called (unsurprisingly) logging. You just need to import logging
and you're ready to go. Minimal example:
# --- app.py --------------------------------------------------------------------
import logging
logging.warn("Hello")
logging.error("Still here...")
logging.warn("Goodbye")
When you run app.py, the results are:
WARNING:root:Hello ERROR:root:Still here... WARNING:root:Goodbye
Don't worry about the format of the output - it's all configurable. Here's a slightly more involved example; if you've just looked at PEP 282 you will probably get a feeling of dejà vu. (This is intentional.)
# --- mymodule.py --------------------------------------------------------------------
import logging
log = logging.getLogger("MyModule")
def doIt():
log.debug("doin' stuff")
#do stuff...but suppose an error occurs?
raise TypeError, "bogus type error for testing"
# --- myapp.py -----------------------------------------------------------------------
import logging, mymodule
logging.basicConfig()
log = logging.getLogger("MyApp")
log.setLevel(logging.DEBUG) #set verbosity to show all messages of severity >= DEBUG
log.info("Starting my app")
try:
mymodule.doIt()
except Exception, e:
log.exception("There was a problem.")
log.info("Ending my app")
When you run myapp.py, the results are:
INFO:MyApp:Starting my app
ERROR:MyApp:There was a problem.
Traceback (most recent call last):
File "myapp.py", line 9, in ?
mymodule.doIt()
File "mymodule.py", line 7, in doIt
raise TypeError, "Bogus type error for testing"
TypeError: Bogus type error for testing
INFO:MyApp:Ending my app
But don't worry - the above output is not hardcoded into the package. It's just an example of what you can do with very little work. As you can see, exceptions are handled as one would expect.
The package pretty much matches the PEP regarding control flow. The user of the package
makes logging calls on instances of Logger, which are organized into a
hierarchy based on a "dotted name" namespace. This hierarchy is embodied in an
encapsulated singleton Manager instance (which can be ignored by users of the
package, for most purposes). Based on the type of logging call and the logging
configuration (see below), the call may be passed through a set of Filter
instances to decide whether it should be dropped. If not, then the logger consults a set
of Handler instances which are associated with it, and asks each handler
instance to "handle" the logging event. By default, the system moves up the
namespace hierarchy and invokes handlers on all loggers at or above the level of the
logger on which the logging call was made. (You can override this by setting a logger's
"propagate" attribute to 0 - no traversal up the hierarchy is made from such a
logger. But I'm getting ahead of myself...)
Handlers are passed LogRecord instances which (should) contain all the
information we're interested in logging. Handlers, too, can invoke filters to determine
whether a record should be dropped. If not, then the handler takes a handler-specific
action to actually log the record to a file, the console or whatever.
The following levels are implemented by default:
DEBUG INFO WARNING ERROR CRITICAL
The CRITICAL level replaces the earlier FATAL level. You can
use either (for now), but CRITICAL is preferred since FATAL
implies that the application is about to terminate. This is not true for many systems
which use logging - for example, a Web server application which encounters a CRITICAL
condition (e.g. running out of resources) will still try to keep going as best it can.
FATAL (and the corresponding fatal() methods) may be removed
in future versions of the package. Currently, CRITICAL is synonymous with FATAL
and critical() methods are synonymous with fatal().
Exceptions logged via exception() use the ERROR level for
logging. If it is desired to log exception information with arbitrary logging levels, this
can be done by passing a keyword argument exc_info with a true value to the
logging methods (see the API documentation for more details).
The levels are not deeply hardcoded into the package - the number of levels, their numeric values and their textual representation are all configurable. The above levels represent the experience of the log4j community and so are provided as the default levels for users who do not have very specific requirements in this area.
The example script log_test4.py shows the use of bespoke logging levels
(as well as filtering by level at logger and handler, as well as use of filter classes).
The package implements loggers pretty much as mentioned in the PEP, except that the
manager class is called Manager rather than LogManager.
Each Logger instance represents "an area" of the application. This somewhat nebulous definition is needed because it's entirely up to each application developer to define an application's "areas".
For example, an application which reads and processes spreadsheet-type data in different formats might have an overall area "input", concerned with reading input files; and areas "input.csv", "input.xls" and "input.gnu", related to processing comma-separated-value, Excel and Gnumeric input files. Logging messages relating to the overall input function (e.g. deciding which files to process) might be logged used the logger named "input"; logging messages relating to reading individual files might be sent to any of "input.csv", "input.xls" or "input.gnu" depending on the type of file being read.
The advantage of the hierarchical structure is that logging verbosity may be controlled either at the high level or the low level. The levels are loosely coupled and new levels can easily be added at a later date, e.g."input.wks" for reading Lotus-123 format files. It's also possible to do things like routing messages relating to Excel file input to whoever is working on Excel imports, messages related to Gnumeric file processing to a different developer, and so on. Even if the same person works on both, they can at different times focus logging verbosity on particular areas of interest - for example, when debugging Excel imports, they can set the "input.xls" logger's verbosity to DEBUG and others to CRITICAL, and when moving to debug Gnumeric imports, they can reduce the "input.xls" verbosity by setting the level to CRITICAL, while increasing "input.gnu"'s verbosity by setting the level to DEBUG.
The following handlers are implemented. I guess they could use more testing ;-)
All of these except the first two are defined in a sub-module, handlers.py. (To use
these handlers, you'll need to import logging.handlers. In addition to the
above list, there are example implementations of XMLHandler (see log_test9.py),
BufferingSMTPHandler (see log_test11.py) and DBHandler
(see log_test14.py) in the test harnesses, on which you can base more
specific classes. There is also a class called SLHandler (see log_test1.py)
which implements an alternative SysLogHandler - one which uses the syslog module in the
standard library (and which is therefore only available on Unix).
SOAPHandler, which sends events to a SOAP server, has moved (as of release 0.4.4) from
the core to an example script (log_test13.py). The SOAP message is packaged as a function
call to a single log() function on the remote server, which takes each
relevant member of the LogRecord as a positional parameter. This is perhaps not ideal -
but then this SOAPHandler is just a proof-of-concept example to get you started ;-)
Note that the handlers are specifically intended not to raise exceptions when errors occur at runtime. This is to avoid error messages from the logging infrastructure polluting logging messages from the application being logged. If, for example, a SocketHandler sees a connection reset by the remote endpoint, it will silently drop all records passed to it (but it will try to connect each time). It may be that due to bugs there are some exceptions incorrectly raised by the logging system, I will try to rectify this kind of problem as soon as it is found and reported!
A basic Formatter has been implemented, which should cater for most immediate
requirements. You basically initialize the Formatter with a format string which knows how
the attribute dictionary of a LogRecord looks. For example, the output in the example
above was produced with a format string of "%(asctime)s %(name)-19s
%(levelname)-5s - %(message)s". Note that the "message" attribute of
the LogRecord is derived from "msg % args" where msg
and args are passed by the the user in a logging call.
Filters are used to refine logging output at either logger or handler level with a finer control than is available by just using logging levels. The basic Filter class takes an optional name argument and passes all logging records from loggers which are at or below the specified name.
For example, a Filter initialized with "A.B" will allow events
logged by loggers "A.B", "A.B.C", "A.B.C.D",
"A.B.D" but not "A.BB", "B.A.B". If no name is specified,
all events are passed by the filter.
A basic configuration is provided via a module-level function, basicConfig().
If you want to use very simple logging, you can just call the module-level
convenience functions and they will call basicConfig() for you if
necessary. It (basically) adds a StreamHandler (which writes to sys.stderr)to
the root Logger.
There are numerous examples of configuration in the test/example scripts included in
the distribution. For example, log_test8.py has an example of using a
file-based logger.
An alternative using ConfigParser-based configuration files is also available (the
older, dict-based function is no more). To use this functionality, you'll need to import
logging.config. Here is an example of such a config file - it's a bit long, but a
full working example so bear with me. I've annotated it as best I can.
In the listing below, some values are used by both the logging configuration API and the GUI configurator, while others are used only by the GUI configurator. To make it clearer which values you absolutely need to have in the .ini file for it to be useful even if you hand-code it, the values used by the configuration API are shown like this. (The other ones are used by the GUI configurator, but ignored by the configuration API.)
# --- logconf.ini ----------------------------------------------------------- #The "loggers" section contains the key names for all the loggers in this #configuration. These are not the actual channel names, but values used to #identify where the parameters for each logger are found in this file. #The section for an individual logger is named "logger_xxx" where the "key" #for a logger is "xxx". So ... "logger_root", "logger_log02", etc. further #down the file, indicate how the root logger is set up, logger "log_02" is set #up, and so on. #Logger key names can be any identifier, except "root" which is reserved for #the root logger. (The names "lognn" are generated by the GUI configurator.) [loggers] keys=root,log02,log03,log04,log05,log06,log07 #The "handlers" section contains the key names for all the handlers in this #configuration. Just as for loggers above, the key names are values used to #identify where the parameters for each handler are found in this file. #The section for an individual handler is named "handler_xxx" where the "key" #for a handler is "xxx". So sections "handler_hand01", "handler_hand02", etc. #further down the file, indicate how the handlers "hand01", "hand02" etc. #are set up. #Handler key names can be any identifier. (The names "handnn" are generated #by the GUI configurator.) [handlers] keys=hand01,hand02,hand03,hand04,hand05,hand06,hand07,hand08,hand09 #The "formatters" section contains the key names for all the formatters in #this configuration. Just as for loggers and handlers above, the key names #are values used to identify where the parameters for each formatter are found #in this file. #The section for an individual formatter is named "formatter_xxx" where the #"key" for a formatter is "xxx". So sections "formatter_form01", #"formatter_form02", etc. further down the file indicate how the formatters #"form01", "form02" etc. are set up. #Formatter key names can be any identifier. (The names "formnn" are generated #by the GUI configurator.) [formatters] keys=form01,form02,form03,form04,form05,form06,form07,form08,form09 #The section below indicates the information relating to the root logger. # #The level value needs to be one of DEBUG, INFO, WARN, ERROR, CRITICAL or NOTSET. #In the root logger, NOTSET indicates that all messages will be logged. #Level values are eval()'d in the context of the logging package's namespace. # #The propagate value indicates whether or not parents of this loggers will #be traversed when looking for handlers. It doesn't really make sense in the #root logger - it's just there because a root logger is almost like any other #logger. # #The channel value indicates the lowest portion of the channel name of the #logger. For a logger called "a.b.c", this value would be "c". # #The parent value indicates the key name of the parent logger, except that #root is shown as "(root)" rather than "root". # #The qualname value is the fully qualified channel name of the logger. For a #logger called "a.b.c", this value would be "a.b.c". # #The handlers value is a comma-separated list of the key names of the handlers #attached to this logger. # [logger_root] level=NOTSET handlers=hand01 qualname=(root) # note - this is used in non-root loggers propagate=1 # note - this is used in non-root loggers channel= parent= # #The explanation for the values in this section is analogous to the above. The #logger is named "log02" and coincidentally has a key name of "log02". It has #a level of DEBUG and handler with key name "hand02". (See section #"handler_hand02" for handler details.) If the level value were NOTSET, this tells #the logging package to consult the parent (as long as propagate is 1) for the #effective level of this logger. If propagate is 0, this level is treated as for #the root logger - a value of NOTSET means "pass everything", and other values are #interpreted at face value. # [logger_log02] level=DEBUG propagate=1 qualname=log02 handlers=hand02 channel=log02 parent=(root) # #The explanation for the values in this section is analogous to the above. The #logger is named "log02.log03" and has a key name of "log03". #It has a level of INFO and handler with key name "hand03". # [logger_log03] level=INFO propagate=1 qualname=log02.log03 handlers=hand03 channel=log03 parent=log02 # #The explanations for the values in this section and subsequent logger sections #are analogous to the above. # [logger_log04] level=WARN propagate=0 qualname=log02.log03.log04 handlers=hand04 channel=log04 parent=log03 [logger_log05] level=ERROR propagate=1 qualname=log02.log03.log04.log05 handlers=hand05 channel=log05 parent=log04 [logger_log06] level=CRITICAL propagate=1 qualname=log02.log03.log04.log05.log06 handlers=hand06 channel=log06 parent=log05 [logger_log07] level=WARN propagate=1 qualname=log02.log03.log04.log05.log06.log07 handlers=hand07 channel=log07 parent=log06 #The section below indicates the information relating to handler "hand01". #The first three keys (class, level and formatter) are common to all handlers. #Any other values are handler-specific, except that "args", when eval()'ed, #is the list of arguments to the constructor for the handler class. # #The class value indicates the handler's class (as determined by eval() in #the logging package's namespace). # #The level value needs to be one of DEBUG, INFO, WARN, ERROR, CRITICAL or NOTSET. #NOTSET means "use the parent's level". # #The formatter value indicates the key name of the formatter for this handler. #If blank, a default formatter (logging._defaultFormatter) is used. # #The stream value indicates the stream for this StreamHandler. It is computed #by doing eval() on the string value in the context of the logging package's #namespace. # #The args value is a tuple of arguments which is passed to the constructor for #this handler's class in addition to the "self" argument. # [handler_hand01] class=StreamHandler level=NOTSET formatter=form01 args=(sys.stdout,) stream=sys.stdout #The section below indicates the information relating to handler "hand02". #The first three keys are common to all handlers. #Any other values are handler-specific, except that "args", when eval()'ed, #is the list of arguments to the constructor for the handler class. # #The filename value is the name of the file to write logging information to. #The mode value is the mode used to open() the file. The maxsize and backcount #values control rollover as described in the package's API documentation. # [handler_hand02] class=FileHandler level=DEBUG formatter=form02 args=('python.log', 'w') filename=python.log mode=w #The section below indicates the information relating to handler "hand03". #The first three keys are common to all handlers. #Any other values are handler-specific, except that "args", when eval()'ed, #is the list of arguments to the constructor for the handler class. # #The host value is the name of the host to send logging information to. #The port value is the port number to use for the socket connection. # [handler_hand03] class=handlers.SocketHandler level=INFO formatter=form03 args=('localhost', handlers.DEFAULT_TCP_LOGGING_PORT) host=localhost port=DEFAULT_TCP_LOGGING_PORT #The section below indicates the information relating to handler "hand04". #The first three keys are common to all handlers. #Any other values are handler-specific, except that "args", when eval()'ed, #is the list of arguments to the constructor for the handler class. # #The host value is the name of the host to send logging information to. #The port value is the port number to use for the socket connection. # [handler_hand04] class=handlers.DatagramHandler level=WARN formatter=form04 args=('localhost', handlers.DEFAULT_UDP_LOGGING_PORT) host=localhost port=DEFAULT_UDP_LOGGING_PORT #The section below indicates the information relating to handler "hand05". #The first three keys are common to all handlers. #Any other values are handler-specific, except that "args", when eval()'ed, #is the list of arguments to the constructor for the handler class. # #The host value is the name of the host to send logging information to. #The port value is the port number to use for the socket connection. #The facility is the syslog facility to use for logging. # [handler_hand05] class=handlers.SysLogHandler level=ERROR formatter=form05 args=(('localhost', handlers.SYSLOG_UDP_PORT), handlers.SysLogHandler.LOG_USER) host=localhost port=SYSLOG_UDP_PORT facility=LOG_USER #The section below indicates the information relating to handler "hand06". #The first three keys are common to all handlers. #Any other values are handler-specific, except that "args", when eval()'ed, #is the list of arguments to the constructor for the handler class. # #The appname value is the name of the application which appears in the #NT event log. #The dllname value is the pathname of a DLL to use for message definitions. #The logtype is the type of NT event log to write to - Application, Security #or System. # [handler_hand06] class=NTEventLogHandler level=CRITICAL formatter=form06 args=('Python Application', '', 'Application') appname=Python Application dllname= logtype=Application #The section below indicates the information relating to handler "hand07". #The first three keys are common to all handlers. #Any other values are handler-specific, except that "args", when eval()'ed, #is the list of arguments to the constructor for the handler class. # #The host value is the name of the SMTP server to connect to. #The port value is the port number to use for the SMTP connection. #The from value is the "From" value in emails. #The to value is a comma-separated list of email addresses. #The subject value is the subject of the email. # [handler_hand07] class=SMTPHandler level=WARN formatter=form07 args=('localhost', 'from@abc', ['user1@abc', 'user2@xyz'], 'Logger Subject') host=localhost port=25 from=from@abc to=user1@abc,user2@xyz subject=Logger Subject #The section below indicates the information relating to handler "hand08". #The first three keys are common to all handlers. #Any other values are handler-specific, except that "args", when eval()'ed, #is the list of arguments to the constructor for the handler class. # #The capacity value is the size of this handler's buffer. #The flushlevel value is the logging level at which the buffer is flushed. #The from value is the "From" value in emails. #The target value is the key name of the handler which messages are flushed #to (i.e. sent to when flushing). # [handler_hand08] class=MemoryHandler level=NOTSET formatter=form08 target= args=(10, ERROR) capacity=10 flushlevel=ERROR #The section below indicates the information relating to handler "hand09". #The first three keys are common to all handlers. #Any other values are handler-specific, except that "args", when eval()'ed, #is the list of arguments to the constructor for the handler class. # #The host value is the name of the HTTP server to connect to. #The port value is the port number to use for the HTTP connection. #The url value is the url to request from the server. #The method value is the HTTP request type (GET or POST). # [handler_hand09] class=HTTPHandler level=NOTSET formatter=form09 args=('localhost:9022', '/log', 'GET') host=localhost port=9022 url=/log method=GET #The sections below indicate the information relating to the various #formatters. The format value is the overall format string, and the #datefmt value is the strftime-compatible date/time format string. If #empty, the logging package substitutes ISO8601 format date/times where #needed. See the package API documentation for more details #of the format string structure. # [formatter_form01] format=F1 %(asctime)s %(levelname)s %(message)s datefmt= [formatter_form02] format=F2 %(asctime)s %(pathname)s(%(lineno)d): %(levelname)s %(message)s datefmt= [formatter_form03] format=F3 %(asctime)s %(levelname)s %(message)s datefmt= [formatter_form04] format=%(asctime)s %(levelname)s %(message)s datefmt= [formatter_form05] format=F5 %(asctime)s %(levelname)s %(message)s datefmt= [formatter_form06] format=F6 %(asctime)s %(levelname)s %(message)s datefmt= [formatter_form07] format=F7 %(asctime)s %(levelname)s %(message)s datefmt= [formatter_form08] format=F8 %(asctime)s %(levelname)s %(message)s datefmt= [formatter_form09] format=F9 %(asctime)s %(levelname)s %(message)s datefmt= # --- end of logconf.ini ----------------------------------------------------
To use a file like this, you would call logging.config.fileConfig("logconf.ini")
whereupon the file is read in and processed. Note that evaluation happens in the context
of the logging package (hence the unqualified class names).
To create a file like the above, you can use the GUI configurator, logconf.py,
which is invoked either with no arguments or with a single argument giving the name of a
configuration file to read. (Or, if you're a masochist/don't have Tk, you can do it by
hand. The configurator is a quick hack, which I hope is reasonably intuitive - have a play
with it and see what you think. I've used it with 1.5.2 and 2.1.2 on Windows and 1.5.2 on
Linux/x86. There's no validation, rudimentary error checking and the usability could be
better, but it's something to build on, hey?)
Please note that the configurator is optional - you don't need to use either the GUI configurator or even file-based configuration. All configuration can be done by calling logging API methods in your code which create loggers, handlers, and formatters, and associate them with each other.
Here's a screenshot of the configurator:
Here's a quick guide on how to use it:
With reference to the PEP, here are my comments on the current state of play.
app.py
above.The package is intended to be threadsafe. Although it makes no use of threads to provide its functionality (except for on-the-fly reconfiguration), shared data in the package is protected by a thread lock which is acquired before, and released after, modifications to shared data. In addition, the Handler class creates a per-handler instance I/O lock which is acquired before, and released after, calling emit(). If you define your own handlers, in most situations you should not need to take any special precautions, as long as your I/O is called only from emit(). The thread locks impose a slight performance penalty, but it's no worse than for any other use of thread locks in Python applications.
The package also allows a program to permit changing of the logging configuration on
the fly, i.e. while the program is still running. This should be a help for
developers of long-running programs such as servers (e.g. Zope, Webware). At this stage,
the on-the-fly configurability is fairly basic - to use it, two new module-level functions
are provided (in the logging.config module).
listen([port]) is used to create a thread which, when started, opens a
socket server which listens on the specified port for configuration requests. The socket
protocol is very basic - a two-byte length followed by a string of that length is sent to
the listener. The string should be in the same format as logging configuration files, i.e.
ConfigParser files conforming to the scheme described above.stopListening() tells the thread created by listening to terminate.Currently, you can't just change part of the logging configuration - the sent
configuration completely replaces the existing configuration, and if previously existing
loggers are not in the new configurations, they will be disabled after the new
configuration takes effect. The script log_test17.py in the distribution
illustrates the on-the-fly configuration feature.
For the casual user, there are module-level convenience functions which operate on the root logger. See the API documentation for details.
The implementation has not been optimized for performance. This is planned to be done in a later phase, following feature stabilization and benchmarking.
The implementation is now part of Python 2.3. This independent distribution is intended for use by people with older versions of Python. It also contains test scripts and examples of additional handlers and filters.
The biggest thank you goes to the log4j developers, whom I am attempting to flatter sincerely by imitation ;-) Thanks also to Trent Mick for PEP 282, which prompted me to offer this implementation.
I'd also like to thank all of the people who have given me feedback, patches and encouragement. In particular (but in no particular order):
No rest for the wicked...
If you can help with any of this, please email me.
The current version is 0.4.9.6. Here is the latest tarball (also in zip format or Windows executable - the latter includes the logging package only). The distribution contains the following files:
| Filename | Contents | |
|---|---|---|
README.txt |
Brief description and change history. | |
__init__.py |
The core logging package itself, including StreamHandler and FileHandler. | |
handlers.py |
The other handlers provided as part of the package. | |
config.py |
The code for configuring the package. | |
setup.py |
The distutils setup script. | |
logrecv.py |
A test server used for testing SocketHandler, DatagramHandler, HTTPHandler and
SOAPHandler. Run it with an argument of one of "TCP", "UDP",
"HTTP" or "SOAP" before running a test harness which logs to one of
these handlers. Note that to use the SOAP handler, you need to have installed PyXML-0.6.6 and the Zolera Soap Infrastructure (ZSI). This is
needed for logrecv.py only, and not for the logging module
itself. (Note that ZSI requires Python 2.x) |
|
app.py |
The minimal example described above. | |
mymodule.py |
Another example described above. | |
myapp.py |
From the second example described above. | |
log_test.py |
A test script intended to work as a regression test harness. Runs a number of the other scripts and generates output to stdout.log and stderr.log. | |
log_test0.py |
A simple test script using basicConfig() only. |
|
log_test1.py |
An example showing slightly more involved configuration and exception handling, as well as a Unix syslog handler which uses the standard library's syslog module. | |
log_test2.py |
A version of log_test0.py which only logs to a SocketHandler. |
|
log_test3.py |
An example showing use of fileConfig() and logging to various loggers. |
|
log_test4.py |
An example showing use of bespoke levels, filtering by level at logger and handler,
and use of filter classes (descendants of Filter). |
|
log_test5.py |
An example showing use of SMTPHandler. Before running this script, be
sure to change the bogus addresses it contains to real ones which you have access to. |
|
log_test6.py |
An example showing use of NTEventLogHandler. This script needs to be run
on an NT system. |
|
log_test7.py |
An example showing use of MemoryHandler. |
|
log_test8.py |
An example showing use of FileHandler with rollover across multiple
files. |
|
log_test9.py |
An example showing use of BufferingHandler and BufferingFormatter
through implementing simple XMLFormatter and XMLHandler classes. |
|
log_test10.py |
An example showing how to get the logging module to create loggers of your own class
(though it needs to be a subclass of Logger). |
|
log_test11.py |
An example SMTP handler, called BufferingSMTPHandler, which buffers
events and sends them via email in batches. |
|
log_test12.py |
An example showing the use of HTTPHandler, for use with logrecv.py. |
|
log_test13.py |
An example showing the use of SOAPHandler, for use with logrecv.py. |
|
log_test14.py |
An example showing an implementation of DBHandler, showing how to log
requests to RDBMS tables using the Python Database API 2.0. |
|
log_test15.py |
An example showing the use of the Filter class with a string initializer. |
|
log_test16.py |
An example showing the use of logging in a multi-threaded program. | |
log_test17.py |
An example showing the use of logging in a multi-threaded program, together with
reconfiguring logging on the fly through the use of listen() and stopListening().
This script serves as both server and client, depending on the arguments it's called with. |
|
log_test18.py |
An example showing the use of an example filter, MatchFilter, which offers flexible match-based filtering of LogRecords. | |
log_test19.py |
A basic test of logger parents. | |
log_test20.py |
Demonstrates the use of custom class instances for messages and filtering based on classes. | |
log_test21.py |
Demonstrates the use of a wildcard name-space filter with and without custom message classes. | |
log_test22.py |
Demonstrates the use of either localtime or gmtime to do date/time formatting. | |
log_test23.py |
Demonstrates the use of basicConfig() with various variants. | |
log_test24.py |
Demonstrates the use of TimedRotatingFileHandler. | |
debug.ini |
An example configuration for use with log_test17.py. | |
warn.ini |
An example configuration for use with log_test17.py. | |
error.ini |
An example configuration for use with log_test17.py. | |
critical.ini |
An example configuration for use with log_test17.py. | |
log_test3.ini |
An example configuration for use with log_test3.py. | |
stdout.exp |
The expected results of stdout.log after running log_test.py. | |
stderr.exp |
The expected results of stderr.log after running log_test.py. | |
logconf.py |
A Tkinter-based GUI configurator. | |
logconf.ini |
Example configuration file, in ConfigParser format, for use with logconf.py
and log_test3.py. |
|
logging.dtd |
A simple example DTD for use with log_test9.py. |
|
logging.xml |
An example XML file for use with log_test9.py. It references events.xml
as external data. |
|
events.xml |
An example XML file for use with log_test9.py. It holds the actual events
in XML format. |
|
python_logging.html |
The page you're reading now. | |
default.css |
Stylesheet for use with the HTML pages. | |
To install, unpack the archive into any directory, and in that directory invoke the
script "setup.py install" to install the module in the default
location used by distutils.
To use, just put logging.py in your Python path, "import
logging" and go. (The installation procedure described above will normally put
the logging module in your Python path. If you want to use file-based configuration API,
you'll also need to import logging.config. To use the more esoteric handlers,
you'll also need to import logging.handlers.)
The change history is as follows.
Version Date Description
=============================================================================
0.4.9.6 02 Mar 2005 Improved error recovery for SysLogHandler
(thanks to Erik Forsberg for the patch).
Added optional encoding for file handlers, which
defaults to None. If specified, codecs.open() is used
for opening files.
-----------------------------------------------------------------------------
0.4.9.5 02 Oct 2004 Removed 1.5.2 incompatibility involving *=, +=
-----------------------------------------------------------------------------
0.4.9.4 22 Sep 2004 Added getLoggerClass() (thanks to Dave Wilson).
Added exception handling in shutdown().
Sort globbed files in doRollover() in
TimedRotatingFileHandler.
Date formatting for SMTPHandler now uses email package
where available.
fileConfig() exception handling added for handler
instantiation.
Minor documentation corrections.
-----------------------------------------------------------------------------
0.4.9.3 24 Aug 2004 Changed basicConfig() to add keyword arguments. Changes
are backward-compatible.
Refactored RotatingFileHandler to create a base class
for rotating handlers. Added TimedRotatingFileHandler
(thanks to Mark Davidson - minor changes have been made
to the patch he supplied).
Added error checking to log() to check that level is
an integer, and raise a TypeError if not (as long as
raiseExceptions is set).
Fixed a bug in DatagramHandler.send() (thanks to Mario
Aleppo and Enrico Sirola for pointing it out).
Minor documentation corrections.
-----------------------------------------------------------------------------
0.4.9.2 28 Feb 2004 Traceback text is now cached.
Tracebacks can be propagated across sockets as text.
Added makeLogRecord() to allow a LogRecord to be
created from a dictionary.
Closing a handler now removes it from the internal list
used by shutdown().
Made close() call flush() for handlers where this makes
sense (thanks to Jim Jewett).
The exc_info keyword parameter can be used to pass an
exception tuple as well as a flag indicating that the
current exception should be logged.
A shutdown hook is registered to call shutdown() on
application (Python) exit (Thanks to Jim Jewett).
Removed redundant error check in setLoggerClass().
Added RESET_ERROR to logging.config.
SocketHandler now uses an exponential backoff strategy
(thanks to Robert Olson).
Minor documentation corrections.
-----------------------------------------------------------------------------
0.4.8 22 Apr 2003 Made _listener global in stopListening().
Made listen() correctly pass the specified port.
Removed some redundant imports in __init__.py.
Added the record being processed as a parameter to
handleError (thanks to Gordon den Otter for the idea).
Handler.handle returns the result of applying the
filter to the record (thanks to Gordon den Otter for
the idea).
Added a seek(0, 2) in RotatingFileHandler before the
tell() call. This is because under Windows, tell()
returns 0 until the first actual write (thanks to
Gordon den Otter for the patch).
Altered findCaller to not use inspect (thanks to
Jeremy Hylton for the patch).
Renamed warn and WARN to warning and WARNING. This may
break existing code, but the standard Python module
will use warning/WARNING rather than warn/WARN. The
fatal and FATAL synonyms for critical and CRITICAL
have also been removed.
Added defaultEncoding and some support for encoding
Unicode messages (thanks to Stéphane Bidoul for the
suggestion).
Added process ID to the list of LogRecord attributes.
Modified Logger.removeHandler so that it does not
close the handler on removal.
Modified SMTPHandler to treat a single "to address"
correctly (thanks to Anthony Baxter).
Modified SMTPHandler to add a date header to the SMTP
message (thanks to David Driver for the suggestion).
Modified HTTPHandler to factor out the mapping of
a LogRecord to a dictionary (thanks to Franz Glasner
for the patch).
-----------------------------------------------------------------------------
0.4.7 15 Nov 2002 Made into a package with three modules: __init__ (the
core code), handlers (all handlers other than
FileHandler and its bases) and config (all the config
stuff). Before doing this:
Updated docstrings to include a short line, then a
blank line, then more descriptive text.
Renamed 'lvl' to 'level' in various functions.
Changed FileHandler to use "a" and "w" instead of "a+"
and "w+".
Moved log file rotation functionality from FileHandler
to a new class RotatingFileHandler.
Improved docstring describing rollover.
Updated makePickle to use 4-byte length and struct
module, likewise logrecv.py. Also updated on-the-fly
config reader to use 4-byte length/struct module.
Altered ConfigParser test to look at 'readline' rather
than 'read'.
Added optional "defaults" argument to fileConfig, to
be passed to ConfigParser.
Renamed ALL to NOTSET to avoid confusion.
Commented out getRootLogger(), as obsolete.
To do regression testing, run log_test.py and compare
the created files stdout.log and stderr.log against
the files stdout.exp and stderr.exp. They should match
except fir a couple of exception messages which give
absolute file paths.
Updated python_logging.html to remove links to
logging_pydoc.html, which has been removed from the
distribution.
Changed default for raiseExceptions to 1.
-----------------------------------------------------------------------------
0.4.6 08 Jul 2002 Added raiseExceptions to allow conditional propagation
of exceptions which occur during handling.
Added converter to Formatter to allow use of any
function to convert time from seconds to a tuple. It
still defaults to time.localtime but now you can also
use time.gmtime.
Added log_test22.py to test the conversion feature.
Changed rootlogger default level to WARN - was DEBUG.
Updated some docstrings.
Moved import of threading to where thread is imported.
If either is unavailable, threading support is off.
Updated minor defects in python_logging.html.
Check to see if ConfigParser has readfp method; if it
does and an object with a 'read' method is passed in,
assumes a file-like object and uses readfp to read it
in.
-----------------------------------------------------------------------------
0.4.5 04 Jun 2002 Fixed bug which caused problem if no args to message
(suggested by Hye-Shik Chang).
Fixed bug in _fixupParents (thanks to Nicholas Veeser)
and added log_test19.py as a test case for this bug.
Added getMessage to LogRecord (code was moved here from
Formatter.format)
Applied str() to record.msg to allow arbitrary classes
to determine the formatting (as msg can now be a class
instance).
Table of Contents added to python_logging.html, the
section on Loggers updated, and the logconf.ini file
section annotated.
Added log_test20.py which demonstrates how to use
class instances to provide alternatives to numeric
severities as mechanisms for control of logging.
Added log_test21.py which builds on log_test20.py to
show how you can use a regular expression-based Filter
for flexible matching similar to e.g. Protomatter
Syslog, where you can filter on e.g. "a.*" or "*.b" or
"a.*.c".
_levelNames changed to contain reverse mappings as well
as forward mappings (leveltext->level as well as level
-> leveltext). The reverse mappings are used by
fileConfig().
fileConfig() now more forgiving of missing options in
.ini file - sensible defaults now used when some
options are absent. Also, eval() is used less when
interpreting .ini file contents - int() and dict lookup
are used in more places. Altered log_test3.py and added
log_test3.ini to show a hand-coded configuration file.
-----------------------------------------------------------------------------
0.4.4 02 May 2002 getEffectiveLevel() returns ALL instead of None when
nothing found. Modified references to level=0 to
level=ALL in a couple of places.
SocketHandler now inherits from Handler (it used to
inherit from StreamHandler, for no good reason).
getLock() renamed to createLock().
Docstring tidy-ups, and some tidying up of
DatagramHandler.
Factored out unpickling in logrecv.py.
Added log_test18.py to illustrate MatchFilter, which is
a general matching filter.
Improved FileHandler.doRollover() so that the base
file name is always the most recent, then .1, then .2
etc. up to the maximum backup count. Renamed formal
args and attributes used in rollover.
Changed LogRecord attributes lvl -> levelno, level ->
levelname (less ambiguity)
Formatter.format searches for "%(asctime)" rather than
"(asctime)"
Renamed _start_time to _startTime
Formatter.formatTime now returns the time
Altered logrecv.py to support stopping servers
programmatically
Added log_test.py as overall test harness
basicConfig() can now be safely called more than once
Modified test scripts to make it easier to call them
from log_test.py
Moved SOAPHandler from core to log_test13.py. It's not
general enough to be in the core; most production use
will have differing RPC signatures.
-----------------------------------------------------------------------------
0.4.3 14 Apr 2002 Bug fix one-off error message to go to sys.stderr
rather than sys.stdout.
logrecv.py fix TCP for busy network.
Thread safety - added locking to Handler and for shared
data in module, and log_test16.py to test it.
Added socket listener to allow on-the-fly configuration
and added log_test17.py to test it.
-----------------------------------------------------------------------------
0.4.2 11 Apr 2002 Bug fix fileConfig() - setup of MemoryHandler target
and errors when loggers have no handlers set or
handlers have no formatters set
logconf.py - seems to hang if window closed when combo
dropdown is showing - added code to close popup on exit
Some tweaks to _srcfile computation (normpath added)
findCaller() optimized, now a lot faster!
Logger.removeHandler now closes the handler before
removing it
fileConfig() removes existing handlers before adding
the new set, to avoid memory leakage when repeated
calls are made
Fixed logrecv.py bug which hogged CPU time when TCP
connection was closed from the client
Added log_test14.py to demonstrate/test a DBHandler
which writes logging records into an RDBMS using the
Python Database API 2.0 (to run, you need something
which supports this already installed - I tested with
mxODBC)
Made getLogger name argument optional - returns root
logger if omitted
Altered Filter to take a string initializer, filtering
a sub-hierarchy rooted at a particular point (idea from
Denis S. Otkidach).
Added log_test15.py to test Filter initializer
-----------------------------------------------------------------------------
0.4.1 03 Apr 2002 Bug fix SMTPHandler - extra \r\n needed (Oleg Orlov)
Added BufferingHandler, BufferingFormatter
Renamed getChainedPriority to getEffectiveLevel
Removed Logger.getRoot as it is redundant
Added log_test9.py to test Buffering classes and
to show an XMLFormatter example.
Added setLoggerClass.
Added log_test10.py to test setLoggerClass, using an
example Logger-derived class which outputs exception
info even for DEBUG level logging calls
Added log_test11.py to test a buffering implementation
of SMTPHandler
Changed logging call implementation to allow keyword
arguments (Kevin Butler and others)
Changed default SysLogHandler implementation.
Renamed "additive" to "propagate" as it better
describes the attribute.
Added HTTPHandler.
Modified logrecv.py to remove "both" option and to add
"HTTP" and "SOAP" options (SOAP option needs you to
have PyXML-0.6.6 and ZSI installed - for logrecv.py
only, and not for the core logging module itself).
Added log_test12.py to test HTTPHandler.
Added log_test13.py to test SOAPHandler.
Formatted to Python source guidelines (spaces, indent
of 4, within 80 columns).
More method renamings (result of feedback) - _handle()
renamed to emit(), _logRecord() renamed to handle().
Renamed FATAL to CRITICAL (David Goodger), but left
fatal() and FATAL in (until PEP is changed)
Changed configuration file format to ConfigParser
format.
Factored filter application functionality out to a new
Filterer class. The isLoggable() method is renamed to
filter() in both Filter and Filterer classes.
Altered SMTPHandler __init__ to accept (host, port)
for the mail internet address.
Added GUI configurator which uses Tkinter and the new
configuration file format. (See logconf.py and an
example configuration file in logconf.ini)
Altered log_test3.py to test with the new file format.
-----------------------------------------------------------------------------
0.4 21 Mar 2002 Incorporated comments/patches from Ollie Rutherfurd:
-Added level filtering for handlers.
-Return root logger if no name specified in getLogger.
Incorporated comments from Greg Ward:
-Added distutils setup.py script.
Added formatter initialization in Handler.__init__.
Tidied up docstrings.
Added removeHandler to Logger.
Added removeFilter to Logger and Handler.
logrecv.py modified to keep connection alive until
client closes it.
SocketHandler modified to not reset connection after
each logging event.
Added shutdown function which closes open sockets
Renamed DEFAULT_LOGGING_PORT->DEFAULT_TCP_LOGGING_PORT
Added DEFAULT_UDP_LOGGING_PORT
Added log_test4.py (example of arbitrary levels)
Added addLevelName, changed behaviour of getLevelName
Fixed bugs in DatagramHandler
Added SMTPHandler implementation
Added log_test5.py to test SMTPHandler
Added SysLogHandler (contribution from Nicolas Untz
based on Sam Rushing's syslog.py)
Modified log_test1.py to add a SysLogHandler
Added rollover functionality to FileHandler
Added NTEventLogHandler (based on Win32 extensions)
Added MemoryHandler implementation
Added log_test7.py to test MemoryHandler
Added log_test8.py to test FileHandler rollover
Added logException method to Logger
Added formatException method to Formatter
Added log_test6.py to test NTEventHandler and
logException
Numerous internal method renamings (sorry - but better
to do this now, rather than when we enter beta status).
-----------------------------------------------------------------------------
0.3 14 Mar 2002 First public release, for early feedback
-----------------------------------------------------------------------------
0.2 Consolidated into single file (for internal use only)
-----------------------------------------------------------------------------
0.1 Initial implementation (for internal use only)
-----------------------------------------------------------------------------
The copyright statement follows.
Copyright 2001-2010 by Vinay Sajip. All Rights Reserved. Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation, and that the name of Vinay Sajip not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission. VINAY SAJIP DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL VINAY SAJIP BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
