Python标准logging模块中主要分为四大块内容：
Logger: 定义应用程序使用的接口
Handler: 将Loggers产生的日志输出到目的地
Filter: 对Loggers产生的日志进行过滤
Formatter: 格式化Loggers产生的日志
其中常用的是Logger, Handler, Formatter。
目标
对logging模块进行封装，使其尽量达到几个目标：
- 易用性：尽量保持python原生标准的用法。
- 扩展性：避免将所有模块集中在一个文件中进行配置，而是对不同模块继承扩展。如此可以确保修改时尽量保持简洁性，而非混淆杂乱。
- 文字加色：同一时间段内出现大量日志输出时，可以更清晰地识别出问题所在。
- 输出多个目的地：在生产代码中，并非所有日志信息都输出到console或者file中，而是日志的重要级别，将那些比较重要的发送到console，将那些不太重要但以后分析问题时可能又会用到的发送到file中。
- 非阻塞发送器：记录日志时如果涉及到网络（如socket, smtp），那么这个记录过程可能会阻塞当前线程，在python 3.2+，我们可以非常容易地设计一个非阻塞的发送器（当然了，非阻塞机制的内部实现大多都是队列）。
目录结构
如下的定制日志主要包括两个包，log_config和tools，目录结构如下：
log_config 模块结构
默认配置
- Logging Level:
logging.NOTSET is default.
You can change it by passing ‘level=’ to BTLogger instance in your application code.
- Sending Level:
Send logging messages to console with levels of logging.INFO and higher, and simultaneously to the disk file with levels of logging.DEBUG and higher.
You can change them by passing ‘level=’ to BTStreamHandler/BTFileHandler instance in log_config\__init__.py.
- Log Config:
LOG_PATH = r'C:log'
LOG_NAME = r'mylogger.log'
You can change them in log_config\__init__.py.
- Console Text Color:
DEBUG
INFO
WARNING
ERROR
CRITICAL
You can change them by passing ‘fore_color=’ and ‘back_color=’ to decorator @text_attribute.set_textcolor(fore_color, back_color) in log_config\__init__.py.
- There also exists a NonBlockingHandler class in log_confighandlers.py, which will use features of version 3.2+. If you unfold it, you can get a non_blocking_handler.
Demo
ts_log_config.py:
1 def log_test(): 2 import log_config 3 4 logger = log_config.BTLogger('mylogger') 5 6 logger.debug('Debug message') 7 logger.info('Info message') 8 logger.warning('Warning message') 9 logger.error('Error message') 10 logger.critical('Critical message') 11 12 def main(): 13 log_test() 14 15 if __name__ == '__main__': 16 main()
outputs:
代码实现
..log_config\__init__.py:
1 """ 2 Wrap the classes of logging module so as to expose more flexible interfaces that application code directly uses. 3 4 By default, when you do some log operations, it will log messages to console with levels of logging.INFO and higher, 5 and simultaneously to the disk file with levels of logging.DEBUG and higher. 6 7 Usages: 8 To use log_config module, simply 'import log_config' in your code, and create a BTLogger instance, then, you can use this 9 instance to log messages: 10 import log_config 11 12 logger = log_config.BTLogger('mylogger') 13 14 logger.debug('Debug message') 15 logger.info('Info message') 16 logger.warning('Warning message') 17 logger.error('Error message') 18 logger.critical('Critical message') 19 """ 20 21 import os 22 import sys 23 import logging 24 from log_config import formatters 25 from log_config import handlers 26 from log_config import text_attribute 27 from tools import os_tools 28 29 __author__ = " " 30 __status__ = "debugging" # {debugging, production} 31 __version__ = "0.1.0" # major_version_number.minor_version_number.revision_number 32 __date__ = " " 33 34 """ 35 # Config root logger 36 basicConfig( 37 filename=__name__, 38 filemode='w', 39 format='%(asctime)s: %(levelname)s: %(message)s', 40 datefmt='%Y-%m-%d %H:%M:%S', 41 level='logging.NOTSET' 42 ) 43 """ 44 45 # Config log path and file name. 46 LOG_PATH = r'C:log' 47 LOG_NAME = r'mylogger.log' 48 filename = 'default.log' if not os_tools.makedir(LOG_PATH) else os.path.abspath(os.path.join(LOG_PATH, LOG_NAME)) 49 50 51 class BTLogger(logging.Logger): 52 def __init__(self, name=None, level=logging.NOTSET): 53 """ 54 Initialize the BTLogger with basic settings. 55 56 :param 57 name: Specify the logger's name. If name is None, default is root logger. 58 Note: 59 All BTLogger instances with a given name return the same logger 60 instance. This means that logger instances never need to be passed 61 between different parts of an application. 62 level: The threshold of logging records, messages which are less severe 63 than level will be ignored. 64 If the level is set to NOTSET, than: 65 1> If current logger is root logger, than all messages will 66 be logged. 67 2> Or, delegate to its parent logger, until an ancestor with 68 a level other than NOTSET is found, or the root is reached. 69 Note: 70 The root logger is created with level logging.WARNING. 71 :return: 72 None 73 """ 74 logging.Logger.__init__(self, name, level) 75 76 # Create handlers 77 # non_blocking_ch = handlers.NonBlockingHandler(handlers.BTStreamHandler(sys.stdout, logging.DEBUG)).get_handler() # version 3.2+ 78 console_handler = handlers.BTStreamHandler(stream=sys.stdout, level=logging.INFO) 79 file_handler = handlers.BTFileHandler(filename=filename, level=logging.DEBUG, mode='w') 80 81 # Set formatters to handlers. 82 # non_blocking_ch.setFormatter(formatters.BTStreamFormatter()) # version 3.2+ 83 console_handler.setFormatter(formatters.BTStreamFormatter()) 84 file_handler.setFormatter(formatters.BTFileFormatter()) 85 86 # Add the handlers to logger. 87 # self.addHandler(non_blocking_ch) # version 3.2+ 88 self.addHandler(console_handler) 89 self.addHandler(file_handler) 90 91 # Override methods in logging.Logger(debug, info, warning, error, critical). 92 # @handlers.NonBlockingHandler.non_blocking # version 3.2+ 93 @text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_WHITE') 94 def debug(self, msg, *args, **kwargs): 95 self.log(logging.DEBUG, msg, *args, **kwargs) 96 97 @text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_GREEN') 98 def info(self, msg, *args, **kwargs): 99 self.log(logging.INFO, msg, *args, **kwargs) 100 101 @text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_YELLOW') 102 def warning(self, msg, *args, **kwargs): 103 self.log(logging.WARNING, msg, *args, **kwargs) 104 105 warn = warning 106 107 @text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_RED') 108 def error(self, msg, *args, **kwargs): 109 self.log(logging.ERROR, msg, *args, **kwargs) 110 111 @text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_RED') 112 def critical(self, msg, *args, **kwargs): 113 self.log(logging.CRITICAL, msg, *args, **kwargs) 114 115 fatal = critical
..log_configformatters.py:
1 """ 2 Formatters specify the output format of logging messages. 3 """ 4 5 from logging import Formatter 6 7 8 # Changed in version 3.2+: The 'style' parameter was added. 9 ''' 10 class BTStreamFormatter(Formatter): 11 def __init__(self, 12 fmt='[%(asctime)s] [%(levelname)s]: %(message)s', 13 datefmt='%Y-%m-%d %H:%M:%S', style='%'): 14 Formatter.__init__(self, fmt, datefmt, style) 15 16 17 class BTFileFormatter(Formatter): 18 def __init__(self, 19 fmt='%(asctime)s: %(levelname)s: %(message)s', 20 datefmt='%Y/%m/%d %H:%M:%S', style='%'): 21 Formatter.__init__(self, fmt, datefmt, style) 22 ''' 23 24 # Be suitable for version 3.2-. 25 class BTStreamFormatter(Formatter): 26 def __init__(self, 27 fmt='[%(asctime)s] [%(levelname)s]: %(message)s', 28 datefmt='%Y-%m-%d %H:%M:%S'): 29 Formatter.__init__(self, fmt, datefmt) 30 31 32 class BTFileFormatter(Formatter): 33 def __init__(self, 34 fmt='%(asctime)s: %(levelname)s: %(message)s', 35 datefmt='%Y/%m/%d %H:%M:%S'): 36 Formatter.__init__(self, fmt, datefmt)
..log_confighandlers.py:
1 """ 2 Handlers send logging messages(created by BTLogger) to the specific destination. 3 """ 4 5 import sys 6 import logging 7 from logging import StreamHandler 8 from logging import FileHandler 9 10 11 class BTStreamHandler(StreamHandler): 12 """ 13 Write logging records to a stream(e.g., console). 14 15 Note that this class does not close the stream automatically, 16 as sys.stdout or sys.stderr may be used. 17 """ 18 def __init__(self, stream=sys.stderr, level=logging.INFO): 19 """ 20 Initialize the handler. 21 22 :param 23 stream: If stream is not specified, sys.stderr is used. 24 :return 25 None 26 """ 27 StreamHandler.__init__(self, stream) 28 self.setLevel(level) # The threshold of handling records, default is logging.INFO. 29 30 31 class BTFileHandler(FileHandler): 32 """ 33 Write logging records to disk files. 34 35 Inherited from logging.FileHandler, only modify the default mode('a') to ('w'). 36 """ 37 def __init__(self, 38 filename='default.log', 39 level=logging.DEBUG, # Handle messages with levels of logging.DEBUG and higher to file. 40 mode='w', 41 encoding=None, 42 delay=False): 43 FileHandler.__init__(self, filename, mode, encoding, delay) 44 self.setLevel(level) 45 46 47 # New in version 3.2+: QueueHandler, QueueListener. 48 # Unfold the following block, and you will get a non_blocking_handler. 49 ''' 50 from queue import Queue 51 from logging.handlers import QueueHandler 52 from logging.handlers import QueueListener 53 from functools import wraps 54 55 class NonBlockingHandler(object): 56 """ 57 Let logging handlers do their work without blocking the thread you're logging from. 58 Especially those network-based handler can block, e.g., SocketHandler, SMTPHandler. 59 """ 60 def __init__(self, handler_instance): 61 self.queue = Queue(-1) 62 self.queue_handler = QueueHandler(self.queue) 63 self.handler_instance = handler_instance 64 global _queue_listener 65 _queue_listener = QueueListener(self.queue, self.handler_instance) 66 67 def get_handler(self): 68 return self.handler_instance 69 70 @classmethod 71 def non_blocking(cls, func): 72 @wraps(func) 73 def wrapper(*args, **kwargs): 74 _queue_listener.start() 75 res = func(*args, **kwargs) 76 _queue_listener.stop() 77 return res 78 79 return wrapper 80 '''
..log_config ext_attribute.py:
1 """ 2 Set the text attributes in console. Currently, you can only set font color. 3 4 # ----------------------------------------- 5 # Author: 6 # Created: 7 # Modified: 8 # Version: 0.1.0 9 # ----------------------------------------- 10 11 Usages: 12 When you want to design a function which outputs messages to console with different colors, simply use '@text_attribute.set_textcolor(fore_color, back_color)' to decorate your function, e.g.: 13 @text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_RED') 14 def func(*args, **kwargs): 15 # output some messages to console 16 # ... 17 18 func(...) 19 """ 20 21 import ctypes 22 from functools import wraps 23 24 25 STD_INPUT_HANDLE = -10 26 STD_OUTPUT_HANDLE = -11 27 STD_ERROR_HANDLE = -12 28 29 30 _CONSOLE_TEXT_COLOR = { 31 'FOREGROUND_RED': 0x04, 32 'FOREGROUND_GREEN': 0x02, 33 'FOREGROUND_BLUE': 0x01, 34 # 'FOREGROUND_INTENSITY': 0x08, 35 'FOREGROUND_WHITE': 0x07, 36 'FOREGROUND_BLACK': 0x00, 37 'FOREGROUND_YELLOW': 0x06, 38 39 'BACKGROUND_RED': 0x40, 40 'BACKGROUND_GREEN': 0x20, 41 'BACKGROUND_BLUE': 0x10, 42 # 'BACKGROUND_INTENSITY': 0x80, 43 'BACKGROUND_WHITE': 0x70, 44 'BACKGROUND_BLACK': 0x00, 45 'BACKGROUND_YELLOW': 0x60 46 } 47 48 49 class TextColor(object): 50 """ 51 Set console text color. 52 """ 53 std_output_handle = ctypes.windll.kernel32.GetStdHandle(STD_OUTPUT_HANDLE) 54 55 @staticmethod 56 def text_color(fore_color, back_color): 57 fore_color = 'FOREGROUND_WHITE' if fore_color not in _CONSOLE_TEXT_COLOR else fore_color 58 back_color = 'BACKGROUND_BLACK' if back_color not in _CONSOLE_TEXT_COLOR else back_color 59 return _CONSOLE_TEXT_COLOR[fore_color] | _CONSOLE_TEXT_COLOR[back_color] 60 61 @staticmethod 62 def set(fore_color='FOREGROUND_WHITE', back_color='BACKGROUND_BLACK', handle=std_output_handle): 63 is_set = ctypes.windll.kernel32.SetConsoleTextAttribute(handle, TextColor.text_color(fore_color, back_color)) 64 return is_set 65 66 @staticmethod 67 def unset(): 68 return TextColor.set() 69 70 71 def set_textcolor(fore_color='FOREGROUND_WHITE', back_color='BACKGROUND_BLACK'): 72 73 def decorate(func): 74 """ 75 A decorator which can set the color of console message, include foreground and background. 76 :param 77 func: The original function which will be wrapped by a wrapper. 78 :return: 79 wrapper 80 """ 81 @wraps(func) 82 def wrapper(*args, **kwargs): 83 TextColor.set(fore_color, back_color) 84 res = func(*args, **kwargs) 85 TextColor.unset() 86 return res 87 88 return wrapper 89 90 return decorate
.. ools\__init__.py: None
.. oolsos_tools.py:
1 """ 2 Originated from os module. 3 4 # ----------------------------------------- 5 # Author: 6 # Created: 7 # Modified: 8 # Version: 0.1.0 9 # ----------------------------------------- 10 """ 11 12 import os 13 14 15 def makedir(path): 16 """ 17 Recursive directory creation, include all intermediate-level directories needed to contain the leaf directory. 18 19 If specific path already exists, do nothing. Or path is not a directory, return False. 20 """ 21 try: 22 os.makedirs(path, 0o777) 23 # os.makedirs(path, 0o777, False) # Version 3.2+ 24 except OSError: 25 if not os.path.isdir(path): # Path is not a valid directory. 26 return False # Path already exists. 27 return True 28 else: # Path does not exist. 29 return True