• [转]scrapy中的logging


    logging模块是Python提供的自己的程序日志记录模块。

    在大型软件使用过程中,出现的错误有时候很难进行重现,因此需要通过分析日志来确认错误位置,这也是写程序时要使用日志的最重要的原因。

    scrapy使用python内置的logging模块记录日志


    日志的级别

    1. logging.CRITICAL - for critical errors (highest severity)

    2. logging.ERROR - for regular errors

    3. logging.WARNING - for warning messages

    4. logging.INFO - for informational messages

    5. logging.DEBUG - for debugging messages (lowest severity)


    基本使用方法

    1.简单使用方法

    import logging

    Logging.warning(“this is a test ”)

    执行结果:

    Python1.png 

    2.通用的记录日志的方法,可加入日志的级别

    import logging

    Logging.log(logging.WARNING,”this is a warning”)

    3,通过logger记录日志

    import logging

    logger=logging.getLogger(_name_)

    Logger.warning(“this is a warning”)


    在scrapy中使用

    Scrapy provides a logger within each Spider instance, that can be accessed and used like this:

    import scrapy

    class MySpider(scrapy.Spider):

    name = 'myspider'

    start_urls = ['http://scrapinghub.com']

    def parse(self, response):

    self.logger.info('Parse function called on %s', response.url)

    That logger is created using the Spider’s name, but you can use any custom Python logger you want. For example:

    import logging import scrapy

    logger = logging.getLogger('mycustomlogger')

    class MySpider(scrapy.Spider):

    name = 'myspider'

    start_urls = ['http://scrapinghub.com']

    def parse(self, response):

    logger.info('Parse function called on %s', response.url)


    在settings.py中配置

    These settings can be used to configure the logging:

    • LOG_FILE 

    • LOG_ENABLED

    • LOG_ENCODING

    • LOG_LEVEL 

    • LOG_FORMAT 

    • LOG_DATEFORMAT 

    • LOG_STDOUT

    转载自:http://www.maiziedu.com/wiki/crawler/logging/

  • 相关阅读:
    正则表达式
    爬虫原理和网页构造
    简单的博客系统之二
    配置编辑器geany
    linux删除多文件
    eNSP交换路由基础
    NTP centOS6.5
    shell脚本之lftp上传
    进度条
    maketrans与translate函数
  • 原文地址:https://www.cnblogs.com/themost/p/7044879.html
Copyright © 2020-2023  润新知