• scrapy RuntimeError: maximum recursion depth exceeded while calling a Python object 超出python最大递归数异常


    2019-10-21 19:01:00 [scrapy.core.engine] INFO: Spider opened
    2019-10-21 19:01:00 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
    2019-10-21 19:01:00 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
    2019-10-21 19:01:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://amp-api-search-edge.apps.apple.com/v1/catalog/cn/search?term=%E7%AE%A1%E8%B4%A6&platform=iphone&include=apps,top-apps&bubble[search]=apps,developers,groupings,editorial-items,app-bundles,in-apps&l=zh-Hans-CN&extend=editorialBadgeInfo,messagesScreenshots,minimumOSVersion,requiredCapabilities,screenshotsByType,supportsFunCamera,videoPreviewsByType> (referer: None)
    2019-10-21 19:01:01 [scrapy.core.scraper] DEBUG: Scraped from <200 https://amp-api-search-edge.apps.apple.com/v1/catalog/cn/search?term=%E7%AE%A1%E8%B4%A6&platform=iphone&include=apps,top-apps&bubble[search]=apps,developers,groupings,editorial-items,app-bundles,in-apps&l=zh-Hans-CN&extend=editorialBadgeInfo,messagesScreenshots,minimumOSVersion,requiredCapabilities,screenshotsByType,supportsFunCamera,videoPreviewsByType>
    None
    2019-10-21 19:01:01 [scrapy.core.engine] INFO: Closing spider (finished)
    Traceback (most recent call last):
    File "/usr/lib/python2.7/logging/__init__.py", line 861, in emit
    msg = self.format(record)
    File "/usr/lib/python2.7/logging/__init__.py", line 734, in format
    return fmt.format(record)
    File "/usr/lib/python2.7/logging/__init__.py", line 465, in format
    record.message = record.getMessage()
    File "/usr/lib/python2.7/logging/__init__.py", line 329, in getMessage
    msg = msg % self.args
    File "/home/project/release/venv2.7/local/lib/python2.7/site-packages/scrapy/spiders/__init__.py", line 107, in __str__
    return "<%s %r at 0x%0x>" % (type(self).__name__, self.name, id(self))
    ...... 重复n行红色日志 .......
    File "/home/project/release/venv2.7/local/lib/python2.7/site-packages/scrapy/spiders/__init__.py", line 107, in __str__
    return "<%s %r at 0x%0x>" % (type(self).__name__, self.name, id(self))
    RuntimeError: maximum recursion depth exceeded while calling a Python object
    Logged from file signal.py, line 57
    2019-10-21 19:01:01 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
    {'downloader/request_bytes': 812,
    'downloader/request_count': 1,
    'downloader/request_method_count/GET': 1,
    'downloader/response_bytes': 19255,
    'downloader/response_count': 1,
    'downloader/response_status_count/200': 1,
    'finish_reason': 'finished',
    'finish_time': datetime.datetime(2019, 10, 21, 11, 1, 1, 765291),
    'item_scraped_count': 1,
    'log_count/DEBUG': 2,
    'log_count/ERROR': 1,
    'log_count/INFO': 9,
    'memusage/max': 1424973824,
    'memusage/startup': 1424973824,
    'response_received_count': 1,
    'scheduler/dequeued': 1,
    'scheduler/dequeued/memory': 1,
    'scheduler/enqueued': 1,
    'scheduler/enqueued/memory': 1,
    'start_time': datetime.datetime(2019, 10, 21, 11, 1, 0, 418993)}
    2019-10-21 19:01:01 [scrapy.core.engine] INFO: Spider closed (finished)

    Process finished with exit code 0

    原因:spiders中的close函数中有异常! 

  • 相关阅读:
    sudo apt-get install openssh-server时提示需要安装1:6.6p1-2ubuntu1的解决办法(图文详解)
    Elasticsearch之Hadoop插件的安装(图文详解)
    [转]VC++的类头文件
    [转]VC++中对文件的写入和读取
    [转]在VS2010 VC++项目中引用Lib静态库(以Openssl为例)
    [转]List of Visual Studio Project Type GUIDs
    [转]如何使用MFC和类型库创建自动化项目
    [转]深入浅出WPF(7)——数据的绿色通道,Binding
    [转]自定义ASP.NET MVC JsonResult序列化结果
    [转]ASP.NET MVC Json()处理大数据异常解决方法 json maxjsonlength
  • 原文地址:https://www.cnblogs.com/aaron-agu/p/11715357.html
Copyright © 2020-2023  润新知