• pyspider 启动错误


    [root@localhost python]# pyspider all
    [W 180629 07:08:26 run:413] phantomjs not found, continue running without it.
    [I 180629 07:08:29 result_worker:49] result_worker starting...
    [I 180629 07:08:31 processor:211] processor starting...
    [I 180629 07:08:31 tornado_fetcher:638] fetcher starting...
    [I 180629 07:08:31 scheduler:675] scheduler starting...
    [I 180629 07:08:31 scheduler:614] in 5m: new:0,success:0,retry:0,failed:0
    [I 180629 07:08:31 scheduler:810] scheduler.xmlrpc listening on 127.0.0.1:23333
    [I 180629 07:08:32 app:84] webui exiting...
    Traceback (most recent call last):
      File "/root/.pyenv/versions/3.6.5/bin/pyspider", line 11, in <module>
        load_entry_point('pyspider==0.3.10', 'console_scripts', 'pyspider')()
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pyspider-0.3.10-py3.6.egg/pyspider/run.py", line 754, in main
        cli()
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click-6.7-py3.6.egg/click/core.py", line 722, in __call__
        return self.main(*args, **kwargs)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click-6.7-py3.6.egg/click/core.py", line 697, in main
        rv = self.invoke(ctx)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click-6.7-py3.6.egg/click/core.py", line 1066, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click-6.7-py3.6.egg/click/core.py", line 895, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click-6.7-py3.6.egg/click/core.py", line 535, in invoke
        return callback(*args, **kwargs)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click-6.7-py3.6.egg/click/decorators.py", line 17, in new_func
        return f(get_current_context(), *args, **kwargs)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pyspider-0.3.10-py3.6.egg/pyspider/run.py", line 497, in all
        ctx.invoke(webui, **webui_config)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click-6.7-py3.6.egg/click/core.py", line 535, in invoke
        return callback(*args, **kwargs)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click-6.7-py3.6.egg/click/decorators.py", line 17, in new_func
        return f(get_current_context(), *args, **kwargs)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pyspider-0.3.10-py3.6.egg/pyspider/run.py", line 384, in webui
        app.run(host=host, port=port)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pyspider-0.3.10-py3.6.egg/pyspider/webui/app.py", line 59, in run
        from .webdav import dav_app
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pyspider-0.3.10-py3.6.egg/pyspider/webui/webdav.py", line 216, in <module>
        dav_app = WsgiDAVApp(config)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/WsgiDAV-3.0.0a2-py3.6.egg/wsgidav/wsgidav_app.py", line 122, in __init__
        _check_config(config)
      File "/root/.pyenv/versions/3.6.5/lib/python3.6/site-packages/WsgiDAV-3.0.0a2-py3.6.egg/wsgidav/wsgidav_app.py", line 104, in _check_config
        raise ValueError("Invalid configuration:
      - " + "
      - ".join(errors))
    ValueError: Invalid configuration:
      - Deprecated option 'dir_browser.enable': use 'middleware_stack' instead.
      - Deprecated option 'domaincontroller': use 'domain_controller' instead.
    

      

    我也遇到此问题,发现github里面也有人在问,我已做回答,把解决办法在这里同步一下。

    这是WsgiDAV发布了版本 pre-release 3.x导致的。

    修改 pyspider/webui/webdav.py 第203行:

    config = DEFAULT_CONFIG.copy()
     config.update({
        'mount_path': '/dav',
        'provider_mapping': {
            '/': ScriptProvider(app)
        },
        #'domaincontroller': NeedAuthController(app),
        'http_authenticator': {
            'HTTPAuthenticator':NeedAuthController(app),
        },
        
        'verbose': 1 if app.debug else 0,
        
        'dir_browser': {'davmount': False,
                        #'enable': True,
                        'msmount': False,
                        'response_trailer': ''},
    })
    dav_app = WsgiDAVApp(config)
    

      然后执行:

    python setup.py install
    

     

  • 相关阅读:
    CrawlSpiders
    从抓取Tencent中学习Scrapy
    对象返回规范的url的两种方式的两种方式
    多对多关系的额外字段
    Django定时任务
    Scripy学习(一)
    Django开发博客一(搭建模型和准备数据)
    求并集
    求子集、交集
    java数学函数Math类中常用的方法
  • 原文地址:https://www.cnblogs.com/yuyu666/p/10495242.html
Copyright © 2020-2023  润新知