• scrapy 自动下载图片


    Item 字段名必须是 image_urls

    即:image_urls = Field()

     item['image_urls']的类型是一个list。

     item['image_urls'] = "http://some.jpg" 是不行的。

    会有如下错误:

    Traceback (most recent call last):
    File "D:Python27libsite-packagesscrapymiddleware.py", line 62, in _process_chain
    return process_chain(self.methods[methodname], obj, *args)
    File "D:Python27libsite-packagesscrapyutilsdefer.py", line 65, in process_chain
    d.callback(input)
    File "D:Python27libsite-packages wistedinternetdefer.py", line 382, in callback
    self._startRunCallbacks(result)
    File "D:Python27libsite-packages wistedinternetdefer.py", line 490, in _startRunCallbacks
    self._runCallbacks()
    --- <exception caught here> ---
    File "D:Python27libsite-packages wistedinternetdefer.py", line 577, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
    File "D:Python27libsite-packagesscrapycontribpipelinemedia.py", line 40, in process_item
    requests = arg_to_iter(self.get_media_requests(item, info))
    File "D:Python27libsite-packagesscrapycontribpipelineimages.py", line 104, in get_media_requests
    return [Request(x) for x in item.get(self.IMAGES_URLS_FIELD, [])]
    File "D:Python27libsite-packagesscrapyhttp equest\__init__.py", line 26, in __init__
    self._set_url(url)
    File "D:Python27libsite-packagesscrapyhttp equest\__init__.py", line 57, in _set_url
    self._set_url(url.encode(self.encoding))
    File "D:Python27libsite-packagesscrapyhttp equest\__init__.py", line 61, in _set_url
    raise ValueError('Missing scheme in request url: %s' % self._url)
    exceptions.ValueError: Missing scheme in request url: h

    必须是  item['image_urls'] = ["http://some.jpg"]

  • 相关阅读:
    oracle数据段详解
    监听静态注册与动态注册
    Oracle网络相关概念与常用配置文件
    pycharm社区版安装及遇到的问题
    强化学习-K摇臂赌博机
    概率图模型
    半监督学习
    卷积神经网络
    递归神经网络
    玻尔兹曼机及其相关模型
  • 原文地址:https://www.cnblogs.com/lance2088/p/4107585.html
Copyright © 2020-2023  润新知