• 安装scrapy框架


    安装scrapy框架之前,需要安装几个必备库

    ps.分享个python库下载地址:https://www.lfd.uci.edu/~gohlke/pythonlibs/

    0、wheel(有了这个库之后可以本地安装pyhton库)

    1、lxml

    2、pyOpenSSL

    3、pywin32

    4、twisted

    anaconda可以使用自带的conda install xxxx来安装组件

    必备组件安装完成后,pip install scrapy 或者conda install scrapy

    安装完成后,test一下:

    命令行输入scrapy

    (C:ProgramDataAnaconda3) C:Userswangguoqiang>Scrapy
    Scrapy 1.5.1 - no active project
    
    Usage:
    scrapy <command> [options] [args]
    
    Available commands:
    bench Run quick benchmark test
    fetch Fetch a URL using the Scrapy downloader
    genspider Generate new spider using pre-defined templates
    runspider Run a self-contained spider (without creating a project)
    settings Get settings values
    shell Interactive scraping console
    startproject Create new project
    version Print Scrapy version
    view Open URL in browser, as seen by Scrapy
    
    [ more ] More commands available when run from project directory
    
    Use "scrapy <command> -h" to see more info about a command

    显示如下即代表安装成功

    继续输入Scrapy startproject hello

    (C:ProgramDataAnaconda3) C:Userswangguoqiang>Scrapy startproject hello
    New Scrapy project 'hello', using template directory 'c:\programdata\anaconda
    \lib\site-packages\scrapy\templates\project', created in:
    C:Userswangguoqianghello
    
    You can start your first spider with:
    cd hello
    scrapy genspider example example.com

    cd 进入创建的hello项目中

    输入scrapy genspider baidu www.baidu.com

    (C:ProgramDataAnaconda3) C:Userswangguoqiang>cd hello
    
    (C:ProgramDataAnaconda3) C:Userswangguoqianghello>scrapy genspider baidu ww
    w.baidu.com
    Created spider 'baidu' using template 'basic' in module:
      hello.spiders.baidu

    输入

    scrapy crawl baidu

  • 相关阅读:
    C#的显式接口和隐式接口
    Working with XML in a Classic COM Application
    规格单位换算
    C#压缩解压缩(文件夹里包含文件夹)
    在线编辑器原理
    右键新建文本文档没有了。
    MemoryStream读写
    gacutil.exe ,RegAsm.exe 和全局缓存(GAC)
    OData services入门使用ASP.NET Web API描述
    Readonly和Disabled
  • 原文地址:https://www.cnblogs.com/wang666/p/9467829.html
Copyright © 2020-2023  润新知