• scrapy常用命令(持续) | Commonly used Scrapy command list (con't)


    以下命令都是在CMD中运行,首先把路径定位到项目文件夹

    ------------------------------------------------------------------------------------------

    1. 创建一个scrapy project

    scrapy startproject project_name

    --------------------------------------------------------------------------------------------

    ------------------------------------------------------------------------------------------

    2. 自动生成spider代码命令

    首先进入项目目录,然后敲入如下命令

    scrapy genspider spider_name start_url

    ------------------------------------------------------------------------------------------

    --------------------------------------------------------------------------------------

    3. 列出项目中所有spider

    scrapy list

    返回项目中所有爬虫的名字

    -------------------------------------------------------------------------------------------

    ---------------------------------------------------------------------------------------

    4. 运行一个scrapy project,并且保存结果到csv文件

    scrapy crawl spider_name -o file.csv

    会通过yield命令生成iteration generator,把数据保存到file.csv文件中

    ---------------------------------------------------------------------------------------------

    English Version

    The Below commands will run at CMD which included python path in system env path. 

    Before run the commands, navigate to the directory where you want to create Scrapy project. 

    ------------------------------------------------------------------------------------------

    1. Createa a Scrapy project

    scrapy startproject project_name

    --------------------------------------------------------------------------------------------

    ------------------------------------------------------------------------------------------

    2. Create a spider. It will create a spider sub-class in your source code based on template.

    Navigete to project directory, then key in below command

    scrapy genspider spider_name start_url

    ------------------------------------------------------------------------------------------

    --------------------------------------------------------------------------------------

    3. List all spiders in your project

    scrapy list

    It will return all spiders with spider name

    -------------------------------------------------------------------------------------------

    ---------------------------------------------------------------------------------------

    4. Run a Scrapy project and save the result set into a CSV file.

    scrapy crawl spider_name -o file.csv -t csv

    Scrapy framework will make a iteration generator thru yield command in code and save your data into file.csv

    ---------------------------------------------------------------------------------------------

  • 相关阅读:
    做小程序的流程总结(基本篇)
    ajax页面请求的做的留言板
    模仿文件存储方式,来进行添加、修改、添加子类的操作。
    轮播图(点击之后,会以滑动的形式滑动到指定的图片-有竖向和横向的显示)
    定时器使用实现的动画,宽度、位置等属性
    利用js、html以及Css简单制作了一个模拟手机短信发送
    利用反射获得数据库中存储数据的实现方法
    反射
    equals、static、单例模式的设计
    FPGA设计中的复位
  • 原文地址:https://www.cnblogs.com/wtang/p/7342211.html
Copyright © 2020-2023  润新知