创建scrapy项目
$ scrapy startproject project_name
创建爬虫文件
scrapy genspider [-t template] <name> <domain>
$ scrapy genspider -l Available templates: basic crawl csvfeed xmlfeed
爬虫进行抓取
$ scrapy crawl spider_name --logfile spider_name.log
“故障排除,调试”
scrapy shell [-s "User-Agent"=" "] http://www.baidu.com
response.text
response.body
response.css(" ::text").extract_first()
response.xpath("/text()").extract()[0]