30、吃什么不会胖

去薄荷网 爬取每种食物的热量值

 

URL    http://www.boohee.com/food/

 

from gevent import monkey
monkey.patch_all()
import gevent,requests,bs4,csv
from gevent.queue import Queue

def crawler():
    headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'}
    while not work.empty():
        url = work.get_nowait()
        res = requests.get(url,headers=headers)
        bs_res = bs4.BeautifulSoup(res.text,'html.parser')
        datas = bs_res.find('ul',class_='food-list').find_all(class_='text-box pull-left')
        for data in datas:
            writer.writerow([data.text.strip().replace('

',',')])

work = Queue()

url = 'http://www.boohee.com/food/group/{group}?page={page}'
for group in range(1,11):
    for page in range(1,11):
        real_url = url.format(group=group,page=page)
        work.put_nowait(real_url)

csv_file = open('timetop.csv','w',newline='',encoding='utf-8-sig')
writer = csv.writer(csv_file)

task_list=[]
for x in range(4):
    task = gevent.spawn(crawler)
    task_list.append(task)
gevent.joinall(task_list)

csv_file.close()

执行结果如下

"红豆，又叫赤小豆、红小豆、红饭豆、米赤豆、...,热量：324 大卡(每100克)"
"餐包，又叫小餐包、面包,热量：284 大卡(每100克)"
"烧饼(加糖),热量：298 大卡(每100克)"
"绿豆（干），又叫青小豆、植豆、交豆,热量：329 大卡(每100克)"


"嘉力缘 黑豆醋参片【清畅】,热量：388 大卡(每100克)"
"NATURE VALLEY 蛋白质能量棒,热量：475 大卡(每100克)"
"康恩贝 维生素C泡腾片,热量：338 大卡(每100克)"