学习任务:爬取豆瓣喜剧类热门TOP60的电影并保存在douban.txt文件中。
代码示例:
import requests url="https://movie.douban.com/j/new_search_subjects" for i in range(0,60,20):#for循环,每次爬取的电影数量加20 param = { "sort": "U", "range": "0,10", "tags": "", "start": i, "genres": "喜剧", } header = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36" } resp = requests.get(url=url, params=param, headers=header) with open("douban.txt", mode="a", encoding="utf-8") as f: f.write(resp.text) print("爬取完成!") print(resp.text)