刚学python 没多久, 代码处处是漏洞,也希望各位大佬理解一下
爬出来的图片...
使用的 是 https://www.tianapi.com/ 接口下的 美女图片...
(需要自己注册一个账号, 得到key 这样才能调用 这个接口)
但是今天看天行 好像炸了
下面开始 撸代码l
爬图片的思路
第一 获取url
第二 分析数据信息,得到有用数据
第三 请求每次 得到的信息
第三 撸 下来
(大致就是这么几块)
下面开始 写代码,,, 假如没安装的 python 环境和编辑器的话,自己百度哦(当初弄这个的时候没想到写博客,如果不会,也可以叫我帮忙)
本次需要用到两个库文件
import requests
import json
运行 输入 pip install requests
就ok了
1 url="http://api.tianapi.com/meinv/?key=你的key&num=50" 2 req=requests.get(url) 3 print(req) #查看状态
4#num =50 (每次最多50条信息 否则的话 就只会显示10条信息 ,不要弄错了)
5
将会得到 这个 如果出现这个 表示 连接成功
6 html=req.text
7 print(“=====”+html)
打印出来下面的数据
很明显
他是一个字典类型然后里面又有列表,列表里面又包含字典
E:pythonpython.exe E:/download/day1/homework5.py <Response [200]> 正在下载中... ==========={"code":200,"msg":"success","newslist":[{"ctime":"2018-04-16 10:00","title":"◎秀展女模……[似曾相识 谢立琪等 355p]……第65辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/15/20/201804152010064641-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8319472-0-1.html"},{"ctime":"2018-04-16 13:00","title":"流浪的风","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/16/12/20180416124745291-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8320452-0-1.html"},{"ctime":"2018-04-16 13:00","title":"青春年华-180","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/16/12/201804161243225981-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8320450-0-1.html"},{"ctime":"2018-04-16 20:00","title":"I邻家女孩……[水暧山温 林温蒂 340p]……第1106辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/16/19/20180416195051451-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8321265-0-1.html"},{"ctime":"2018-04-16 22:00","title":"素敏--清爽自然","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/16/21/201804162101097271-5058976.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8321389-0-1.html"},{"ctime":"2018-04-17 12:00","title":"国内顶级丝袜模特系列1","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/16/20/201804162044176741-5058976.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8321357-0-1.html"},{"ctime":"2018-04-17 12:00","title":"青春年华-181","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/17/08/20180417082451741-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8321924-0-1.html"},{"ctime":"2018-04-17 16:00","title":"秋的记忆","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/17/08/20180417082809761-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8321927-0-1.html"},{"ctime":"2018-04-18 00:00","title":"I邻家女孩……[花朝月夕 花倾 73p]……第1107辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/17/23/20180417231039861-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8323786-0-1.html"},{"ctime":"2018-04-18 21:00","title":"☆成熟女人……[无欲则刚 郑倪蓉 50p]……第984辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/18/20/201804182032396691-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8325584-0-1.html"},{"ctime":"2018-04-18 22:00","title":"I邻家女孩……[拟于岱伦 李岱伦 152p]……第1108辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/18/21/20180418213612321-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8325672-0-1.html"},{"ctime":"2018-04-18 22:00","title":"☆成熟女人……[玉洁松贞 陈燕贞 218p]……第985辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/18/20/201804182043572831-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8325610-0-1.html"},{"ctime":"2018-04-18 23:00","title":"I邻家女孩……[月章星句 鲂怯 155p]……第1109辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/18/22/201804182236563791-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8325753-0-1.html"},{"ctime":"2018-04-20 00:00","title":"当鲜花灿烂时","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/18/08/201804180844296291-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8324129-0-1.html"},{"ctime":"2018-04-20 00:00","title":"美女66","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/19/10/20180419102531581-3030167.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8326403-0-1.html"},{"ctime":"2018-04-20 02:00","title":"清纯少女[原创]","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/20/00/201804200057463921-4657629.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8327834-0-1.html"},{"ctime":"2018-04-20 09:00","title":"蔷薇少女","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/20/08/20180420084025741-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8328121-0-1.html"},{"ctime":"2018-04-20 09:00","title":"青春年华-183","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/20/08/201804200835499611-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8328112-0-1.html"},{"ctime":"2018-04-21 00:00","title":"日本最可爱夜场DJ Mel 深夜福利系列私房照","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/20/15/20180420152931381-2053485.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8329154-0-1.html"},{"ctime":"2018-04-21 21:00","title":"I邻家女孩……[无序排列 罗佩恩 102p]……第1110辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/21/20/201804212035208281-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8330834-0-1.html"},{"ctime":"2018-04-21 21:00","title":"★制服女生……[顺序乱了 林艾欣 131p]……第225辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/21/19/201804211957264591-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8330807-0-1.html"},{"ctime":"2018-04-21 22:00","title":"I邻家女孩……[漂亮媳妇 庄g榕 175p]……第1111辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/21/21/201804212105392921-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8330847-0-1.html"},{"ctime":"2018-04-22 09:00","title":"银杏黄了","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/21/15/201804211511057621-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8330537-0-1.html"},{"ctime":"2018-04-22 09:00","title":"簇拥","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/21/15/201804211509295771-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8330535-0-1.html"},{"ctime":"2018-04-22 15:00","title":"I邻家女孩……[相思成灾 忆忆 175p]……第1112辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/22/14/201804221432413751-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8331393-0-1.html"},{"ctime":"2018-04-22 15:00","title":"午後光","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/22/14/201804221430183511-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8331390-0-1.html"},{"ctime":"2018-04-22 15:00","title":"慧心","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/22/14/201804221428205141-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8331389-0-1.html"},{"ctime":"2018-04-22 15:00","title":"古装魅力","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/22/14/201804221407161841-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8331372-0-1.html"},{"ctime":"2018-04-22 16:00","title":"☆成熟女人……[挺挺欲立 苏小立 144p]……第986辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/22/15/201804221521313471-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8331440-0-1.html"},{"ctime":"2018-04-22 16:00","title":"日本美女池田七帆学生制服写真","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/22/15/201804221535331971-2053485.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8331457-0-1.html"},{"ctime":"2018-04-22 16:00","title":"I邻家女孩……[相思成灾 忆忆 180p]……第1112辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/22/14/201804221432413751-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8331393-0-1.html"},{"ctime":"2018-04-22 17:00","title":"☆成熟女人……[橘子熟了 翁 153p]……第987辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/22/15/201804221559212591-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8331489-0-1.html"},{"ctime":"2018-04-22 19:00","title":"乐基儿近照身材壮硕肉感足[贴图]","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/22/17/20180422175812791-3194050.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8331608-0-1.html"},{"ctime":"2018-04-23 11:00","title":"老上海风情","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/23/07/20180423075206181-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8332060-0-1.html"},{"ctime":"2018-04-23 16:00","title":"I邻家女孩……[无序排列 罗佩恩 114p]……第1110辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/21/20/201804212035208281-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8330834-0-1.html"},{"ctime":"2018-04-24 10:00","title":"衣模","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/24/09/201804240919072211-3030167.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8334463-0-1.html"},{"ctime":"2018-04-24 12:00","title":"清新甜美氧气美女肌肤白嫩酥胸美腿私房写真","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/23/18/201804231827029341-2053485.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8333580-0-1.html"},{"ctime":"2018-04-24 12:00","title":"葵林倩影","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/24/06/201804240629131861-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8334223-0-1.html"},{"ctime":"2018-04-25 00:00","title":"人面桃花相映红","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/24/06/20180424063202231-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8334227-0-1.html"},{"ctime":"2018-04-25 10:00","title":"紫春","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/25/09/20180425091032981-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8336584-0-1.html"},{"ctime":"2018-04-25 10:00","title":"雾熙","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/25/09/20180425090828261-239867.gif","url":"http://bbs.voc.com.cn/mm/meinv-8336578-0-1.html"},{"ctime":"2018-04-26 20:00","title":"上国料萌衣~容貌和歌声格外醒目的美少女","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/26/18/2018042618574951-2053485.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8339976-0-1.html"},{"ctime":"2018-04-26 21:00","title":"☆成熟女人……[妍希 151p]……第988辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/26/20/201804262026445371-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8340081-0-1.html"},{"ctime":"2018-04-26 22:00","title":"I邻家女孩……[百卉千葩 潘雅卉 144p]……第1113辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/26/21/20180426210649511-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8340141-0-1.html"},{"ctime":"2018-04-26 22:00","title":"☆成熟女人……[妍姿艳质 妍希 151p]……第988辑","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/26/20/201804262026445371-1559530.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8340081-0-1.html"},{"ctime":"2018-04-27 11:00","title":"孤芳自赏","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/26/21/201804262149345441-619336.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8340219-0-1.html"},{"ctime":"2018-04-27 12:00","title":"宁静的女孩","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/26/22/201804262212048191-619336.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8340243-0-1.html"},{"ctime":"2018-04-28 10:00","title":"人生若只如初见","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/28/09/201804280919423791-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8342677-0-1.html"},{"ctime":"2018-04-28 10:00","title":"醉春","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/28/09/201804280917461191-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8342672-0-1.html"},{"ctime":"2018-04-28 10:00","title":"名模风采","description":"华声美女","picUrl":"http://image.hnol.net/c/2018-04/28/09/20180428091324361-239867.jpg","url":"http://bbs.voc.com.cn/mm/meinv-8342667-0-1.html"}]} 图片已经下载完了哦!!! Process finished with exit code 0
但是 它的类型现在目前是str (C#叫做string) 我们必须要转换一下
dict1=json.loads(html) # 格式化4 调用json 的模块 然后 调用 loads (这里不使用load)打印一下类型 发现已经变成字典类型了
接下来 我们研究一下 刚刚打印出来的数据
发现了 newlist:[{XXXXXX}]
我们一层一层解析出来
dict1_data=dict1["newslist"]
print(dict1_data)
[{'ctime': '2017-01-24 16:00', 'title': '花花', 'description': '华声美女', 'picUrl': 'http://image.hnol.net/c/2017-01/23/15/20170123152957561-1876615.jpg'。。。。。。'url': 'http://bbs.voc.com.cn/mm/meinv-7641359-0-1.html'}]
变成这样就好办了
我们自己 请求图片地址,然后每次请求都把图片写下来
但是大家注意到了没有, 解析出来的 东西 并不是仅仅是图片, 而且还有文字
{'ctime': '2017-01-24 16:00', 'title': '花花', 'description': '华声美女', 'picUrl': 'http://image.hnol.net/c/2017-01/23/15/20170123152957561-1876615.jpg'
但是我们要的只要图片 不要其他的
这里,我们又要再一次解析...
但是 你看见了 我们的图片连接 有很多啊
那我岂不是要请求很多次
这个时候 我们使用 for循环 来请求
然后下载保存
for list_data in dict1_data:
#print(list_data["picUrl"]) 图片 仅仅是 要图片地址,不要其他东西
reqs=requests.get(list_data["picUrl"])
print(reqs) #每次请求
f=open('./image/'+‘ssss.jpg’,"wb")
for reqs_data in reqs:
f.write(reqs_data)
f.close() #关闭
这样子,我们已经把图片下载下来了
可是我们不知道 什么时候下载完了 ,你也不可能一直盯着电脑
我们来个提示
for list_data in dict1_data:
#print(list_data["picUrl"]) 图片 仅仅是 要图片地址,不要其他东西
reqs=requests.get(list_data["picUrl"])
print(reqs) #每次请求
f=open('./image/'+‘ssss.jpg’,"wb")
for reqs_data in reqs:
f.write(reqs_data)
f.close() #关闭
print("图片已经下载完了哦!!!")
这样子 图片下载完毕之后 我们就知道已经完成了
但是 这里也会有个问题
如果你运行的话,只会有一张图片
因为 你一直请求,然后写入到 ssss.jpg ...
这里 你不能写死了 ,我们换一下,注意看看图片的连接名字,发现他们的地址都不一样,名字也不一样
http://sd/sd//sd/.01.jpg 类似这样...
我们能不能也这样子呢
再改一下代码
for list_data in dict1_data:
#print(list_data["picUrl"]) 图片 仅仅是 要图片地址,不要其他东西
reqs=requests.get(list_data["picUrl"])
print(reqs) #每次请求
strimg=list_data["picUrl"].split("/")[-1] #split 是 字符串截取 split("/") 是截图每次遇见/ 就截图 [-1] 是 获取最后 一次的以/的 数据 如 http://的/d/d/d1.jpg 得到的 是d1.jpg
f=open('./image/'+strimg,"wb")
for reqs_data in reqs:
f.write(reqs_data)
f.close() #关闭
print("图片已经下载完了哦!!!")
运行一下
这个错误呢 是我们 抓的太频繁了 ,,, 浏览器 拒绝了本次请求。。重新运行一下就行了(主要是这里没写反扒...)
下面我们重新运行一下
当然在这里 觉得代码太多 看着烦,可以写个函数 封装一下哦
就是这样哦 ,,,一不小心就爬了几百张图片了... 我只会爬这种哦.. 其他的什么都不会(一脸严肃)
1 #coding:utf8 2 # 插入requests 库 3 import requests 4 import json 5 def Mian(): 6 url="http://api.tianapi.com/meinv/?key=XXX&num=10" 7 req=requests.get(url) 8 print(req) #查看状态 9 print("正在下载中...") 10 html=req.text 11 print(html) 12 #print(html_doc) #结果 为 字典 列表 字典 打印 html 内容 13 dict1=json.loads(html) # 格式化4 14 #print("==========="+html) 15 dict1_data=dict1["newslist"] 16 #print(dict1_data) 17 for list_data in dict1_data: 18 #print(list_data["picUrl"]) 图片 19 reqs=requests.get(list_data["picUrl"]) 20 print(reqs) 21 strimg=list_data["picUrl"].split("/")[-1] 22 f=open('./image/'+strimg,"wb") 23 for reqs_data in reqs: 24 f.write(reqs_data) 25 f.close() 26 print("图片已经下载完了哦!!!") 27 28 29 30 31 Mian()
重要的事情说三遍!!!
觉得这篇文章好的 就点点关注哦
觉得这篇文章好的 就点点关注哦
觉得这篇文章好的 就点点关注哦