Requests模块

Requests模块

1.headers关键字

import requests
from urllib.parse import urlencode

keyword = input('>>:').strip()
res = urlencode({'wd': keyword}, encoding='utf-8')
url = 'https://www.baidu.com/s?' + res
print(url)
response = requests.get(url,
                        headers={
                            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36'
                        },
                        )
print(response.status_code)
with open('a.html','w',encoding='utf-8') as f:
    f.write(response.text)

2.params方法实现上面关键字的方法。（优化了）

import requests
from urllib.parse import urlencode

keyword = input('>>:').strip()
response = requests.get('https://www.baidu.com/s?',
                        params={
                            'wd':keyword,
                            'pn':20
                        },
                        headers={
                            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36'
                        },
                        )
print(response.status_code)
with open('a.html','w',encoding='utf-8') as f:
    f.write(response.text)

3.headers涉及到User_agent,加了user_agent才能获取到内容。

import requests

response = requests.get('https://www.zhihu.com/explore',
                        headers={
                            'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36',}
                        )   
print(response.status_code)
print(response.text)

4.headers涉及到cookie，加了cookie才能获取到内容。

response=requests.get(
    url='https://github.com/settings/emails',
    headers={
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36',
    },
    cookies={
        "k1":"v1",
    },
)
print('378533872@qq.com' in response.text)

5.allow_redirects: 跳转，  默认跳转，改为Flask就不跳转

import requests
response=requests.get(
    url='https://github.com/settings/emails',
    headers={
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36',
    },
    cookies={
        "k1":"v1",
    },
    allow_redirects=False,
)