• 【Python 库】fake-useragent 伪装请求头


    写爬虫的时候,在进行 request 请求的时候,多数情况下需要添加请求头,否则就不能正常请求。

    添加请求头最常用的做法是修改 User-Agent 来伪装浏览器。

    以前在写请求头的时候,都是通过 copy 来解决的,昨天看到了一个库 fake-useragent,以后再也不用烦恼了。

    官网地址:https://pypi.org/project/fake-useragent/

    用法非常的简单:

    headers = {"User-Agent": UserAgent().chrome}

    看下官网的示例:

    from fake_useragent import UserAgent
    ua = UserAgent()
    
    ua.ie
    # Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US);
    ua.msie
    # Mozilla/5.0 (compatible; MSIE 10.0; Macintosh; Intel Mac OS X 10_7_3; Trident/6.0)'
    ua['Internet Explorer']
    # Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; GTB7.4; InfoPath.2; SV1; .NET CLR 3.3.69573; WOW64; en-US)
    ua.opera
    # Opera/9.80 (X11; Linux i686; U; ru) Presto/2.8.131 Version/11.11
    ua.chrome
    # Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.2 (KHTML, like Gecko) Chrome/22.0.1216.0 Safari/537.2'
    ua.google
    # Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.13 (KHTML, like Gecko) Chrome/24.0.1290.1 Safari/537.13
    ua['google chrome']
    # Mozilla/5.0 (X11; CrOS i686 2268.111.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11
    ua.firefox
    # Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1
    ua.ff
    # Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:15.0) Gecko/20100101 Firefox/15.0.1
    ua.safari
    # Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5355d Safari/8536.25
    
    # and the best one, random via real world browser usage statistic
    ua.random

    最后一个 random 是随机获取,真的是非常的方便。

    有时候会报 fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached,我重新运行了下就好了,也有说可以禁用服务器缓存

    headers = {"User-Agent": UserAgent(use_cache_server=False).chrome}
  • 相关阅读:
    CF 142B Tprimes
    CF 231A Team
    poj 2001 Shortest Prefixes ——字典树入门
    hdu 1039 Easier Done Than Said?
    poj 2528 Mayor's posters
    hdu 1061 Rightmost Digit
    poj 2503 Babelfish
    CF271 A. Beautiful Year
    poj 2752
    CF271 B. Prime Matrix
  • 原文地址:https://www.cnblogs.com/gl1573/p/9869650.html
Copyright © 2020-2023  润新知