通常情况下用selenium开发爬虫时,会使用以下方式添加UA
chrome_options.add_argument('user-agent=' + self.ua)
但是每次更换需要重启浏览器,这会造成资源浪费。好在selenium支持CDP,于是翻了文档终于找到了解决方法。解决方法如下
driver.execute_cdp_cmd("Emulation.setUserAgentOverride", { "userAgent": "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20100101 Firefox/6.0" })
执行后打开控制台,输入navigator.userAgent查看,会发现userAgent已经改变,访问任何页面都会是目前设置的
同时官方文档中还有以下参数可以使用:
{ "name": "setUserAgentOverride", "description": "Allows overriding user agent with the given string.", "redirect": "Emulation", "parameters": [ { "name": "userAgent", "description": "User agent to use.", "type": "string" }, { "name": "acceptLanguage", "description": "Browser langugage to emulate.", "optional": true, "type": "string" }, { "name": "platform", "description": "The platform navigator.platform should return.", "optional": true, "type": "string" }, { "name": "userAgentMetadata", "description": "To be sent in Sec-CH-UA-* headers and returned in navigator.userAgentData", "experimental": true, "optional": true, "$ref": "Emulation.UserAgentMetadata" } ] }