1、安装Google-chrome(cent-os系统):
wget https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm
yum install google-chrome-stable_current_x86_64.rpm
检查是否安装成功:google-chrome --version
2、安装Webdriver驱动:
wget https://chromedriver.storage.googleapis.com/2.41/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
添加到可执行文件命令里cp chromedriver /usr/bin/
检查是否安装成功:chromedriver --version
3、编写脚本
from selenium import webdriver
def spider(url='https://baidu.com'):
option = webdriver.ChromeOptions()
option.add_argument('--headless')
option.add_argument('--disable-dev-shm-usage')
option.add_argument('--no-sandbox')
driver = webdriver.Chrome(executable_path='chromedriver', chrome_options=option)
driver.get(url)
print(driver.page_source)
if __name__ == '__main__':
spider()
其中:
“–no-sandbox”参数是让Chrome在root权限下跑
“–headless”参数是不用打开图形界面