1 安装
pip3 install beautifulsoup4
小测：
import requests
r = requests.get("http://python123.io/ws/demo.html")
from bs4 import BeautifulSoup
soup = BeautifulSoup(r.text, 'html.parser')
print(soup.prettify())
3 beautiful soup基本元素
<html>
<body>
<p class=“title”> … </p>
</body>
</html>
标签树
Beautiful Soup库是解析、遍历、维护“标签树”的功能库
Beautiful Soup库的引用
Beautiful Soup库，也叫beautifulsoup4 或 bs4
约定引用方式如下，即主要是用BeautifulSoup类
from bs4 import BeautifulSoup
import bs4
from bs4 import BeautifulSoup
soup = BeautifulSoup("<html>data</html>", "html.parser")
soup2 = BeautifulSoup(open("D://demo.html"), "html.parser")
BeautifulSoup对应一个HTML/XML文档的全部内容
演示：
2 基于bs4库的HTML内容遍历方法
import requests
r = requests.get("http://python123.io/ws/demo.html")
demo = r.text
demo
Out[26]: '<html><head><title>This is a python demo page</title></head> <body> <p class="title"><b>The demo python introduces several python courses.</b></p> <p class="course">Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses: <a href="http://www.icourse163.org/course/BIT-268001" class="py1" id="link1">Basic Python</a> and <a href="http://www.icourse163.org/course/BIT-1001870001" class="py2" id="link2">Advanced Python</a>.</p> </body></html>'
HTML基本格式
<>...</>构成了所属关系
<html> <head> <title> This is a python demo page </title> </head> <body> <p class="title"> <b> The demo python introduces several python courses. </b> </p> <p class="course"> Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses: <a class="py1" href="http://www.icourse163.org/course/BIT-268001" id="link1"> Basic Python </a> and <a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2"> Advanced Python </a> . </p> </body> </html>
标签树的下行遍历
soup = BeautifulSoup(demo, 'html.parser') soup.head Out[33]: <head><title>This is a python demo page</title></head> soup.head.contents Out[34]: [<title>This is a python demo page</title>] soup.body.contents Out[35]: [' ', <p class="title"><b>The demo python introduces several python courses.</b></p>, ' ', <p class="course">Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses: <a class="py1" href="http://www.icourse163.org/course/BIT-268001" id="link1">Basic Python</a> and <a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2">Advanced Python</a>.</p>, ' '] len(soup.body.contents) Out[36]: 5 soup.body.contents[1] Out[37]: <p class="title"><b>The demo python introduces several python courses.</b></p>
