python爬虫lxml xpath测试
Posted 皓月盈江
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了python爬虫lxml xpath测试相关的知识，希望对你有一定的参考价值。
xpath测试1：
main.py
"""=== coding: UTF8 ==="""
from lxml import etree
xml = """
<book>
<id>1</id>
<name>春风</name>
<price>1.56</price>
<nick>万里</nick>
<author>
<nick id="10086">周大庆</nick>
<nick id="10010">黄天山</nick>
<nick class="joy">周谈浩</nick>
<div>
<nick>嘟嘟</nick>
</div>
<span>
<nick>珊瑚</nick>
</span>
</author>
</book>
"""
"""
========================================
主函数功能测试
========================================
"""
if __name__ == '__main__':
tree = etree.XML(xml)
# result = tree.xpath("/book") # /表示层级关系，第一个/是根节点
# result = tree.xpath("/book/name/text()") # text()拿文本
# result = tree.xpath("/book/author//nick/text()") # //后代
result = tree.xpath("/book/author/*/nick/text()") # *任意的节点，通配符
print(result)
xpath测试2：
test.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>测试</title>
</head>
<body>
<ul>
<li><a href="http://www.baidu.com">百度</a></li>
<li><a href="http://www.google.com">谷歌</a></li>
<li><a href="http://www.sogou.com">搜狗</a></li>
</ul>
<ol>
<li><a href="feiji">飞机</a></li>
<li><a href="dapao">大炮</a></li>
<li><a href="huoche">火车</a></li>
</ol>
<div class="job">高凡尔</div>
<div class="common">刘珂</div>
</body>
</html>
main.py
"""=== coding: UTF8 ==="""
from lxml import etree
"""
========================================
主函数功能测试
========================================
"""
if __name__ == '__main__':
parser = etree.HTMLParser(encoding='utf-8')
tree = etree.parse("test.html", parser=parser)
# result = tree.xpath("/html") # /表示层级关系，第一个/是根节点
# result = tree.xpath("/html/body/ul/li/a/text()") # text()拿文本
# result = tree.xpath("/html/body/ul/li[1]/a/text()") # xpath的顺序是从1开始数的，[]表示索引
# result = tree.xpath("/html/body/ol/li/a[@href='dapao']/text()") # @xxx=xxx表示属性的筛选
# print(result)
ol_li_list = tree.xpath("/html/body/ol/li")
for li in ol_li_list:
# 从每一个li中提取到文字信息
result = li.xpath("./a/text()") # 在li中继续查找，相对查找
print(result)
result = li.xpath("./a/@href") # 拿到属性值: @属性
print(result)
print(tree.xpath("/html/body/ul/li/a/@href"))
print(tree.xpath("/html/body/div[1]/text()"))
print(tree.xpath("/html/body/ol/li/a/text()"))
关注公众号，获取更多资料
以上是关于python爬虫lxml xpath测试的主要内容，如果未能解决你的问题，请参考以下文章
Python爬虫基础——XPath语法的学习与lxml模块的使用
Python爬虫：通过爬取CSDN博客信息，学习lxml库与XPath语法