笔趣阁小说爬取

Posted Sakura_柏

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了笔趣阁小说爬取相关的知识,希望对你有一定的参考价值。

 

http://blog.csdn.net/qq_22073849/article/details/78018980  小说爬取参考

完成到一半

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import requests
from bs4 import BeautifulSoup

def get_url_list(url):
    content = requests.get(url).content
    soup = BeautifulSoup(content,lxml)
    url_list = []
    # urls = soup.find(‘div‘,{‘id‘:‘list‘}).find(‘dl‘).find_all(‘dd‘)
    urls = soup.select(#list > dl > dd > a)
    for i in urls:
        i = i.get(href)
        print(i)
        i = http://www.biquge.com.tw + i
        url_list.append(i)
    print (url_list)
    return url_list

def get_data(url):
    content = requests.get(url).content
    soup = BeautifulSoup(content, lxml)
    f = open(rD:\Test EXCE1\HMXX.txt,a+,encoding=utf-8)
    text_name = soup.find(div,{class:bookname}).find(h1).text
    print(text_name)



if __name__ ==__main__:
    url = http://www.biquge.com.tw/18_18049/8057787.html
    # get_url_list(url)
    get_data(url)

 

以上是关于笔趣阁小说爬取的主要内容,如果未能解决你的问题,请参考以下文章

Python 爬取笔趣阁小说

笔趣阁小说爬取

爬取笔趣阁小说

Requests和Xpath笔趣阁小说采集爬取教程

bs4爬取笔趣阁小说

Python BeautifulSoup 爬取笔趣阁所有的小说