2021-05-11

Posted 2021-06-12 离落想AC

爬取网页

# -*- coding: UTF-8 -*-
# @Time : 2021/5/9 19:13
# @Author : 李如旭
# @File ：yaopon.py
# @Software: PyCharm
import requests
import json
if __name__ == '__main__':

    url = 'http://scxk.nmpa.gov.cn:81/xk/itownet/portalAction.do?method=getXkzsList'
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (Khtml, like Gecko) Chrome/90.0.4430.93 Safari/537.36'
    }

    all_data = []

    for index in range(1,5):
        index = str(index)
        data ={
            'on': 'true',
            'index': index,
            'pageSize': '15',
            'productName': '',
            'conditionType': '1',
            'applyname': '',
            'applysn': ''
        }

        response = requests.post(url=url, data=data, headers=headers)

        page_text = response.text
        all_data.append(page_text)

    # 持久化存储all_data_list
    fp = open('./yao.json', 'w', encoding='utf-8')
    json.dump(all_data, fp=fp, ensure_ascii=False)
    print('over!!!')

实现效果

爬取结果