python之小米应用商店搜索

Posted 2020-12-15 夏日的向日葵

需求：我有一系列appname，想要判断这些appname是否在小米应用市场存在

解决方案：小编写了一个爬虫小程序，返回搜索第一条appname，以此判断该APP是否在小米应用市场存在。

import requests
from bs4 import BeautifulSoup

def save_file(str_content, save_path):
    import os
    #判断写入的路径是否正确
    #assert save_path and os.path.exists(save_path), "the path you specified is error!"
    #a+为追加
    with open(save_path, \'a+\', encoding="utf-8") as file: # gkb
        file.write(str(str_content)+"\\n")
    #print(">>> save over!")
with open("F:\\\\data\\\\appname_20200714_ysf.txt", "r",encoding=\'utf-8\') as f:
    list_a = f.readlines()
    list_b = [i.rsplit(\'\\n\')[0] for i in list_a]
    for line in list_b:
        #print(line)
        url1 = "http://app.mi.com/search?keywords=" + line
        wbdata = requests.get(url1).text
        try:
            soup = BeautifulSoup(wbdata, \'lxml\')
            applist = soup.find(class_=\'applist\')
            first_app = applist.find_all(name=\'h5\')
            content = line + \'|\' + first_app[0].string
            print(content)
            save_file(content, "F:/data/result20200714.txt")
        except Exception as e:
            content = line + \'|\' +"cannot found"
            print(content)
            save_file(content, "F:/data/result20200714.txt")

结果：

51零用钱|51信用钱包
51用车|51车主加油
…………