中秋节出门游玩住宿找不到，酒店信息掌握不准？爬虫帮你解决

Posted 2021-09-17 白又白学Python

采集数据目标

网址：美团酒店

效果展示

工具使用

开发工具：pycharm
开发环境：python3.7， Windows10
使用工具包：requests

项目思路解析

首先选取自己想去旅游的景点，获取景点附近的数据，就比方说白又白中秋国庆打算去长沙游玩获取的就是长沙的住宿酒店（要长沙游玩的可以滴滴白又白）

获取到数据后确定数据的加载是静态数据还是动态数据，当前网页数据为动态加载动态数据需要通过抓包的方式获取，打开调试工具获取到数据

确定好数据后就可以进入代码环节了，对网址发送网络请求

url = \'https://ihotel.meituan.com/hbsearch/HotelSearch?utm_medium=pc&version_name=999.9&cateId=20&attr_28=129&uuid=73B01E7C1051ACFB5730B1C1CD456776945CB50F887E09211F020EB1F6C89996%401631623416857&cityId=198&offset={}&limit=20&startDay=20210914&endDay=20210914&q=&sort=defaults&X-FOR-WITH=4ArIbixslGNtC2oCAtBb2cXDcp3jPZ5xX01C1%2FuNnCXQqh0Edqc3Dkag7qJcicwPAPLY%2FljJLm6wQMNIxLvp9b%2BYD0zfZCkSVgXL0zJuhuGKUZIaSNcfRtkjSISQqvXCOBdIJU9o2Kiz1YxsEqKX%2BlSNhmge6otjb%2B%2FQSr5lMWEicjgDCcQNg0jLrkAO1WXcFHMYZO40i6QdyAWbmxLV6TnJetfiLBxM0oQEvvcnOyA%3D\'.format(i * 20)
        headers = {
            \'Host\': \'ihotel.meituan.com\',
            \'Origin\': \'https://hotel.meituan.com\',
            \'Referer\': \'https://hotel.meituan.com/\',
            \'Cookie\': \'uuid=3c6a1ffa63c44f609095.1631265937.1.0.0; _lxsdk_cuid=178d04766dac8-08045ec98b749f-3f356b-1fa400-178d04766dbc8; mtcdn=K; IJSESSIONID=node01wdulauewgz8610kyb50ik2ee51588346; iuuid=73B01E7C1051ACFB5730B1C1CD456776945CB50F887E09211F020EB1F6C89996; _lxsdk=73B01E7C1051ACFB5730B1C1CD456776945CB50F887E09211F020EB1F6C89996; backurl=http://i.meituan.com/awp/h5/hotel/list/list.html?cityId=96&accommodationType=1&checkIn=2021-09-14&checkOut=2021-09-15; i_extend=Gempty; ci=1; rvct=1%2C96; cityname=%E5%8C%97%E4%BA%AC; _lx_utm=utm_source%3DBaidu%26utm_medium%3Dorganic; _lxsdk_s=17be4558492-2b-59d-9d2%7C%7C2\',
            \'User-Agent\': \'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36\'
        }
        response = requests.get(url, headers=headers)
        print(response.text)

记得带上对应数据请求头获取到json数据

从数据中选取自己想要的数据参数

# 提取想要的数据
            item = {}
            item[\'historySaleCount\'] = data[\'historySaleCount\']
            item[\'cityName\'] = data[\'cityName\']
            item[\'areaName\'] = data[\'areaName\']
            item[\'name\'] = data[\'name\']
            item[\'scoreIntro\'] = data[\'scoreIntro\']
            # item[\'positionDescList\'] = [i[\'text\'] for i in data[\'forward\'].get(\'positionDescList\', "没有")]
            item[\'poiTagList\'] = [i[\'text\'] for i in data[\'poiTagList\']]
            print(item)

提取的数据有各位大佬自己决定
最后将数据保存在csv表格里

就完成啦，你想去哪里玩呢？但是不管去哪里记得勤洗手，戴口罩呀！