如何将我的 main() 函数输出为 CSV 文件

Posted 2023-02-23

【中文标题】如何将我的 main() 函数输出为 CSV 文件

【英文标题】：How to output my main() function as a CSV file

【发布时间】：2021-12-11 10:56:20

【问题描述】：

我正在尝试将我的主要功能输出为 CSV 文件。我曾尝试返回我的数据框、打印我的数据框等，但似乎无法使其正常工作。任何人都可以解释一下吗？谢谢。

CSV 输出代码：

if __name__ == '__main__':
    with open(f'todayHPD.csv', 'w') as fp:
        a = csv.writer(fp, delimiter = ',')
        a.writerows(main())

更新的 CSV 输出代码： Click for CSV output image

我使用下面用户提供的答案进行了更新，但我抓取的西里尔文字正在返回替换块 ����

if __name__ == '__main__':
    df = main()
    df.to_csv(f'todayHPD.csv', encoding='cp1251', errors='ignore', index=False)

完整代码：

import requests
from bs4 import BeautifulSoup as BS
from datetime import datetime
import pandas as pd
import re
import csv

today = datetime.today().strftime('%y%m%d ')

def main():
    page = 0
    name = []
    date = []
    address = []
    district = []
    city = []
    price = []
    area_sqm = []
    rooms = []
    floor = []
    commission_year = []
    building_floors = []
    garage = []
    balcony = []
    windows = []
    window_type = []
    floor_type = []
    door_type = []
    leasing = []
    description = []
    link = []

    BASE = 'https://www.unegui.mn'
    URL = f'BASE/l-hdlh/l-hdlh-zarna/oron-suuts-zarna/5-r/?page='
    COLUMNS=['Name','Date','Address','District','City','Price','Area_sqm','Rooms','Floor','Commission_year',
             'Building_floors','Garage', 'Balcony','Windows','Window_type','Floor_type','door_type','Leasing','Description','Link']
    with requests.Session() as session:
        while True:
            (r := session.get(f'URLpage+1')).raise_for_status()
            m = re.search('.*page=(\d+)$', r.url)
            if m and int(m.group(1)) == page:
                break
            page += 1
            print(f'Scrapping page page')
            soup = BS(r.text, 'lxml')
            for tag in soup.findAll('div', class_='list-announcement-block'):
                _name = tag.find('a', attrs='itemprop': 'name')
                name.append(_name.get('content', 'N/A'))
                if (_link := _name.get('href', None)):
                    link.append(f'BASE_link')
                    (_r := session.get(link[-1])).raise_for_status()
                    _spanlist = BS(_r.text, 'lxml').find_all('span', class_='value-chars')
                    floor_type.append(_spanlist[0])
                    balcony.append(_spanlist[1])
                    garage.append(_spanlist[2])
                    window_type.append(_spanlist[3])
                    door_type.append(_spanlist[4])    
                    windows.append(_spanlist[5])
                    
                    _alist = BS(_r.text, 'lxml').find_all('a', class_='value-chars')
                    commission_year.append(_alist[0])
                    building_floors.append(_alist[1])
                    area_sqm.append(_alist[2])
                    floor.append(_alist[3])
                    leasing.append(_alist[4])
                    district.append(_alist[5])
                    address.append(_alist[6])
                    
                    commission_year.append(_spanlist[2])
                    
                rooms.append(tag.find('div', attrs='announcement-block__breadcrumbs').get_text().split('»')[1].strip())
                description.append(tag.find('div', class_='announcement-block__description').get_text().strip())
                date.append(tag.find('div', class_='announcement-block__date').get_text().split(',')[0].strip())
                city.append((tag.find('meta', attrs='itemprop': 'areaServed')).get('content'))
                if (_price := tag.find('meta', attrs='itemprop': 'price')) is None:
                    _price = tag.find('div', class_='announcement-block__price _premium')
                price.append(_price.get_text().strip() if _price else 'N/A')
            df = pd.DataFrame(zip(name, date, address, district, city, 
                                  price, area_sqm, rooms, floor, commission_year,
                                  building_floors, garage, balcony, windows, window_type,
                                  floor_type, door_type, leasing, description, link), columns=COLUMNS)
            print(df)

if __name__ == '__main__':
    with open(f'todayHPD.csv', 'w') as fp:
        a = csv.writer(fp, delimiter = ',')
        a.writerows(main())

【参考方案1】：

从您的main 函数返回df，而不是print(df)。 您还可以在数据帧上使用to_csv 函数。

def main()
    ...
    return df

if __name__ == '__main__':
    df = main()
    df.to_csv(f'todayHPD.csv', index=False)

【讨论】：

嗨！当我创建这样的 csv 文件并使用编码 UTF-8 或编码 CP1251 时，我不断得到 ??????作为我的输出。我正在尝试输出西里尔字符。我把代码作为更新放在上面。你知道有什么方法可以解决这个问题吗？

@Sod 你能试试 UTF-16 编码吗？

utf-16 仍然返回 ����

我为我得到的输出添加了图片链接。