在 Python CSV Writer 循环中写入标题一次

Posted 2023-02-23

【中文标题】在 Python CSV Writer 循环中写入标题一次

【英文标题】：Write Headers Once in Python CSV Writer Loop

【发布时间】：2018-12-23 11:25:41

【问题描述】：

下面是一个抓取器，它遍历两个网站，抓取一个团队的花名册信息，将信息放入一个数组中，然后将数组导出到一个 CSV 文件中。一切都很好，但唯一的问题是每次刮板移动到第二个网站时，csv 文件中的 writerow 标题都会重复。是否可以调整代码的 CSV 部分，以便当刮板循环通过多个网站时，标题只出现一次？提前致谢！

import requests
import csv
from bs4 import BeautifulSoup

team_list='yankees','redsox'

for team in team_list:
    page = requests.get('http://m..mlb.com/roster/'.format(team))
    soup = BeautifulSoup(page.text, 'html.parser')

    soup.find(class_='nav-tabset-container').decompose()
    soup.find(class_='column secondary span-5 right').decompose()

    roster = soup.find(class_='layout layout-roster')
    names = [n.contents[0] for n in roster.find_all('a')]
    ids = [n['href'].split('/')[2] for n in roster.find_all('a')]
    number = [n.contents[0] for n in roster.find_all('td', index='0')]
    handedness = [n.contents[0] for n in roster.find_all('td', index='3')]
    height = [n.contents[0] for n in roster.find_all('td', index='4')]
    weight = [n.contents[0] for n in roster.find_all('td', index='5')]
    DOB = [n.contents[0] for n in roster.find_all('td', index='6')]
    team = [soup.find('meta',property='og:site_name')['content']] * len(names)

    with open('MLB_Active_Roster.csv', 'a', newline='') as fp:
        f = csv.writer(fp)
        f.writerow(['Name','ID','Number','Hand','Height','Weight','DOB','Team'])
        f.writerows(zip(names, ids, number, handedness, height, weight, DOB, team))

【问题讨论】：

你试过将f.writerow移到for team in team_list上方吗？

只需在for 循环之前写入标题即可。这意味着for 循环应该包装在with 上下文管理器中。

【参考方案1】：

使用变量检查是否添加了标头可能会有所帮助。如果添加了标题，则不会添加第二次

header_added = False
for team in team_list:
    do_some stuff

    with open('MLB_Active_Roster.csv', 'a', newline='') as fp:
        f = csv.writer(fp)
        if not header_added:
            f.writerow(['Name','ID','Number','Hand','Height','Weight','DOB','Team'])
            header_added = True
        f.writerows(zip(names, ids, number, handedness, height, weight, DOB, team))

【参考方案2】：

另一种方法是在 for 循环之前简单地执行它，这样您就不必检查是否已经编写。

import requests
import csv
from bs4 import BeautifulSoup

team_list='yankees','redsox'

with open('MLB_Active_Roster.csv', 'w', newline='') as fp:
    f = csv.writer(fp)
    f.writerow(['Name','ID','Number','Hand','Height','Weight','DOB','Team'])

for team in team_list:
    do_your_bs4_and_parsing_stuff

    with open('MLB_Active_Roster.csv', 'a', newline='') as fp:
        f = csv.writer(fp)
        f.writerows(zip(names, ids, number, handedness, height, weight, DOB, team))

您也可以只打开文档一次而不是三次

import requests
import csv
from bs4 import BeautifulSoup

team_list='yankees','redsox'

with open('MLB_Active_Roster.csv', 'w', newline='') as fp:
    f = csv.writer(fp)
    f.writerow(['Name','ID','Number','Hand','Height','Weight','DOB','Team'])

    for team in team_list:
        do_your_bs4_and_parsing_stuff

        f.writerows(zip(names, ids, number, handedness, height, weight, DOB, team))

【参考方案3】：

只需在循环之前写标题，并将循环放在with 上下文管理器中：

import requests
import csv
from bs4 import BeautifulSoup

team_list = 'yankees', 'redsox'

headers = ['Name', 'ID', 'Number', 'Hand', 'Height', 'Weight', 'DOB', 'Team']

# 1. wrap everything in context manager
with open('MLB_Active_Roster.csv', 'a', newline='') as fp:
    f = csv.writer(fp)

    # 2. write headers before anything else
    f.writerow(headers)

    # 3. now process the loop
    for team in team_list:
        # Do everything else...

您也可以在循环外以类似于team_list 的方式定义您的标头，从而使代码更简洁。

【讨论】：

感谢 RoadRunner 的建议！代码运行了，但不幸的是，它返回了一个空的 CSV 文件。我必须在 for 循环的末尾包含 writerow zip 行吗？

@NateWalker 是的，writerrow 应该在循环的末尾。

@NateWalker 别担心。