迭代python中的特定csv行输出一个空白文件

Posted 2023-02-25

【中文标题】迭代python中的特定csv行输出一个空白文件

【英文标题】：Iterating over specific csv rows in python outputs a blank file

【发布时间】：2017-06-16 00:00:47

【问题描述】：

这里的python newb - 我正在尝试格式化一组发送给我的非常严重的csv，以便我可以将它们放入一个漂亮的postgres表中进行查询和分析。为了做到这一点，我首先使用 csv.writer 清理它们以删除包装每个条目的空白行和双引号。这是我的代码的样子：

import os
import csv
import glob
from itertools import islice

files = glob.glob('/Users/foo/bar/*.csv')

# Loop through all of the csv's  
for file in files:
    # Get the filename from the path
    outfile = os.path.basename(file)

    with open(file, 'rb') as inp, open('/Users/foo/baz/' + outfile, 'wb') as out:

        reader = csv.reader(inp)
        writer = csv.writer(out)
        for row in reader:
            if row:
                writer.writerow(row)
        out.close() 

它工作得非常好，并且完全符合我的要求。输出 csv 看起来很棒。接下来，我尝试从新清理的 csv 文件的开头和结尾删除一定数量的包含完全不必要的垃圾的行（省略前 8 行和最后 2 行）。由于我确实无法确定的原因，这部分代码的 csv 输出（缩进与前面的“with”块相同）完全为空：

with open('/Users/foo/baz/' + outfile, 'rb') as inp2, open('/Users/foo/qux/' + outfile, 'wb') as out2:
    writer2 = csv.writer(out2)
    reader2 = csv.reader(inp2)
    row_count = sum(1 for row in reader2)
    last_line_index = row_count - 3 
    for row in islice(reader2, 7, last_line_index):
            writer2.writerow(row)
    out2.close()

我知道由于我使用“with”，每个块末尾的 close() 是多余的 - 我在查看 here 后尝试了它作为一种方法。我还尝试将第二个“with”块放入另一个文件并在运行第一个“with”块后运行该文件，但仍然无济于事。非常感谢您的帮助！

另外，这是整个文件：

import os
import csv
import glob
from itertools import islice

files = glob.glob('/Users/foo/bar/*.csv')

# Loop through all of the csv's  
for file in files:
    # Get the filename from the path
    outfile = os.path.basename(file)

    with open(file, 'rb') as inp, open('/Users/foo/baz/' + outfile, 'wb') as out:

        reader = csv.reader(inp)
        writer = csv.writer(out)
        for row in reader:
            if row:
                writer.writerow(row)
        out.close() 

    with open('/Users/foo/baz/' + outfile, 'rb') as inp2, open('/Users/foo/qux/' + outfile, 'wb') as out2:
        writer2 = csv.writer(out2)
        reader2 = csv.reader(inp2)
        row_count = sum(1 for row in reader2)
        last_line_index = row_count - 3 
        for row in islice(reader2, 7, last_line_index):
                writer2.writerow(row)
        out2.close()

谢谢！

【参考方案1】：

有罪的一方是

row_count = sum(1 for row in reader2)

它从reader2 读取所有数据；现在，当您尝试for row in islice(reader2, 7, last_line_index) 时，您不会获得任何数据。

此外，您可能正在读取大量空白行，因为您将文件作为二进制文件打开；而是这样做

with open('file.csv', newline='') as inf:
    rd = csv.reader(inf)

【讨论】：

确实是这个问题！加一个速度！你太快了！.....

啊哈！我不知道阅读是一次性交易！非常感谢您的快速响应！

【参考方案2】：

您可以像这样快速修复代码（我评论了问题所在的行，正如@Hugh Bothwell 所说，您已经从变量reader2 中读取了所有数据）：

import os
import csv
import glob
from itertools import islice

files = glob.glob('/Users/foo/bar/*.csv')

# Loop through all of the csv's  
for file in files:
    # Get the filename from the path
    outfile = os.path.basename(file)

    with open(file, 'rb') as inp, open('/Users/foo/baz/' + outfile, 'wb') as out:

        reader = csv.reader(inp)
        writer = csv.writer(out)
        for row in reader:
            if row:
                writer.writerow(row)
        out.close() 

    with open('/Users/foo/baz/' + outfile, 'rb') as inp2, open('/Users/foo/qux/' + outfile, 'wb') as out2:
            writer2 = csv.writer(out2)
            reader2 = csv.reader(inp2)
            row_count = sum(1 for row in csv.reader(inp2)) #here you separately count the amount of rows without read the variable reader2
            last_line_index = row_count - 3 
            for row in islice(reader2, 7, last_line_index):
                    writer2.writerow(row)
            out2.close()

【讨论】：

我很感激，伙计！你的解决方案也很完美，Hugh 刚刚到达我的收件箱有点快:)