只将包含某个单词的 Excel sheet_names 读入 pandas 数据框

Posted 2023-03-11

【中文标题】只将包含某个单词的 Excel sheet_names 读入 pandas 数据框

【英文标题】：Only read Excel sheet_names containing a certain word into a pandas dataframe

【发布时间】：2021-10-11 16:20:15

【问题描述】：

我有很多报告要在 python 中编译成单个数据框。

此代码用于循环遍历我的目录并读取每个文件中工作表名称相同的所有报告文件...我在每个工作簿中有很多工作表但只想找到包含特定字符串的 sheet_names , '报告'。

import pandas as pd
from pathlib import Path
import os
import glob

pathsting= 'path/to/working/directory'
rootdir = Path(pathsting)
onlydirs = [f for f in os.listdir(rootdir) if os.path.isdir(os.path.join(rootdir, f))]

df0 = pd.DataFrame()
for direct in onlydirs:
    print(direct)
    dirpathstring = pathsting + '\\' + direct
    dirpath = Path(dirpathstring)
    onlyfiles = [f for f in os.listdir(dirpath) if os.path.isfile(os.path.join(dirpath, f))]
    for f in dirpath.glob("*Report.xlsm"):
        print(f.name)
        temp = pd.read_excel(f, sheet_name='Report')
        df0 = pd.concat([df0, temp])
display(df0)

现在假设随着时间的推移，报告的格式会发生变化，而不是 sheet_name='Report'，而是变为 sheet_name='XYZ Report'。我有很多报告，并且名称更改了几次。我不想在多个不同的循环中硬编码所有可能的报告名称。

我能够使用 glob 读取所有以“Report.xlsm”结尾的文件，但是是否有类似的方法可以读取包含文本“Report”而不是确切字符串的 sheet_names？

【参考方案1】：

试试：

import pandas as pd
import glob
import re

path = r'./files' # use your path
all_files = glob.glob(path + "/*.xlsm")

# case insensitive pattern for file names like blahReportblah or fooreportingss etc.  Modify as required if necessary.
pattern = r'(?i)(.*report.*)'

# create empty list to hold dataframes from sheets found
dfs = []

# for each file in the path above ending .xlsm
for file in all_files:
    #if the file name has the word 'report' or even 'rEpOrTs' in it
    if re.search(pattern, file):
        #open the file
        ex_file = pd.ExcelFile(file)
        #then for each sheet in that file
        for sheet in ex_file.sheet_names:
            #check if the sheet has 'RePORting' etc. in it
            if re.search(pattern, sheet):
                #if so create a dataframe (maybe parse_dates isn't required).  Tweak as required
                df = ex_file.parse(sheet, parse_dates=True)
                #add this new (temp during the looping) frame to the end of the list
                dfs.append(df)
            else:
                #if sheet doesn't have the word 'report' move on, nothing to see here
                continue
    else:
        #if file doesn't have the word 'report' move on, nothing to see here
        continue

#handle a list that is empty
if len(dfs) == 0:
    print('No file or sheets found.')
    #create a dummy frame
    df = pd.DataFrame()
#or have only one item/frame and get it out
elif len(dfs) == 1:
    df = dfs[0]
#or concatenate more than one frame together
else:
    df = pd.concat(dfs, ignore_index=True)
    df = df.reset_index(drop=True)

#check what you've got
print(df.head())

【讨论】：

感谢您的详细回复。这似乎是合理的。我会尝试这种方法。

【参考方案2】：

您需要编写一个函数来读取工作表名称以查看它是否包含“报告”一词。这可以帮助您获得工作表名称：

How to obtain sheet names from XLS files without loading the whole file?