我如何输入路径并在同一路径中创建所有 csv 文件的 hdf5 文件？

Posted 2023-03-11

【中文标题】我如何输入路径并在同一路径中创建所有 csv 文件的 hdf5 文件？

【英文标题】：how i can enter a path and create hdf5 file with all csv file in the same path?

【发布时间】：2021-01-26 14:28:26

【问题描述】：

我试过这个脚本，但我看不到 hdf5 文件，脚本运行没有错误，但我什么也没看到。

import glob
import os
import pandas as pd

# inputs
path = input('Insert the directory path:')
group = input('Insert a group name: ')

# create a list of file paths
file_list = [file for file in glob.glob(path)]
# dict comprehension to create keys from file name and values from the csv files
dfs = os.path.basename(os.path.normpath(filename)).split('.')[0]: pd.read_csv(filename) for filename in file_list

# loop though the dataframes
for k,df in dfs.items():
    # store the HDF5 file
    store = pd.HDFStore('test.h5')
    # append df to a group and assign the key with f-strings
    store.append(f'group/k', df, format='table', data_columns=df.columns)
    # close the file
    store.close()

【参考方案1】：

尝试添加存储路径：

store = pd.HDFStore('test.h5', mode='w')

【讨论】：

实际上，运行良好的脚本仍然是 hdf5 未创建的相同问题

我仍然想念我无法创建 hdf5 文件如果有任何其他建议将包含许多 csv 文件的路径转换为 ​​hdf5 文件请建议

【参考方案2】：

import glob
import os
import pandas as pd

# inputs
pattern = input('Insert the file pattern:')
group = input('Insert a group name: ')

# create a list of file paths
# file_list = [file for file in glob.iglob(path)]
# pattern exp：d:\test\*.csv
file_list = list(glob.iglob(pattern))
# dict comprehension to create keys from file name and values from the csv files
dfs = os.path.basename(os.path.normpath(filename)).split('.')[0]: pd.read_csv(filename) for filename in file_list

# store the HDF5 file
store = pd.HDFStore('test.h5')
# loop though the dataframes
for k,df in dfs.items():
    # append df to a group and assign the key with f-strings
    store.append(f'group/k', df, format='table', data_columns=df.columns)
    # close the file
store.close()