Pandas：合并数据框并将多个连接值合并到一个数组中

Posted 2023-03-11

【中文标题】Pandas：合并数据框并将多个连接值合并到一个数组中

【英文标题】：Pandas: merge dataframes and consolidate multiple joined values into an array

【发布时间】：2017-12-02 07:32:12

【问题描述】：

我是 Python 的新手，我正在使用 Pandas 将一堆 mysql 表转换为 JSON。我当前的解决方案工作得很好，但是（1）它不是很pythonic，并且（2）我觉得必须有一些预先烘焙的Pandas fucntion来满足我的需要......？以下问题的任何指导都会有所帮助。

假设我有两个数据框，authors 和一个连接表 plays_authors，它表示作者与戏剧之间的 1:many 关系。

print authors
>   author_id  dates notes
> 0         1  1700s     a 
> 1         2  1800s     b 
> 2         3  1900s     c 


print plays_authors
>      author_id play_id
> 0         1      12
> 1         1      13
> 2         1      21
> 3         2      18
> 4         3       3
> 5         3       7

我想将plays_authors 合并到authors，但不是每个作者有多行（每个play_id 1 个），我希望每个作者有一行，并带有一组play_id 值，以便我可以轻松将它们导出为 json 记录。

print authors
>   author_id  dates notes       play_id
> 0         1  1700s     a  [12, 13, 21]
> 1         2  1800s     b          [18]
> 2         3  1900s     c        [3, 7]

authors.to_json(orient="records")
> '[
>    "author_id":"1",
>    "dates":"1700s",
>    "notes":"a",
>    "play_id":["12","13","21"]
>   ,
>   
>    "author_id":"2",
>    "dates":"1800s",
>    "notes":"b",
>    "play_id":["18"]
>   ,
>   
>    "author_id":"3",
>    "dates":"1900s",
>    "notes":"c",
>    "play_id":["3","7"]
>  ]'

我目前的解决方案：

# main_df: main dataframe to transform
# join_df: the dataframe of the join table w values to add to df
# main_index: name of main_df index column
# multi_index: name of column w/ multiple values per main_index, added by merge with join_df
# jointype: type of merge to perform, e.g. left, right, inner, outer

def consolidate(main_df, join_df, main_index, multi_index, jointype):
    # merge
    main_df = pd.merge(main_df, join_df, on=main_index, how=jointype)
    # consolidate
    new_df = pd.DataFrame()

    for i in main_df[main_index].unique():
        i_rows = main_df.loc[main_df[main_index] == i]
        values = []

        for column in main_df.columns:
            values.append(i_rows[:1][column].values[0])

        row_dict = dict(zip(main_df.columns, values))
        row_dict[multi_index] = list(i_rows[multi_index])
        new_df = new_df.append(row_dict, ignore_index=True)

    return new_df


authors = consolidate(authors, plays_authors, 'author_id', 'play_id', 'left')

是否有一个简单的 groupby / 更好的 dict 解决方案目前就在我头上？

【参考方案1】：

数据：

In [131]: a
Out[131]:
   author_id  dates notes
0          1  1700s     a
1          2  1800s     b
2          3  1900s     c

In [132]: pa
Out[132]:
   author_id  play_id
0          1       12
1          1       13
2          1       21
3          2       18
4          3        3
5          3        7

解决方案：

In [133]: a.merge(pa.groupby('author_id')['play_id'].apply(list).reset_index())
Out[133]:
   author_id  dates notes       play_id
0          1  1700s     a  [12, 13, 21]
1          2  1800s     b          [18]
2          3  1900s     c        [3, 7]

【讨论】：

完美！谢谢！