大熊猫的条件前向填充

Posted 2023-02-16

【中文标题】大熊猫的条件前向填充

【英文标题】：Conditional forward fill in pandas

【发布时间】：2018-06-27 01:37:01

【问题描述】：

我有一个数据框：

>>> k
Out[87]: 
      Date        S       E           cp  Last         Q            code
30 2017-11-10   22500 2017-11-17       P 170.00     828.47  11/17/2017P22500
32 2017-11-10   22625 2017-11-17       P 180.00     646.91  11/17/2017P22625
35 2017-11-10   22750 2017-11-17       C 145.00     651.84  11/17/2017C22750
36 2017-11-13   22500 2017-11-17       P 245.00        nan  11/17/2017P22500
38 2017-11-13   22625 2017-11-17       P 315.00        nan  11/17/2017P22625
41 2017-11-13   22750 2017-11-17       C  35.00        nan  11/17/2017C22750
42 2017-11-14   22500 2017-11-17       P 215.00        nan  11/17/2017P22500
44 2017-11-14   22625 2017-11-17       P 305.00        nan  11/17/2017P22625
47 2017-11-14   22750 2017-11-17       C  26.00        nan  11/17/2017C22750
48 2017-11-15   22500 2017-11-17       P 490.00        nan  11/17/2017P22500
50 2017-11-15   22625 2017-11-17       P 605.00        nan  11/17/2017P22625
53 2017-11-15   22750 2017-11-17       C   4.00        nan  11/17/2017C22750
54 2017-11-16   22500 2017-11-17       P 140.00        nan  11/17/2017P22500
56 2017-11-16   22625 2017-11-17       P 295.00        nan  11/17/2017P22625
59 2017-11-16   22750 2017-11-17       C   4.00        nan  11/17/2017C22750
60 2017-11-17   22250 2017-11-24       P 165.00     707.57  11/24/2017P22250
61 2017-11-17   22375 2017-11-24       P 195.00     607.16  11/24/2017P22375
65 2017-11-17   22500 2017-11-24       C 175.00     666.56  11/24/2017C22500
66 2017-11-20   22250 2017-11-24       P 175.00        nan  11/24/2017P22250
67 2017-11-20   22375 2017-11-24       P 225.00        nan  11/24/2017P22375
71 2017-11-20   22500 2017-11-24       C  75.00        nan  11/24/2017C22500
72 2017-11-21   22250 2017-11-24       P  70.00        nan  11/24/2017P22250
73 2017-11-21   22375 2017-11-24       P 120.00        nan  11/24/2017P22375
77 2017-11-21   22500 2017-11-24       C  95.00        nan  11/24/2017C22500
78 2017-11-22   22250 2017-11-24       P  15.00        nan  11/24/2017P22250
79 2017-11-22   22375 2017-11-24       P  35.00        nan  11/24/2017P22375
83 2017-11-22   22500 2017-11-24       C 125.00        nan  11/24/2017C22500
84 2017-11-24   22375 2017-12-01       P 140.00     834.13  12/01/2017P22375
85 2017-11-24   22500 2017-12-01       P 185.00     763.76  12/01/2017P22500
89 2017-11-24   22625 2017-12-01       C 165.00     750.45  12/01/2017C22625

我想根据 code 列在 Q 列中填写 nans。例如索引为 30 的行中的代码与第 36 行中的代码相同，因此我想将相同的 Q 放在那里。

我目前是这样做的，有没有更好的方法？

k1= k.drop(['Date','S','E','cp','Last'],axis=1).dropna()
k1.columns =['Q_new', 'code']
k2 = k.merge(k1, on = 'code')
k2= k2.drop(['Q'],axis=1)
k2 = k2.sort('Date')

【问题讨论】：

没有运行循环？向我们展示您的循环解决方案以及预期的输出

忽略循环，我在上面介绍了另一种方式

顺便说一句，对于工作代码，您应该首先尝试codereview.stackexchange.com/questions/tagged/pandas

好的，谢谢，下次试试。

【参考方案1】：

groupby + ffill 和 bfill

df.Q=df.groupby('code').Q.apply(lambda x : x.ffill().bfill())
df
Out[755]: 
          Date      S           E cp   Last       Q              code
30  2017-11-10  22500  2017-11-17  P  170.0  828.47  11/17/2017P22500
32  2017-11-10  22625  2017-11-17  P  180.0  646.91  11/17/2017P22625
35  2017-11-10  22750  2017-11-17  C  145.0  651.84  11/17/2017C22750
36  2017-11-13  22500  2017-11-17  P  245.0  828.47  11/17/2017P22500
38  2017-11-13  22625  2017-11-17  P  315.0  646.91  11/17/2017P22625
41  2017-11-13  22750  2017-11-17  C   35.0  651.84  11/17/2017C22750
42  2017-11-14  22500  2017-11-17  P  215.0  828.47  11/17/2017P22500
44  2017-11-14  22625  2017-11-17  P  305.0  646.91  11/17/2017P22625
47  2017-11-14  22750  2017-11-17  C   26.0  651.84  11/17/2017C22750
48  2017-11-15  22500  2017-11-17  P  490.0  828.47  11/17/2017P22500
50  2017-11-15  22625  2017-11-17  P  605.0  646.91  11/17/2017P22625
53  2017-11-15  22750  2017-11-17  C    4.0  651.84  11/17/2017C22750
54  2017-11-16  22500  2017-11-17  P  140.0  828.47  11/17/2017P22500
56  2017-11-16  22625  2017-11-17  P  295.0  646.91  11/17/2017P22625
59  2017-11-16  22750  2017-11-17  C    4.0  651.84  11/17/2017C22750
60  2017-11-17  22250  2017-11-24  P  165.0  707.57  11/24/2017P22250
61  2017-11-17  22375  2017-11-24  P  195.0  607.16  11/24/2017P22375
65  2017-11-17  22500  2017-11-24  C  175.0  666.56  11/24/2017C22500
66  2017-11-20  22250  2017-11-24  P  175.0  707.57  11/24/2017P22250
67  2017-11-20  22375  2017-11-24  P  225.0  607.16  11/24/2017P22375
71  2017-11-20  22500  2017-11-24  C   75.0  666.56  11/24/2017C22500
72  2017-11-21  22250  2017-11-24  P   70.0  707.57  11/24/2017P22250
73  2017-11-21  22375  2017-11-24  P  120.0  607.16  11/24/2017P22375
77  2017-11-21  22500  2017-11-24  C   95.0  666.56  11/24/2017C22500
78  2017-11-22  22250  2017-11-24  P   15.0  707.57  11/24/2017P22250
79  2017-11-22  22375  2017-11-24  P   35.0  607.16  11/24/2017P22375
83  2017-11-22  22500  2017-11-24  C  125.0  666.56  11/24/2017C22500
84  2017-11-24  22375  2017-12-01  P  140.0  834.13  12/01/2017P22375
85  2017-11-24  22500  2017-12-01  P  185.0  763.76  12/01/2017P22500
89  2017-11-24  22625  2017-12-01  C  165.0  750.45  12/01/2017C22625

【讨论】：

为什么同时使用 ffill() 和 bfill()？你的意思是 ffill() 还是 bfill()？

@Florent 只需确保根本没有NaN，ffill 或 bfill 都有机会不填充 NaN

【参考方案2】：

您可以在 groupby 对象上使用transform。

df.loc[:, 'Q'] = df.groupby('code')['Q'].transform(lambda group: group.ffill())

时间安排

%timeit -n 1000 df.loc[:, 'Q'] = df.groupby('code')['Q'].transform(lambda group: group.ffill())
# 1000 loops, best of 3: 2.41 ms per loop

%timeit -n 1000 df.loc[:, 'Q'] = df.groupby('code')['Q'].ffill()
# 1000 loops, best of 3: 3.66 ms per loop

【讨论】：

实际上，我不会支持填充价格。我会修改的。