如何从 cvs 连续数据集中计算平均值/最小值?
Posted
技术标签:
【中文标题】如何从 cvs 连续数据集中计算平均值/最小值?【英文标题】:How can I calculate the mean value/ min from a cvs continous dataset? 【发布时间】:2016-07-31 11:59:58 【问题描述】:我是 python 新手,这是我的第一个问题,如有错误请见谅。
我有一个连续测量的大 csv 文件(大约每秒测量一次,但间隔不固定)。我需要得到每分钟的平均值。我发现 groupby 可能会帮助我这样做,但我坚持将 DATE_TIME 列指定为索引和 dtype'datetime'。 csv 文件如下所示:
,DATE_TIME,N2O_dry 0,2016-03-01 02:32:02.651,0.70714453962 1,2016-03-01 02:32:03.762,0.7071444254000001 2,2016-03-01 02:32:05.257,0.70373171894 3,2016-03-01 02:32:05.953,0.70083729096 4,2016-03-01 02:32:07.049,0.69760065648 5,2016-03-01 02:32:07.928,0.6954438788699999 6,2016-03-01 02:32:08.726,0.6874527606899999 7,2016-03-01 02:32:10.005,0.6724201105500001 8,2016-03-01 02:32:10.851,0.6607286568199999 . . . 104503,2016-03-02 08:21:18.421,0.26879397415 104504,2016-03-02 08:21:19.532,0.26884030311 104505,2016-03-02 08:21:20.359,0.26887979686
到目前为止,我只成功地读取了数据框中的文件并将 DATE_TIME 列指定为索引,并将 DATE_TIME 列设为 dtype='datetime64[ns]' 对象:
import pandas
df=pandas.read_csv(file,usecols=[1,'N2O_dry'])
df=df.set_index('DATE_TIME')
df=pandas.to_datetime(df.index)
但是,现在我似乎只剩下 DATE_TIME 列了。有人可以帮帮我吗?
`
【问题讨论】:
【参考方案1】:我认为您可以将参数parse_dates
和index_col
添加到read_csv
,然后将resample
与mean
一起使用(这适用于pandas 0.18.0
):
import pandas as pd
import io
temp=u""",DATE_TIME,N2O_dry
0,2016-03-01 02:32:02.651,0.70714453962
1,2016-03-01 02:32:03.762,0.7071444254000001
2,2016-03-01 02:32:05.257,0.70373171894
3,2016-03-01 02:32:05.953,0.70083729096
4,2016-03-01 02:32:07.049,0.69760065648
5,2016-03-01 02:32:07.928,0.6954438788699999
6,2016-03-01 02:32:08.726,0.6874527606899999
7,2016-03-01 02:32:10.005,0.6724201105500001
8,2016-03-01 02:32:10.851,0.6607286568199999"""
#after testing replace io.StringIO(temp) to filename
df = pd.read_csv(io.StringIO(temp),
usecols=[1,'N2O_dry'],
parse_dates=['DATE_TIME'],
index_col=['DATE_TIME'])
print df
N2O_dry
DATE_TIME
2016-03-01 02:32:02.651 0.707145
2016-03-01 02:32:03.762 0.707144
2016-03-01 02:32:05.257 0.703732
2016-03-01 02:32:05.953 0.700837
2016-03-01 02:32:07.049 0.697601
2016-03-01 02:32:07.928 0.695444
2016-03-01 02:32:08.726 0.687453
2016-03-01 02:32:10.005 0.672420
2016-03-01 02:32:10.851 0.660729
print df.resample('1Min').mean()
N2O_dry
DATE_TIME
2016-03-01 02:32:00 0.6925
【讨论】:
谢谢! read_csv 的参数完美运行! resample 函数并不完全符合我的要求,因为我试图获取时间序列中每一分钟的平均值,而这个函数似乎给了我每分钟的整体平均值(输出只是一个值) .我想我在表达我的问题时不够具体。如果有任何进一步的帮助,我将不胜感激,但我也许可以从这里开始! 嗯,也许最好写Minimal, Complete, and Verifiable example 和所需的输出。【参考方案2】:如果我理解正确,请使用
df.index = pd.to_datetime(df.index)
而不是
df = pd.to_datetime(df.index)
这应该解决问题,只剩下DATE_TIME
列。
然后你得到(在 iPython 中):
In [27]:df.index
Out[27]:
DatetimeIndex(['2016-03-01 02:32:02.651000', '2016-03-01 02:32:03.762000',
'2016-03-01 02:32:05.257000', '2016-03-01 02:32:05.953000',
'2016-03-01 02:32:07.049000', '2016-03-01 02:32:07.928000',
'2016-03-01 02:32:08.726000', '2016-03-01 02:32:10.005000',
'2016-03-01 02:32:10.851000'],
dtype='datetime64[ns]', name=u'DATE_TIME', freq=None)
但还是:
In [26]: df
Out[26]:
N2O_dry
DATE_TIME
2016-03-01 02:32:02.651 0.707145
2016-03-01 02:32:03.762 0.707144
2016-03-01 02:32:05.257 0.703732
2016-03-01 02:32:05.953 0.700837
2016-03-01 02:32:07.049 0.697601
2016-03-01 02:32:07.928 0.695444
2016-03-01 02:32:08.726 0.687453
2016-03-01 02:32:10.005 0.672420
2016-03-01 02:32:10.851 0.660729
【讨论】:
@vera 太好了,很高兴我能帮上忙。考虑接受这个答案(点击勾号),这样它就不会出现在未回答的问题中。以上是关于如何从 cvs 连续数据集中计算平均值/最小值?的主要内容,如果未能解决你的问题,请参考以下文章