Numpy使用genfromtxt解析日期

Posted 2021-04-02

我想用numpy的genfromtxt加载一些csv数据。我正在努力使用时间字段的正确数据类型。对于这两个版本的parse_time，我得到了相同的错误

无法根据规则'same_kind'将元数据[us]中的datetime.datetime对象转换为

这是我的代码：

import numpy as np
import datetime as dt

parse_time  = lambda x: dt.datetime.strptime(x.decode('utf-8'), "%Y-%m-%dT%H:%M:%S.%fZ")
parse_time2 = lambda x: np.datetime64(dt.datetime.strptime(x.decode('utf-8'), '%Y-%m-%dT%H:%M:%S.%fZ'))
col_names = ['Time','Temperature','Humidity']
lines = ['2018-10-03T11:28:35.325Z;23.0;17.0', '2018-10-03T11:28:35.325Z;23.0;17.0']

np.genfromtxt(lines, delimiter=';',dtype=[('Time',"datetime64"),('Temperature','f'),('Humidity','f')], converters={"Time": parse_time2},names=col_names)

这是堆栈跟踪：

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-96-cd725618b291> in <module>
      7 lines = ['2018-10-03T11:28:35.325Z;23.0;17.0', '2018-10-03T11:28:35.325Z;23.0;17.0']
      8 
----> 9 a = np.genfromtxt(lines, delimiter=';',dtype=[('Time',"datetime64"),('Temperature','f'),('Humidity','f')], converters={"Time": parse_time},names=col_names)

~/.local/lib/python3.6/site-packages/numpy/lib/npyio.py in genfromtxt(fname, dtype, comments, delimiter, skip_header, skip_footer, converters, missing_values, filling_values, usecols, names, excludelist, deletechars, replace_space, autostrip, case_sensitive, defaultfmt, unpack, usemask, loose, invalid_raise, max_rows, encoding)
   2163                     output = np.array(data, dtype=dtype)
   2164             else:
-> 2165                 rows = np.array(data, dtype=[('', _) for _ in dtype_flat])
   2166                 output = rows.view(dtype)
   2167             # Now, process the rowmasks the same way

TypeError: Cannot cast datetime.datetime object from metadata [us] to  according to the rule 'same_kind'

答案

正如@hpaulj评论的那样，将数据类型更改为datetime64 [us]解决了它：

import numpy as np
import datetime as dt

parse_time  = lambda x: dt.datetime.strptime(x.decode('utf-8'), "%Y-%m-%dT%H:%M:%S.%fZ")
parse_time2 = lambda x: np.datetime64(dt.datetime.strptime(x.decode('utf-8'), '%Y-%m-%dT%H:%M:%S.%fZ'))
col_names = ['Time','Temperature','Humidity']
lines = ['2018-10-03T11:28:35.325Z;23.0;17.0', '2018-10-03T11:28:35.325Z;23.0;17.0']

np.genfromtxt(lines, delimiter=';',dtype=[('Time',"datetime64[us]"),('Temperature','f'),('Humidity','f')], converters={"Time": parse_time2},names=col_names)