如何从数据框字典创建数据框？
Posted
技术标签:
【中文标题】如何从数据框字典创建数据框？【英文标题】：How to create a dataframe from a dictionary of dataframes? 【发布时间】：2019-05-27 05:59:40 【问题描述】：问题是我想从递归嵌套的 JSON 文件创建一个 Dataframe，我想出了自己的方法。如果您有其他更简单的方法，请告诉我
我在一个 json 文件中记录了 24 小时的航班数据，并想使用它创建一个数据框来形成一天的数据框。我可以通过每小时创建一个对象然后使用 append 合并所有对象来做到这一点，但这似乎是一个乏味的过程，即使是 24 小时，如果我有 1000 条记录以这种方式合并的问题怎么办。因此我想出了这个代码 - 我尝试遍历 JSON 文件以获取每小时数据并一次创建两个数据帧，使用 concat 合并它们。我稍后将这两个数据帧存储在字典中，其中的键值由变量 z 指示。当我打印它时，输出一次包含 2 条记录并打印 12 次，因此使其成为 24 小时内的所有数据，但是当我将数据存储在 csv 文件中时，仅存储与最后两个小时相对应的数据.
import pandas as pd
z = 0
result1 =
for i in range(0,24):
if (i%2)==0:
dataF = data["data"]["weather"][0]["hourly"][i]
dataF2 = data["data"]["weather"][0]["hourly"][i+1]
dataFrame = pd.DataFrame(dataF,index=[i+1])
dataFrame2 = pd.DataFrame(dataF2,index=[i+2])
frame = [dataFrame,dataFrame2]
result = pd.concat(frame)
#result.append(result)
#print(result1)
result1[z] = result
z += 1
print(result1)`
打印结果：
0: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC windspeedMiles \
1 11 46 NW 8 8 7
2 13 42 NW 6 7 8
DewPointF HeatIndexF cloudcover HeatIndexC ... \
1 47 49 100 10 ...
2 44 46 100 8 ...
weatherDesc visibility weatherCode tempC tempF \
1 'value': 'Patchy rain possible' 10 176 10 49
2 'value': 'Patchy rain possible' 10 176 8 46
WindChillF WindChillC winddirDegree humidity time
1 46 8 318 91 0
2 42 6 317 92 100
[2 rows x 25 columns], 1: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC windspeedMiles \
3 14 38 NW 3 5 9
4 16 33 NW 1 3 10
DewPointF HeatIndexF cloudcover HeatIndexC ... weatherDesc \
3 41 43 100 6 ... 'value': 'Overcast'
4 38 40 100 4 ... 'value': 'Overcast'
visibility weatherCode tempC tempF WindChillF WindChillC winddirDegree \
3 10 122 6 43 38 3 315
4 10 122 4 40 33 1 314
humidity time
3 92 200
4 93 300
[2 rows x 25 columns], 2: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC windspeedMiles \
5 17 33 NW 1 3 10
6 17 33 NW 0 3 11
DewPointF HeatIndexF cloudcover HeatIndexC ... weatherDesc \
5 38 40 100 4 ... 'value': 'Overcast'
6 38 39 100 4 ... 'value': 'Overcast'
visibility weatherCode tempC tempF WindChillF WindChillC winddirDegree \
5 10 122 4 40 33 1 314
6 10 122 4 40 33 0 315
humidity time
5 93 400
6 93 500
[2 rows x 25 columns], 3: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC windspeedMiles \
7 18 32 NW 0 3 11
8 18 33 NW 1 3 11
DewPointF HeatIndexF cloudcover HeatIndexC ... weatherDesc \
7 38 39 100 4 ... 'value': 'Overcast'
8 38 40 100 4 ... 'value': 'Overcast'
visibility weatherCode tempC tempF WindChillF WindChillC winddirDegree \
7 10 122 4 40 32 0 315
8 10 122 5 41 33 1 317
humidity time
7 93 600
8 93 700
[2 rows x 25 columns], 4: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC \
9 17 34 NW 1 4
10 17 35 NW 2 4
windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC ... \
9 11 38 40 100 5 ...
10 11 39 41 100 5 ...
weatherDesc visibility weatherCode tempC tempF WindChillF \
9 'value': 'Overcast' 10 122 5 42 34
10 'value': 'Overcast' 10 122 6 42 35
WindChillC winddirDegree humidity time
9 1 318 92 800
10 2 319 92 900
[2 rows x 25 columns], 5: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC \
11 17 36 NW 2 4
12 17 37 NW 3 4
windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC ... \
11 10 38 42 100 6 ...
12 10 38 43 100 6 ...
weatherDesc visibility weatherCode tempC tempF WindChillF \
11 'value': 'Overcast' 10 122 7 44 36
12 'value': 'Overcast' 10 122 7 45 37
WindChillC winddirDegree humidity time
11 2 318 88 1000
12 3 317 84 1100
[2 rows x 25 columns], 6: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC \
13 17 38 NW 3 3
14 17 38 NW 4 3
windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC ... \
13 10 38 44 100 7 ...
14 10 38 44 100 7 ...
weatherDesc visibility weatherCode tempC tempF WindChillF \
13 'value': 'Overcast' 10 122 8 47 38
14 'value': 'Overcast' 10 122 9 47 38
WindChillC winddirDegree humidity time
13 3 317 81 1200
14 4 317 79 1300
[2 rows x 25 columns], 7: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC \
15 17 39 NW 4 3
16 17 39 NW 4 3
windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC ... \
15 10 38 44 100 7 ...
16 11 37 45 100 7 ...
weatherDesc visibility weatherCode tempC tempF WindChillF \
15 'value': 'Overcast' 10 122 9 48 39
16 'value': 'Overcast' 10 122 10 49 39
WindChillC winddirDegree humidity time
15 4 317 76 1400
16 4 317 74 1500
[2 rows x 25 columns], 8: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC \
17 17 39 NW 4 3
18 17 39 NW 4 2
windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC ... \
17 11 37 44 100 7 ...
18 11 36 44 100 7 ...
weatherDesc visibility weatherCode tempC tempF WindChillF \
17 'value': 'Overcast' 10 122 9 48 39
18 'value': 'Overcast' 10 122 8 46 39
WindChillC winddirDegree humidity time
17 4 318 74 1600
18 4 318 73 1700
[2 rows x 25 columns], 9: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC \
19 17 38 NW 3 2
20 17 37 NW 3 2
windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC ... \
19 11 36 44 100 7 ...
20 11 35 43 100 6 ...
weatherDesc visibility weatherCode tempC tempF WindChillF \
19 'value': 'Overcast' 10 122 7 45 38
20 'value': 'Overcast' 10 122 7 44 37
WindChillC winddirDegree humidity time
19 3 319 73 1800
20 3 318 72 1900
[2 rows x 25 columns], 10: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC \
21 17 37 NW 3 1
22 16 36 NW 2 1
windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC ... \
21 10 34 43 100 6 ...
22 10 33 42 100 6 ...
weatherDesc visibility weatherCode tempC tempF WindChillF \
21 'value': 'Overcast' 10 122 6 43 37
22 'value': 'Overcast' 10 122 6 42 36
WindChillC winddirDegree humidity time
21 3 317 71 2000
22 2 316 71 2100
[2 rows x 25 columns], 11: windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC \
23 16 35 NW 2 1
24 16 35 NW 2 1
windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC ... \
23 10 33 41 95 5 ...
24 10 33 41 91 5 ...
weatherDesc visibility weatherCode tempC tempF WindChillF \
23 'value': 'Overcast' 10 122 5 41 35
24 'value': 'Overcast' 10 122 5 41 35
WindChillC winddirDegree humidity time
23 2 317 72 2200
24 2 318 74 2300
[2 rows x 25 columns]
【问题讨论】：
Pravin，您需要提供原始数据样本才能重现，尽管我在答案部分添加了一个借用的示例。 你可以检查一下 pandasread_json 函数是否可以帮助你解决这个pandas.pydata.org/pandas-docs/stable/generated/…
【参考方案1】：
只是为了你的例子：
如果您将整个 json 作为字典（或列表）加载，例如使用 json.load，你可以使用 json_normalize：
>>> import pandas as pd
>>> import json
>>> from pandas.io.json import json_normalize
数据：
>>> d = "response": "body": "contact": "email": "mr@abc.com", "mobile_number": "0123456789", "personal": "last_name": "Muster", "gender": "m", "first_name": "Max", "dob": "1985-12-23", "family_status": "single", "title": "Dr.", "customer": "verified": "true", "customer_id": "1234567", "token": "dsfgf", "version": "1.1"
>>> df = pd.io.json.json_normalize(d)
>>> df
response.body.contact.email response.body.contact.mobile_number response.body.customer.customer_id ... response.body.personal.title response.token response.version
0 mr@abc.com 0123456789 1234567 ... Dr. dsfgf 1.1
[1 rows x 12 columns]
>>> df.columns = df.columns.map(lambda x: x.split(".")[-1])
>>> df
email mobile_number customer_id verified dob family_status first_name gender last_name title token version
0 mr@abc.com 0123456789 1234567 true 1985-12-23 single Max m Muster Dr. dsfgf 1.1
看here
另一个post
【讨论】：
以上是关于如何从数据框字典创建数据框？的主要内容，如果未能解决你的问题，请参考以下文章