Washing&Processing Data(JSON) with Python

Posted 2021-04-30 少年在木星

The USDA made a database about the information of food which was remade by a geek to JSON.This article aims at washing&processing Data(JSON) with Python.  

CODE：

#1st 
import pandas as pd
from pandas import DataFrame,Series
import json

db=json.load(open('pydata-book-master/ch07/foods-2011-10-03.json'))
info_keys=['description','group','id','manufacturer']
info=DataFrame(db,columns=info_keys)
#2nd

nutrients=[]
for rec in db:

   fnuts=DataFrame(rec['nutrients'])
   fnuts['id']=rec['id']
   nutrients.append(fnuts)
nutrients=pd.concat(nutrients,ignore_index=True)
nutrients=nutrients.drop_duplicates()

#3rd
col_mapping={'description':'food','group':'fgroup'}
info=info.rename(columns=col_mapping,copy=False)
col_mapping={'description':'nutrient','group':'nutgroup'}
nutrients=nutrients.rename(columns=col_mapping,copy=False)
ndata=pd.merge(nutrients,info,on='id',how='outer')

#4th
result=ndata.groupby(['nutrient','fgroup'])['value'].quantile(0.5)
result['Zinc, Zn'].plot()

#5th

by_nutrient=ndata.groupby(['nutgroup','nutrient','food'])
get_maximum=lambda x:x.xs(x.value.idxmax())
get_minimum=lambda x:x.xs(x.value.idxmin())
max_foods=by_nutrient.apply(get_maximum)[['value','food']]
max_foods.food=max_foods.food.str[:50]
max_foods.ix['Amino Acids']['food']

Explanation:

1st:1)use json/pands to load data

     2)choose exact 'keys' to form 'db' which we need to analyze

 

2nd:1)create a DataFrame of nutrients

      2)wash data:drop the duplicates

3rd:1)rename keys of the two 'DataFrame' so as to avoid confusion

      2）combine 'nutrients' with 'info' to form new data 'ndata'

4th:draw a picture for fun

5th:to get some data we need such as the food containing the most exact nutrient

Results:

ndata:

picture for fun:

to get some data:

    

（Python for Data Analysis）

少年在木星