如何构造循环API调用以使JSON结果可用？

Posted 2023-01-16

我有一个API，我正在对此进行调用，该请求限制了每个请求50个结果以及每分钟180个请求。但是，一个对象总共可能有10到50,000个结果。我需要将生成的JSON构造为可用的格式，或者将其转换为我可以更轻松地访问的东西，例如pandas，我只是不确定哪个是更好的选择。计划是遍历需要数据的每个对象ID。我当前的API调用看起来像这样：

import requests, time
import json, pandas as pd

#object id's to iterate through
# object_id = ['13','16','95','93','77']
object_id = ['13']
#this block finds the count of entries
for obj in object_id:
  data = []
  info_url = 'https://api.ontraport.com/1/objects/getInfo?objectID='.format(obj)
  headers = 
    'Api-Appid': 'XXXXXXXXX',
    'Api-Key': 'XXXXXXXXX'
  
  response = requests.get(info_url, headers=headers).json()
  obj_count = int(response['data']['count'])
#this block ensures all entries are pulled as it iterates in groups of 50
  start = 1
  while start / obj_count <= 1:
    url = 'https://api.ontraport.com/1/objects?objectID=&start='.format(obj, start)
    headers = 
      'Api-Appid': 'XXXXXXXXX',
      'Api-Key': 'XXXXXXXXX'
    
    response = requests.get(url, headers=headers).json()
    data.append(response)
    start += 50
    time.sleep(0.34)
  with open('C:/Desktop/data.txt'.format(obj), 'w') as outfile:
    json.dump(data, outfile)

API产生以下结果：

   "code": 0,
   "data": [
              "id":"111",
              "date":"1441326063"
           ,
           
              "id":"132",
              "date":"1441526112"
           ],
   "account_id": 0

我只关心“数据”标签内的部分。但是，当我将结果附加在一起时，对于每个请求，它变成一个由JSON对象组成的大型JSON数组，并且在每个对象内，“数据”是另一个数组。是否可以将其构造为仅附加数据结果，然后格式化为熊猫数据框？

答案

我希望这项工作，有关更多信息，请查看List comprehension

import pandas as pd
import json

json_str = """[

   "code": 0,
   "data": [
              "id":"111",
              "date":"1441326063"
           ,
           
              "id":"132",
              "date":"1441526112"
           ],
   "account_id": 0
,

   "code": 0,
   "data": [
              "id":"111",
              "date":"1441326063"
           ,
           
              "id":"132",
              "date":"1441526112"
           ],
   "account_id": 0

]"""

json_dict = json.loads(json_str)

df = pd.DataFrame([d for y in json_dict for d in y['data']])

另一答案

如果只有1个数据块，您可以像使用json字典那样简单地保存它

data.append(response['Data'])