从提取的 JSON 数据中追加和/或写入 CSV

Posted 2023-03-11

【中文标题】从提取的 JSON 数据中追加和/或写入 CSV

【英文标题】：Appending and or writing to a CSV from pulled JSON data

【发布时间】：2019-04-11 11:59:45

【问题描述】：

下面编写的代码旨在从给定的 url 中提取 JSON 数据。（我已经删除了密钥）。然后我将其格式化并将其更改为 CSV。当这个程序运行和打印时，我看到第一行被提取，但是我打算附加的后续行只添加到第一行（df），而不是在下一行编译。有没有一种简单的方法可以完成？这样在每个 time.sleep() 周期完成后，该行将添加到前一个睡眠周期而不是原始 df。

df = pd.read_json('https://forex.1forge.com/1.0.3/quotes?pairs=EURUSD,EURJPY,GBPUSD,USDCAD,&api_key=KEY')
df = df.pivot_table('price', 'timestamp', 'symbol')
df.to_csv('datapull.csv')
df = pd.read_csv('datapull.csv', index_col='timestamp')

x = range(6)
for n in x:

  df2 = pd.read_json('https://forex.1forge.com/1.0.3/quotes?pairs=EURUSD,EURJPY,GBPUSD,USDCAD,&api_key=KEY')
  df2 = df2.pivot_table('price', 'timestamp', 'symbol')
  df2.to_csv('datapull2.csv')
  df2 = pd.read_csv('datapull2.csv', index_col='timestamp')
  df3 = df.append(df2)
  time.sleep(1)
  print(df3)  

输入 JSON 输入数据的示例是：

["symbol":"EURUSD","bid":1.13913,"ask":1.13913,"price":1.13913,"timestamp":1541703878,"symbol":"EURJPY","bid" :129.75,"ask":129.753,"price":129.7515,"timestamp":1541703878,"symbol":"GBPUSD","bid":1.30907,"ask":1.30908,"price":1.30907," timestamp":1541703878,"symbol":"USDCAD","bid":1.31059,"ask":1.31059,"price":1.31059,"timestamp":1541703878]

下面是上面代码的输出。

                       EURJPY   EURUSD   GBPUSD   USDCAD
timestamp                                               
2018-11-08 00:06:06  129.8615  1.14359  1.31276  1.31167
2018-11-08 00:06:06  129.8610  1.14359  1.31276  1.31167
                       EURJPY   EURUSD   GBPUSD   USDCAD
timestamp                                               
2018-11-08 00:06:06  129.8615  1.14359  1.31276  1.31167
2018-11-08 00:06:08  129.8605  1.14359  1.31277  1.31163
                       EURJPY   EURUSD   GBPUSD   USDCAD
timestamp                                               
2018-11-08 00:06:06  129.8615  1.14359  1.31276  1.31167
2018-11-08 00:06:09  129.8660  1.14362  1.31278  1.31156
                       EURJPY   EURUSD   GBPUSD   USDCAD
timestamp                                               
2018-11-08 00:06:06  129.8615  1.14359  1.31276  1.31167
2018-11-08 00:06:10  129.8725  1.14363  1.31289  1.31155
                       EURJPY   EURUSD   GBPUSD   USDCAD
timestamp                                               
2018-11-08 00:06:06  129.8615  1.14359  1.31276  1.31167
2018-11-08 00:06:12  129.8750  1.14363  1.31289  1.31155
                       EURJPY   EURUSD   GBPUSD   USDCAD
timestamp                                               
2018-11-08 00:06:06  129.8615  1.14359  1.31276  1.31167
2018-11-08 00:06:13  129.8735  1.14363  1.31289  1.31155

【问题讨论】：

创建minimal reproducible example，提供提取的数据样本，而不是无法重现问题的残缺代码。从文件而不是 HTTP 读取数据。

["symbol":"EURUSD","bid":1.13913,"ask":1.13913,"price":1.13913,"timestamp":1541703878,"symbol":"EURJPY ","bid":129.75,"ask":129.753,"price":129.7515,"timestamp":1541703878,"symbol":"GBPUSD","bid":1.30907,"ask":1.30908,"price ":1.30907,"timestamp":1541703878,"symbol":"USDCAD","bid":1.31059,"ask":1.31059,"price":1.31059,"timestamp":1541703878]

【参考方案1】：

您以错误的方式累积数据，因为您在每一轮中将第一行 (df) 附加到刚刚获取的行 (df2)。

每轮附加的结果 (df3) 会在每轮结束时被覆盖并丢失。

这个呢：

df = pd.read_json('https://forex.1forge.com/1.0.3/quotes?pairs=EURUSD,EURJPY,GBPUSD,USDCAD,&api_key=KEY')
df = df.pivot_table('price', 'timestamp', 'symbol')
df.to_csv('datapull.csv')
df = pd.read_csv('datapull.csv', index_col='timestamp')

x = range(6)
for n in x:

  df2 = pd.read_json('https://forex.1forge.com/1.0.3/quotes?pairs=EURUSD,EURJPY,GBPUSD,USDCAD,&api_key=KEY')
  df2 = df2.pivot_table('price', 'timestamp', 'symbol')
  df2.to_csv('datapull2.csv')
  df2 = pd.read_csv('datapull2.csv', index_col='timestamp')
  df = df.append(df2)
  time.sleep(1)
  print(df)  

即只需将 df 定义为自身的附加，以及刚刚获取的新行 (df2)。