如何使用 Python 将嵌套字典/json 插入 MySQL 表

Posted 2023-03-23

【中文标题】如何使用 Python 将嵌套字典/json 插入 MySQL 表

【英文标题】：How to insert nested dictionary/json into MySQL table with Python

【发布时间】：2020-03-04 08:58:52

【问题描述】：

我已经像这样连接到https://exchangeratesapi.io/ api：

import requests, json, pymysql
from db_credentials import db_config

get_url = "https://api.exchangeratesapi.io/history?start_at=2018-01-01&end_at=2018-02-02&base=GBP"
response = requests.get(get_url)
results = response.content
data = json.loads(results.decode('utf-8'))['rates']

print(data.keys())
print(data)

data.keys() 和 data 看起来像这样：

dict_keys(['2018-01-22', '2018-01-09', '2018-01-24', '2018-01-03', '2018-01-23', '2018-01-25', '2018-01-18', '2018-02-02', '2018-01-15', '2018-01-02', '2018-01-16', '2018-01-11', '2018-01-17', '2018-01-12', '2018-01-30', '2018-02-01', '2018-01-29', '2018-01-08', '2018-01-10', '2018-01-05', '2018-01-04', '2018-01-26', '2018-01-31', '2018-01-19'])

'2018-01-02': 'AUD': 1.7327127809,
                'BGN': 2.1986891954,
                'BRL': 4.440996931,
                'CAD': 1.7006733893,
                ...,
 '2018-01-03': 'AUD': 1.730482852,
                'BGN': 2.2064530686,
                'BRL': 4.4264440433,
                ...,
 ...

我有一个空的 MySQL 表，其中包含日期、货币和汇率列，我想使用以下代码行中的内容填充上面的内容：

connection = pymysql.connect(**db_config, cursorclass=pymysql.cursors.DictCursor)
with connection.cursor() as cursor:
    insert_statement = "INSERT INTO gbpExchangeRates VALUES (%s)" % data
    cursor.execute(insert_statement)
connection.commit()
connection.close()

显然它不起作用，需要事先进行一些操作...... 我试过data = json.dumps(data)，但不知道下一步该做什么......

【参考方案1】：

你不能简单地将 JSON 扔到数据库中并期望它像那样工作。您必须将其展平为准备插入的行。

看看我用SQLite写的例子。它使用相同的DB API v2，因此它应该与您的 MySQL 驱动程序相当兼容。

import requests
import json
import sqlite3

get_url = "https://api.exchangeratesapi.io/history?start_at=2018-01-01&end_at=2018-02-02&base=GBP"
response = requests.get(get_url)
results = response.content
data = json.loads(results.decode('utf-8'))['rates']

if data:
    conn = sqlite3.connect('rates.db')

    c = conn.cursor()

    # here I simply created an example table containing all the fields you specified above
    c.execute("""
        CREATE TABLE exchange_rates (
            "id" INTEGER PRIMARY KEY,
            "date" DATE, 
            "currency" TEXT, 
            "rate" NUMERIC
        )
    """)

    c.executemany(
        """INSERT INTO exchange_rates ("date", "currency", "rate") VALUES (?,?,?)""",
        # this is where you iterate over the data and flatten it for SQL
        ((date, currency, value) for currency, value in rates.items() for date, rates in data.items())
    )

    conn.commit()

    conn.close()

【讨论】：

获取每个日期和货币的重复汇率

【参考方案2】：

以 techouse 答案为基础。

删除了列名周围的引号，即插入语句中的日期、货币、汇率 将((date, currency, value) for currency, value in rates.items() for date, rates in data.items()) 替换为insert_data

...

insert_data = []
for date, rates in data.items():
    for currency, value in rates.items():
        list_data = []
        list_data.append(date)
        list_data.append(currency)
        list_data.append(value)
        insert_data.append(list_data)

insert_data