通过 python 将 csv 文件插入 MySQL。运行但数据未填充到表中

Posted

技术标签:

【中文标题】通过 python 将 csv 文件插入 MySQL。运行但数据未填充到表中【英文标题】:INSERT csv file into MySQL via python. Runs but data does not populate in Table 【发布时间】:2020-08-28 19:48:05 【问题描述】:

首先,我对 python 很陌生,所以请善待。 我正在尝试获取从位于我的驱动器上的文件夹中的 SQL 查询生成的 CSV 文件,并通过 python 将该文件上的数据“上传”/插入到 mysql 表中。我不想简单地手动上传文件。该脚本运行并显示数据处理,但是当我进入表格并尝试搜索它时,什么也没有出现。以下是我写的:

cursor = cnx.cursor()

if cnx.is_connected():
    print('Successfully connected to MySQL database')

cursor.execute("create database if not exists Character_Stats;")

cursor.execute("use Character_Stats;")

DB_NAME = 'Character_Stats_DB'
TABLES = 
cursor.execute('DROP TABLE IF EXISTS Local_Players')

cursor.execute("""CREATE TABLE IF NOT EXISTS `Local_Players` (
  `game_account_id` int,
  `character_key` char(20) DEFAULT NULL,
  `game_region_key` int(20) DEFAULT NULL,
  `character_id` int(20) DEFAULT NULL,
  `class_name` TEXT(20) DEFAULT NULL,
  `specialization_id` int(20) DEFAULT NULL,
  `character_name` TEXT(20) DEFAULT NULL,
  `name` TEXT DEFAULT NULL,
  `character_level` int(20) DEFAULT NULL,
  `total_dmg` int(20) DEFAULT NULL,
  `avg_dmg_per_game` int(20) DEFAULT NULL,
  `total_healing` int(20) DEFAULT NULL,
  `total_dmg_absorbed` int(20) DEFAULT NULL,
  `total_heal_absorbed` int(20) DEFAULT NULL,
  `avg_heal_absorbed_per_game` int(20) DEFAULT NULL,
  `total_killing_blows` int(20) DEFAULT NULL,
  `avg_killing_blows_per_game` int(20) DEFAULT NULL,
  `total_deaths` int(20) DEFAULT NULL,
  `avg_deaths_per_game` int(20) DEFAULT NULL,
  `total_time_hrs` int(20) DEFAULT NULL,
  `total_games_played` int(20) DEFAULT NULL,
  PRIMARY KEY (game_account_id))
  ENGINE = InnoDB""")

cnx.commit()

for data in query:
    with open('Player_stats.csv', 'r', encoding="utf-8", newline='') as playerdata:
        csv_reader = csv.reader(playerdata, delimiter=',')
        for row in csv_reader:
            print(row)
            character_key = row[0]
            game_region_key = row[1]
            character_id = row[2]
            class_name = row[3]
            character_name = row[4]
            realm_key = row[5]
            name = row[6]
            character_level = row[7]
            total_dmg = row[8]
            avg_dmg_per_game = row[9]
            total_healing = row[10]
            avg_healing_per_game = row[11]
            total_dmg_absorbed = row[12]
            total_heal_absorbed = row[13]
            avg_heal_absorbed_per_game = row[14]
            total_killing_blows = row[15]
            avg_killing_blows_per_game = row[16]
            total_deaths = row[17]
            avg_deaths_per_game = row[18]
            total_time_hrs = row[19]
            total_games_played = row[20]
            cursor.execute('''INSERT INTO Character_Stats.Local_Players(character_key,game_region_key
        , character_id, class_name, character_name, realm_key, name, character_level, total_dmg, avg_dmg_per_game, 
        total_healing, avg_healing_per_game, total_dmg_absorbed, total_heal_absorbed, avg_heal_absorbed_per_game, 
        total_killing_blows, avg_killing_blows_per_game, total_deaths, avg_deaths_per_game, total_time_hrs, total_games_played),
        VALUES(%s,%s,%s, %s,%s, %s,%s, %s,%s, %s,%s, %s,%s, %s,%s, %s,%s, %s,%s, %s,%s)''', multi=True)
            print(data)


print("Data loading complete. \n")

任何帮助将不胜感激。

这是我的 CSV 的输出:

【问题讨论】:

你介意我告诉你另一种方法吗,这更容易。你能显示你的 Player_stats.csv 吗? @ashishmishra 我已经附上了我的 CSV 文件 我的代码正在运行,如果出现任何错误,您可以执行它并 lmk。 代码完美运行!谢谢! 您需要在完成所有插入后致电cnx.commit() 【参考方案1】:

此代码将使用 csv 文件填充表格中的所有列。只需尝试阅读这些模块的文档,因为 Python 是一种非常智能且简单的语言,带有库:

安装 pandas、pymsql 和 sqlalchemy:

pip install sqlalchemy
pip install pymysql
pip install pandas

试试这个:

import pandas as pd
from sqlalchemy import create_engine
connection_string = "mysql+pymysql://%s:%s@%s:%s/%s" % ('user_name','ur_connection_password' , 'host', PORT, 'schema_name')
engine = create_engine(connection_string)
connection = engine.connect()
connection.execute("""
CREATE TABLE IF NOT EXISTS `Local_Players` (
  `game_account_id` int NOT NULL AUTO_INCREMENT,
  `character_key` varchar(20) DEFAULT NULL,
  `game_region_key` int(20) DEFAULT NULL,
  `character_id` int(20) DEFAULT NULL,
  `class_name` TEXT(20) DEFAULT NULL,
  `specialization_id` int(20) DEFAULT NULL,
  `character_name` TEXT(20) DEFAULT NULL,
  `name` TEXT DEFAULT NULL,
  `character_level` int(20) DEFAULT NULL,
  `total_dmg` int(20) DEFAULT NULL,
  `avg_dmg_per_game` int(20) DEFAULT NULL,
  `total_healing` int(20) DEFAULT NULL,
  `total_dmg_absorbed` int(20) DEFAULT NULL,
  `total_heal_absorbed` int(20) DEFAULT NULL,
  `avg_heal_absorbed_per_game` int(20) DEFAULT NULL,
  `total_killing_blows` int(20) DEFAULT NULL,
  `avg_killing_blows_per_game` int(20) DEFAULT NULL,
  `total_deaths` int(20) DEFAULT NULL,
  `avg_deaths_per_game` int(20) DEFAULT NULL,
  `total_time_hrs` int(20) DEFAULT NULL,
  `total_games_played` int(20) DEFAULT NULL,
  PRIMARY KEY (`game_account_id`))""")
pd.read_sql_table(table_name='Local_Players',schema='Test_DB', con=engine)
df = pd.read_csv(r"Player_stats.csv")
df.to_sql(name="Local_Players", if_exists='append', chunksize=1000, con=engine, index=False)

【讨论】:

【参考方案2】:

由于您的字符串格式错误,没有插入任何内容

cursor.execute('''INSERT INTO Character_Stats.Local_Players(character_key,game_region_key
        , character_id, class_name, character_name, realm_key, name, character_level, total_dmg, avg_dmg_per_game, 
        total_healing, avg_healing_per_game, total_dmg_absorbed, total_heal_absorbed, avg_heal_absorbed_per_game, 
        total_killing_blows, avg_killing_blows_per_game, total_deaths, avg_deaths_per_game, total_time_hrs, total_games_played),
        VALUES(%s,%s,%s, %s,%s, %s,%s, %s,%s, %s,%s, %s,%s, %s,%s, %s,%s, %s,%s, %s,%s)''' %(character_key,game_region_key
        , character_id, class_name, character_name, realm_key, name, character_level, total_dmg, avg_dmg_per_game, 
        total_healing, avg_healing_per_game, total_dmg_absorbed, total_heal_absorbed, avg_heal_absorbed_per_game, 
        total_killing_blows, avg_killing_blows_per_game, total_deaths, avg_deaths_per_game, total_time_hrs, total_games_played), multi=True)

你忘了赋值,测试一下,你现在应该可以看到数据了。

【讨论】:

我试过这个并收到相同的结果。脚本运行,我可以看出它正在提取数据,但是当我进入我的表时它不存在。

以上是关于通过 python 将 csv 文件插入 MySQL。运行但数据未填充到表中的主要内容,如果未能解决你的问题,请参考以下文章

python 从csv文件插入mysql数据库

使用python将数据从csv文件插入oracle时出现错误ORA-01722:无效数字

将特定信息从 CSV 插入 SQL python3

将 csv 文件加载到 BIGquery 中并在插入数据时添加日期列

批量插入数据, 将DataTable里的数据批量写入数据库的方法

通过 bigquery-python 库向 BigQuery 插入大量数据