查询期间丢失与 MySQL 服务器的连接 - Python、MySql

Posted 2023-03-23

【中文标题】查询期间丢失与 MySQL 服务器的连接 - Python、MySql

【英文标题】：Lost connection to MySQL server during query - Python, MySql

【发布时间】：2020-07-11 13:45:10

【问题描述】：

我正在尝试使用 pymysql 从 Python 向 mysql 发送 INSERT 语句。如果我找到一个特定的文件，我会根据某些条件将它的内容插入到它在数据库中的相应表中。目前我有两张表 CACCT 和 PAUTH。对于特定于表的 INSERT，我有单独的函数。功能是：

 1. load_json_sql_CACCT()
 2. load_json_sql_PAUTH()

我的主要函数是recon()。

mysql_conn_dev = mysql.connect(host="xxxx",
                     user="xxx",
                     db="xxx")

cur = mysql_conn_dev.cursor(mysql.cursors.DictCursor)

tables = ["CACCT", "PAUTH"]

def recon():
    global unzipped_flg
    unzipped_flg = ''
    for item in tables:
        name_pattern = item + '_' + recon_date + '.*'
        for subdir, dirs, files in os.walk(src):
            for file in files:
                filepath = subdir + os.sep + file
                if fnmatch.fnmatch(file, name_pattern) and os.stat(filepath).st_size != 0:
                    if not result_match.empty:

                        if item == 'PAUTH':
                            load_json_sql_PAUTH(item, src + item + '_MISSING_RECORDS_' + recon_date + '.json')
                        elif item == 'CACCT':
                            load_json_sql_CACCT(item, src + item + '_MISSING_RECORDS_' + recon_date + '.json')
                    else:
                        print('No mismatch for ' + item)

def load_json_sql_CACCT(item, json_file):
    missing_record_list = []
    with open(json_file) as f:
        for json_obj in f:
            data_dict = json.loads(json_obj)
            missing_record_list.append(data_dict)
    print(json_file)
    for record in missing_record_list:
        print(record)

        if item == 'CACCT':
            with mysql_conn_dev.cursor() as cursor:
                cur.execute(
                r"INSERT INTO sayyed_sandpit.CACCT_UNI (instanceNumber,classNumber,parentClassNumber,subLevel,subId,edition,oid,u_acct_no,u_astatus,u_cust_no,u_myGlobal,u_preferred,u_status) "
                r"VALUES ("
                r"CAST(substring_index(substring_index(%s, '.', 2), '.', -1) as unsigned integer)"
                r",substring_index(%s, '.', 1)"
                r",0"
                r",0"
                r",0"
                r",%s"
                r",CONCAT(%s, '.0.0.0')"
                r",%s"
                r",%s"
                r",%s"
                r",CASE WHEN %s is not null then CONCAT(TRIM(LEADING '[' FROM TRIM(TRAILING ']' FROM %s)), '.0.0.0') end"
                r",%s"
                r",%s)",
                (record["OID"], record["OID"], record["edition"], record["OID"], record["acct_no"],
                 record["astatus"], record["cust_no"], record["myGlobal"], record["myGlobal"], record["preferred"], record["status"]))
            mysql_conn_dev.commit()
            print("CACCT inserted")


def load_json_sql_PAUTH(item, json_file):
    missing_record_list = []
    with open(json_file) as f:
        for json_obj in f:
            data_dict = json.loads(json_obj)
            missing_record_list.append(data_dict)

    for record in missing_record_list:
        print(record)
        with mysql_conn_dev.cursor() as cursor:
            cursor.execute(
                r"INSERT INTO sayyed_sandpit.PAUTH_UNI (instanceNumber, classNumber, parentClassNumber, subLevel, subId, edition,...) "
                r"VALUES ()",
                (record["OID"], record["OID"], record["edition"], record["OID"],...))

        mysql_conn_dev.commit()



if __name__ == '__main__':
    recon()
    #load_json_sql_PAUTH()

    mysql_conn.close()

由于某种原因，它只失败了：

load_json_sql_PAUTH()

如果我单独执行 load_json_sql_PAUTH() 拥有它自己的，它可以完美地工作，但是当我将它放在 recon() 中的条件逻辑中时，它会因为一旦遇到 INSERT 语句就失去连接而失败。

以下是错误信息：

    load_json_sql_PAUTH(item, src + item + '_MISSING_RECORDS_' + recon_date + '.json')
  File "C:/Users/Shahsa/PycharmProjects/mymisc/recon_json.py", line 370, in load_json_sql_test
    cursor.execute(
  File "C:\Users\Shahsa\PycharmProjects\mymisc\venv\lib\site-packages\pymysql\cursors.py", line 170, in execute
    result = self._query(query)
  File "C:\Users\Shahsa\PycharmProjects\mymisc\venv\lib\site-packages\pymysql\cursors.py", line 328, in _query
    conn.query(q)
  File "C:\Users\Shahsa\PycharmProjects\mymisc\venv\lib\site-packages\pymysql\connections.py", line 517, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "C:\Users\Shahsa\PycharmProjects\mymisc\venv\lib\site-packages\pymysql\connections.py", line 732, in _read_query_result
    result.read()
  File "C:\Users\Shahsa\PycharmProjects\mymisc\venv\lib\site-packages\pymysql\connections.py", line 1075, in read
    first_packet = self.connection._read_packet()
  File "C:\Users\Shahsa\PycharmProjects\mymisc\venv\lib\site-packages\pymysql\connections.py", line 657, in _read_packet
    packet_header = self._read_bytes(4)
  File "C:\Users\Shahsa\PycharmProjects\mymisc\venv\lib\site-packages\pymysql\connections.py", line 706, in _read_bytes
    raise err.OperationalError(
pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query')

【问题讨论】：

mysql错误日志有没有错误。

你的意思是登录数据库吗？我是 mysql 新手...我确实尝试过运行 SELECT * FROM mysql。 general_log 什么也没返回，所以我猜它没有打开。我正在使用 Windows...

mysqld 数据库的日志。不是一般日志中的查询。请参阅 mysql 文档以在 Window 上找到它。在您的问题中包含其最近的内容。

【参考方案1】：

我通过在每个函数中创建单独的连接来解决此问题。 @xvga 在此question

中提到了类似的东西