pd.read_sql - 不支持的格式字符错误 (0x27)

Posted

技术标签:

【中文标题】pd.read_sql - 不支持的格式字符错误 (0x27)【英文标题】:pd.read_sql - Unsupported format character error (0x27) 【发布时间】:2021-06-06 08:13:44 【问题描述】:

如上所述,我正在尝试使用 pd.read_sql 来查询我们的 mysql 数据库,并收到双引号/单引号错误。

当我从 LIKE 子句(第 84-87 行)中删除 % 运算符时,查询运行,但这些是必需的。我知道我需要格式化字符串,但我不知道如何在这么大的查询中。

这是查询:

SELECT
    s.offer_id,
    s.cap_id,
    vi.make,
    vi.model,
    vi.derivative,
    i.vehicle_orders,
    s.lowest_offer,
    CASE
        WHEN f.previous_avg = f.previous_low THEN "n/a"
        ELSE FORMAT(f.previous_avg, 2)
    END as previous_avg,
    f.previous_low,
    CASE
        WHEN ( ( (s.lowest_offer - f.previous_avg) / f.previous_avg) * 100) = ( ( (s.lowest_offer - f.previous_low) / f.previous_low) * 100) THEN "n/a"
        ELSE CONCAT(FORMAT( ( ( (s.lowest_offer - f.previous_avg) / f.previous_avg) * 100), 2), "%")
    END as diff_avg,
    CONCAT(FORMAT( ( ( (s.lowest_offer - f.previous_low) / f.previous_low) * 100), 2), "%") as diff_low,
    s.broker,
    CASE
        WHEN s.in_stock = '1' THEN "In Stock"
        ELSE "Factory Order"
    END as in_stock,
    CASE
        WHEN s.special IS NOT NULL THEN "Already in Specials"
        ELSE "n/a"
    END as special
FROM

    (   SELECT o.id as offer_id,
               o.cap_id as cap_id,
               MIN(o.monthly_payment) as lowest_offer,
               b.name as broker,
               o.stock as in_stock,
               so.id as special
            FROM
                offers o
                INNER JOIN brands b ON ( o.brand_id = b.id )
                LEFT JOIN special_offers so ON ( so.cap_id = o.cap_id )
            WHERE
                ( o.date_modified >= DATE_ADD(NOW(), INTERVAL -1 DAY) OR o.date_created >= DATE_ADD(NOW(), INTERVAL -1 DAY) )
                AND o.deposit_value = 9
                AND o.term = 48
                AND o.annual_mileage = 8000
                AND o.finance_type = 'P'
                AND o.monthly_payment > 100
            GROUP BY 
                o.cap_id
            ORDER BY
                special DESC) s
                
INNER JOIN      
        
    (   SELECT o.cap_id as cap_id,
               AVG(o.monthly_payment) as previous_avg,
               MIN(o.monthly_payment) as previous_low
            FROM
                offers o
            WHERE
                o.date_modified < DATE_ADD(NOW(), INTERVAL -1 DAY)
                AND o.date_modified >= DATE_ADD(NOW(), INTERVAL -1 WEEK)
                AND o.deposit_value = 9
                AND o.term = 48
                AND o.annual_mileage = 8000
                AND o.finance_type = 'P'
                AND o.monthly_payment > 100
            GROUP BY
                o.cap_id ) f ON ( s.cap_id = f.cap_id )
                
LEFT JOIN

    (   SELECT a.cap_id as cap_id,
               v.manufacturer as make,
               v.model as model,
               v.derivative as derivative,
               COUNT(*) as vehicle_orders
            FROM
                    (    SELECT o.id,
                                o.name as name,
                                o.email as email,
                                o.date_created as date,
                                SUBSTRING_INDEX(SUBSTRING(offer_serialized, LOCATE("capId", offer_serialized) +12, 10), '"', 1) as cap_id
                            FROM moneyshake.orders o
                            WHERE o.name NOT LIKE 'test%'
                                  AND o.email NOT LIKE 'jawor%'
                                  AND o.email NOT LIKE 'test%'
                                  AND o.email NOT LIKE '%moneyshake%'
                                  AND o.phone IS NOT NULL
                                  AND o.date_created > DATE_ADD(NOW(), INTERVAL -1 MONTH)
                    ) a JOIN moneyshake.vehicles_view v ON a.cap_id = v.id
            GROUP BY
                v.manufacturer,
                v.model,
                v.derivative,
                a.cap_id) i ON ( f.cap_id = i.cap_id )
                
INNER JOIN
            
    (   SELECT v.id as id,
               v.manufacturer as make,
               v.model as model,
               v.derivative as derivative
            FROM moneyshake.vehicles_view v
            GROUP BY v.id ) vi ON s.cap_id = vi.id

WHERE
    ( ( s.lowest_offer - f.previous_low ) / f.previous_low) * 100 <= -15
GROUP BY
    s.cap_id

谢谢!

【问题讨论】:

您用于尝试运行提供的查询的实际 Python 代码是什么? 【参考方案1】:

发生该错误后,DBAPI 层(例如,mysqlclient)本机使用“格式”paramstyle,并且百分号 (%) 被误解为格式字符而不是 LIKE 通配符。

解决方法是将 SQL 语句包装在 SQLAlchemy text() 对象中。例如,这将失败:

import pandas as pd
import sqlalchemy as sa

engine = sa.create_engine("mysql+mysqldb://scott:tiger@localhost:3307/mydb")

sql = """\
SELECT * FROM million_rows
WHERE varchar_col LIKE 'record00000%'
ORDER BY id
"""
df = pd.read_sql_query(sql, engine)

但只需将 read_sql_query() 调用更改为

df = pd.read_sql_query(sa.text(sql), engine)

会起作用的。

【讨论】:

听起来确实可以解决我的问题。我明天就知道了:)谢谢!

以上是关于pd.read_sql - 不支持的格式字符错误 (0x27)的主要内容,如果未能解决你的问题,请参考以下文章

窗口函数在 pd.read_sql 中不起作用;它显示错误

您可以并行或异步运行“pd.read_sql”吗?

如何使用 `mysql.connector` 使用 `pd.read_sql`

pandas

Python pandas read_sql 返回生成器对象

Pandas总结第二节 Pandas 的数据读取_pd.read_csv()的使用详解(非常全面,推荐收藏)