为 Postgres 分区创建自定义哈希运算符

Posted 2023-02-16

【中文标题】为 Postgres 分区创建自定义哈希运算符

【英文标题】：Create custom hash operator for Postgres partitioning

【发布时间】：2021-06-01 01:41:53

【问题描述】：

我想创建一个自定义哈希函数，Postgres（13.2 版）将使用它来跨分区分配行。问题是当前的解决方案 Postgres 不使用分区修剪。 这是我的代码：

-- dummy hash function
CREATE OR REPLACE FUNCTION partition_custom_bigint_hash(value BIGINT, seed
BIGINT)
RETURNS BIGINT AS $$
    SELECT value;
$$ LANGUAGE SQL IMMUTABLE PARALLEL SAFE;

-- operator
CREATE OPERATOR CLASS partition_custom_bigint_hash_op
    FOR TYPE int8
    USING hash AS
    OPERATOR 1 =,
    FUNCTION 2 partition_custom_bigint_hash(BIGINT, BIGINT);

-- table partitioned by hash with custom operator
CREATE TABLE sample(part_id BIGINT) PARTITION BY hash(part_id partition_custom_bigint_hash_op);
CREATE TABLE sample_part_1 PARTITION OF SAMPLE FOR VALUES WITH (modulus 3, remainder 0);
CREATE TABLE sample_part_2 PARTITION OF SAMPLE FOR VALUES WITH (modulus 3, remainder 1);
CREATE TABLE sample_part_3 PARTITION OF SAMPLE FOR VALUES WITH (modulus 3, remainder 2);

现在确保分区修剪已启用并正常工作：

SHOW enable_partition_pruning;
--  enable_partition_pruning
-- --------------------------
--  on

EXPLAIN * FROM sample WHERE part_id = 1::BIGINT;
--                               QUERY PLAN                              
-- ----------------------------------------------------------------------
--  Seq Scan on sample_part_1 sample  (cost=0.00..38.25 rows=11 width=8)
--    Filter: (part_id = '1'::bigint)
-- (2 rows)

所以在使用条件part_id=1::BIGINT 时效果很好，但如果我跳过转换为 BIGINT，我会得到：

EXPLAIN SELECT * FROM sample WHERE part_id = 1;
--                                   QUERY PLAN
-- ------------------------------------------------------------------------------
--  Append  (cost=0.00..101.36 rows=33 width=8)
--    ->  Seq Scan on sample_part_1 sample_1  (cost=0.00..33.73 rows=11 width=8)
--          Filter: (part_id = 1)
--    ->  Seq Scan on sample_part_2 sample_2  (cost=0.00..33.73 rows=11 width=8)
--          Filter: (part_id = 1)
--    ->  Seq Scan on sample_part_3 sample_3  (cost=0.00..33.73 rows=11 width=8)
--          Filter: (part_id = 1)

问题：为了使分区修剪在part_id=1 和part_id=1::BIGINT 这两种情况下都起作用，我需要进行哪些更改？

【参考方案1】：

有几个等式运算符，左侧有bigint：

SELECT oid,
       oprcode::regproc AS function,
       oprright::regtype AS right_side
FROM pg_operator
WHERE oprname = '='
  AND oprleft = 'bigint'::regtype;

 oid  | function | right_side 
------+----------+------------
  410 | int8eq   | bigint
  416 | int84eq  | integer
 1868 | int82eq  | smallint
(3 rows)

现在第二个查询使用这些运算符中的第二个，但该运算符不属于您的自定义运算符系列，因此不会进行分区修剪。

在match_clause_to_partition_keysrc/backend/partitioning/partprune.c 中查看此评论：

/*
 * See if the operator is relevant to the partitioning opfamily.
 *
 * Normally we only care about operators that are listed as being part
 * of the partitioning operator family.  But there is one exception:
 * the not-equals operators are not listed in any operator family
 * whatsoever, but their negators (equality) are.  We can use one of
 * those if we find it, but only for list partitioning.
 *
 * Note: we report NOMATCH on failure, in case a later partkey has the
 * same expression but different opfamily.  That's unlikely, but not
 * much more so than duplicate expressions with different collations.
 */

创建一个包含所需运算符的运算符族：

CREATE FUNCTION partition_custom_hash(value int8, seed int8) RETURNS int8
   AS 'SELECT value' LANGUAGE SQL IMMUTABLE PARALLEL SAFE;

CREATE FUNCTION partition_custom_hash(value int4, seed int4) RETURNS int8
   AS 'SELECT value::int8' LANGUAGE SQL IMMUTABLE PARALLEL SAFE;

CREATE FUNCTION partition_custom_hash(value int2, seed int2) RETURNS int8 
   AS 'SELECT value::int8' LANGUAGE SQL IMMUTABLE PARALLEL SAFE;

CREATE OPERATOR FAMILY partition_custom_integer_hash_ops USING hash;

CREATE OPERATOR CLASS partition_custom_int8_hash_ops FOR TYPE int8 USING hash
   FAMILY partition_custom_integer_hash_ops AS
   OPERATOR 1 = (int8, int8),
   FUNCTION 2 partition_custom_hash(int8, int8);

CREATE OPERATOR CLASS partition_custom_int4_hash_ops FOR TYPE int4 USING hash
   FAMILY partition_custom_integer_hash_ops AS
   OPERATOR 1 = (int8, int4),
   FUNCTION 2 partition_custom_hash(int4, int4);

CREATE OPERATOR CLASS partition_custom_int2_hash_ops FOR TYPE int2 USING hash 
   FAMILY partition_custom_integer_hash_ops AS 
   OPERATOR 1 = (int8, int2),
   FUNCTION 2 partition_custom_hash(int2, int2);

那么如果您使用partition_custom_int8_hash_ops，它应该可以按您的意愿工作。