clickhouse 发生代码：999 where optimize MATERIALIZED VIEW table with ReplicatedReplacingMergeTree engine

Posted 2023-03-25

技术标签:

【中文标题】clickhouse 发生代码：999 where optimize MATERIALIZED VIEW table with ReplicatedReplacingMergeTree engine【英文标题】：clickhouse occurred Code：999 where optimize MATERIALIZED VIEW table with ReplicatedReplacingMergeTree engine 【发布时间】：2021-08-17 08:21:28 【问题描述】：

它是 2 个共享 2 个副本 clickhouse 集群，它是 4 个 clickhouse 节点我在一个节点优化表，出现如下错误：但是在任何其他clickhouse节点上执行都是正常的。

risk-luck2.dg.163.org :) optimize table risk_detect_test.risk_doubtful_user_daily_device_view_lyp;

OPTIMIZE TABLE risk_detect_test.risk_doubtful_user_daily_device_view_lyp

Received exception from server (version 20.4.4):
Code: 999. DB::Exception: Received from localhost:9000. DB::Exception: Can't get data for node /clickhouse/tables/test/01-02/risk_doubtful_user_daily_device_view_lyp/replicas/risk-olap6.dg.163.org (multiple leaders Ok)/host: node doesn't exist (No node).

0 rows in set. Elapsed: 0.002 sec.

risk-luck2.dg.163.org :) show create table risk_detect_test.risk_doubtful_user_daily_device_view_lyp;

SHOW CREATE TABLE risk_detect_test.risk_doubtful_user_daily_device_view_lyp

┌─statement──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ CREATE MATERIALIZED VIEW risk_detect_test.risk_doubtful_user_daily_device_view_lyp
(
    `app_id` String,
    `event_date` Date,
    `device_id` UInt32
)
ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/test/layer-shard/risk_doubtful_user_daily_device_view_lyp', 'replica')
PARTITION BY toYYYYMM(event_date)
PRIMARY KEY app_id
ORDER BY (app_id, event_date, device_id)
SETTINGS index_granularity = 8192 AS
SELECT
    app_id,
    event_date,
    xxHash32(device_id) AS device_id
FROM risk_detect_online.dwd_risk_doubtful_detail │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

【问题讨论】：

20.4 不受支持。我需要 clickhouse-server.log 围绕此错误的完整跟踪。怎么给你看 【参考方案1】：

这似乎是 CH 中的另一个错误

ENGINE = ReplicatedReplacingMergeTree(
'/clickhouse/tables/test/layer-shard/risk_doubtful_user_daily_device_view_lyp', 'replica')


Can't get data for node 
 /clickhouse/tables/online/01-02/risk_doubtful_user_daily_device_view/replicas/risk-olap6.dg.163.org

在 Mat.View 的情况下，CH 尝试使用不正确的 Zookeeper 路径。 risk_doubtful_user_daily_device_view 而不是 risk_doubtful_user_daily_device_view_lyp。

数据库也不正确tables/online/01-02//tables/test/layer-shard/

我建议您切换到“TO”符号。 https://den-crane.github.io/Everything_you_should_know_about_materialized_views_commented.pdf

或者针对内表运行优化

OPTIMIZE TABLE "risk_detect_test"."inner.risk_doubtful_user_daily_device_view_lyp";

【讨论】：

优化表 "risk_detect_test"."inner.risk_doubtful_user_daily_device_view_lyp";不行，出现同样的错误【参考方案2】：

clickhouse-server.log 如下：

2021.08.18 16:37:11.384434 [ 128614 ] b6de1d84-a238-4e2f-9af4-3ce0ddf8551d <Debug> executeQuery: (from 10.200.128.91:40236) insert into dwd_risk_detect_detail(app_id, app_type, app_version, city, created_at, defense_count, defense_result, detect_count, device_code, device_id, id, ip, model, os_version, package_name, phone_brand, platform, province, region, risk_type1, risk_type2, risk_type3, role_account, role_id, sdk_version, sign_hash, ts)  FORMAT TabSeparated
2021.08.18 16:37:11.384735 [ 128614 ] b6de1d84-a238-4e2f-9af4-3ce0ddf8551d <Trace> ContextAccess (default): Access granted: INSERT(app_id, app_type, app_version, city, created_at, defense_count, defense_result, detect_count, device_code, device_id, id, ip, model, os_version, package_name, phone_brand, platform, province, region, risk_type1, risk_type2, risk_type3, role_account, role_id, sdk_version, sign_hash, ts) ON risk_detect_online.dwd_risk_detect_detail
2021.08.18 16:37:11.385706 [ 128614 ] b6de1d84-a238-4e2f-9af4-3ce0ddf8551d <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "risk_type1 != 0" moved to PREWHERE
2021.08.18 16:37:11.386554 [ 128614 ] b6de1d84-a238-4e2f-9af4-3ce0ddf8551d <Trace> ContextAccess (default): Access granted: SELECT(id, app_id, app_type, device_id, role_id, defense_result, risk_type1, risk_type2, risk_type3, defense_count, detect_count, event_date, event_hour, event_minute) ON risk_detect_online.dwd_risk_detect_detail
2021.08.18 16:37:11.386764 [ 128614 ] b6de1d84-a238-4e2f-9af4-3ce0ddf8551d <Trace> ContextAccess (default): Access granted: INSERT(app_id, app_type, event_date, event_hour, event_minute, risk_type1, risk_type2, risk_type3, defense_result, defense_count, detect_count, device_id, role_id, id) ON risk_detect_online.`.inner.risk_stat_view`
2021.08.18 16:37:11.387323 [ 128614 ] b6de1d84-a238-4e2f-9af4-3ce0ddf8551d <Trace> ContextAccess (default): Access granted: SELECT(app_id, app_type, device_id, role_id, event_date) ON risk_detect_online.dwd_risk_detect_detail
2021.08.18 16:37:11.387434 [ 128614 ] b6de1d84-a238-4e2f-9af4-3ce0ddf8551d <Trace> ContextAccess (default): Access granted: INSERT(app_id, app_type, event_date, device_id, role_id) ON risk_detect_online.`.inner.risk_total_user_stat_view`
2021.08.18 16:37:11.578506 [ 128861 ] 819b05a8-5ad0-414f-a0a7-111c765cac57 <Debug> executeQuery: (from 127.0.0.1:40932) OPTIMIZE TABLE risk_detect_online.risk_doubtful_user_daily_device_view
2021.08.18 16:37:11.578659 [ 128861 ] 819b05a8-5ad0-414f-a0a7-111c765cac57 <Trace> ContextAccess (default): Access granted: OPTIMIZE ON risk_detect_online.risk_doubtful_user_daily_device_view
2021.08.18 16:37:11.580097 [ 128861 ] 819b05a8-5ad0-414f-a0a7-111c765cac57 <Error> executeQuery: Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/tables/online/01-02/risk_doubtful_user_daily_device_view/replicas/risk-olap6.dg.163.org (multiple leaders Ok)/host: node doesn't exist (No node) (version 20.4.4.18 (official build)) (from 127.0.0.1:40932) (in query: OPTIMIZE TABLE risk_detect_online.risk_doubtful_user_daily_device_view), Stack trace (when copying this message, always include the lines below):

0. Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x104191d0 in /usr/bin/clickhouse
1. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x8fff8ad in /usr/bin/clickhouse
2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) @ 0xdddf7d8 in /usr/bin/clickhouse
3. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0xdddfe02 in /usr/bin/clickhouse
4. ? @ 0xddf1f60 in /usr/bin/clickhouse
5. DB::StorageReplicatedMergeTree::sendRequestToLeaderReplica(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&) @ 0xd76117e in /usr/bin/clickhouse
6. DB::StorageReplicatedMergeTree::optimize(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::IAST> const&, bool, bool, DB::Context const&) @ 0xd762546 in /usr/bin/clickhouse
7. DB::StorageMaterializedView::optimize(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::IAST> const&, bool, bool, DB::Context const&) @ 0xd6d5a9d in /usr/bin/clickhouse
8. DB::InterpreterOptimizeQuery::execute() @ 0xd225346 in /usr/bin/clickhouse
9. ? @ 0xd5499f9 in /usr/bin/clickhouse
10. DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, bool) @ 0xd54d025 in /usr/bin/clickhouse
11. DB::TCPHandler::runImpl() @ 0x9106678 in /usr/bin/clickhouse
12. DB::TCPHandler::run() @ 0x9107650 in /usr/bin/clickhouse
13. Poco::Net::TCPServerConnection::start() @ 0x10304f4b in /usr/bin/clickhouse
14. Poco::Net::TCPServerDispatcher::run() @ 0x103053db in /usr/bin/clickhouse
15. Poco::PooledThread::run() @ 0x104b2fa6 in /usr/bin/clickhouse
16. Poco::ThreadImpl::runnableEntry(void*) @ 0x104ae260 in /usr/bin/clickhouse
17. start_thread @ 0x74a4 in /lib/x86_64-linux-gnu/libpthread-2.24.so
18. __clone @ 0xe8d0f in /lib/x86_64-linux-gnu/libc-2.24.so

2021.08.18 16:37:11.580526 [ 128861 ] 819b05a8-5ad0-414f-a0a7-111c765cac57 <Debug> MemoryTracker: Peak memory usage (for query): 0.00 B.
2021.08.18 16:37:11.580592 [ 128861 ]  <Information> TCPHandler: Processed in 0.002 sec.

【讨论】：

您应该将此添加到问题中，而不是作为答案。

以上是关于clickhouse 发生代码：999 where optimize MATERIALIZED VIEW table with ReplicatedReplacingMergeTree engine的主要内容，如果未能解决你的问题，请参考以下文章

为啥 clickhouse 同时具有 WHERE 和 PREWHERE 构造？

Where 子句不适用于 ClickHouse 视图

ClickHouse 21.7.3.14-2(十) 查询优化

ORA-01722: 无效的数字 where 子句

在 ClickHouse 中重命名表时究竟会发生啥？

ClickHouse-尚硅谷（11. 高级-查询优化）学习笔记