由于对物化视图的查询，clickhouse 客户端正在关闭

Posted 2023-03-25

【中文标题】由于对物化视图的查询，clickhouse 客户端正在关闭

【英文标题】：clickhouse client is going down beacuse of query on Materialized Views

【发布时间】：2020-12-07 12:01:44

【问题描述】：

使用目标表创建 mv 时

目标表：

CREATE TABLE target_test
(
    day date,
    hour UInt32,
    test_sum SimpleAggregateFunction(sum, Float64)
)
ENGINE = SummingMergeTree()
PARTITION BY toRelativeWeekNum(toDateTime(day)) 
ORDER BY (day, hour)


CREATE MATERIALIZED VIEW target_test_mv
TO target_test
as select 
toDate(session_ts) as day, 
toHour(toDateTime(session_ts)) as hour,
sum(if (test is null, 0,test)) as test_sum
from events
where session_ts>= now()-1000
group by day, hour

当数据插入到目标表中以及当我运行这个查询时 clickhouse 服务器因为查询原因崩溃了？？

select sum(revenue_sum) from target_test_mv

这是来自日志：

2020.12.07 12:03:48.059559 [ 19243 ]  <Fatal> BaseDaemon: ########################################
2020.12.07 12:03:48.059595 [ 19244 ]  <Fatal> BaseDaemon: ########################################
2020.12.07 12:03:48.059642 [ 19244 ]  <Fatal> BaseDaemon: (version 20.4.2.9 (official build)) (from thread 19242) (query_id: 6651689c-484a-4b53-a174-06fd01848208) Received signal Segmentation fault (11).
2020.12.07 12:03:48.059644 [ 19243 ]  <Fatal> BaseDaemon: (version 20.4.2.9 (official build)) (from thread 19241) (query_id: 6651689c-484a-4b53-a174-06fd01848208) Received signal Segmentation fault (11).
2020.12.07 12:03:48.059685 [ 19244 ]  <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.
2020.12.07 12:03:48.059691 [ 19243 ]  <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.
2020.12.07 12:03:48.059710 [ 19244 ]  <Fatal> BaseDaemon: Stack trace: 0xcad040b 0xd26b20d 0xd281202 0xdcbb2f3 0xdcbf909 0xdb0b9a1 0xdb0f95d 0xdb10324 0x901e26b 0x901c753 0x7fec413f0dd5 0x7fec41a04ead
2020.12.07 12:03:48.059712 [ 19243 ]  <Fatal> BaseDaemon: Stack trace: 0xcad040b 0xd26b20d 0xd281202 0xdcbb2f3 0xdcbf909 0xdb0b9a1 0xdb0f95d 0xdb10324 0x901e26b 0x901c753 0x7fec413f0dd5 0x7fec41a04ead
2020.12.07 12:03:48.059781 [ 19244 ]  <Fatal> BaseDaemon: 3. DB::IAggregateFunctionHelper<DB::AggregateFunctionNullUnary<true> >::addBatchSinglePlace(unsigned long, char*, DB::IColumn const**, DB::Arena*) const @ 0xcad040b in /opt/clickhouse/engine/clickhouse-common-static-20.4.2.9/usr/bin/clickhouse
2020.12.07 12:03:48.059781 [ 19243 ]  <Fatal> BaseDaemon: 3. DB::IAggregateFunctionHelper<DB::AggregateFunctionNullUnary<true> >::addBatchSinglePlace(unsigned long, char*, DB::IColumn const**, DB::Arena*) const @ 0xcad040b in /opt/clickhouse/engine/clickhouse-common-static-20.4.2.9/usr/bin/clickhouse
2020.12.07 12:03:48.059856 [ 19243 ]  <Fatal> BaseDaemon: 4. DB::Aggregator::executeWithoutKeyImpl(char*&, unsigned long, DB::Aggregator::AggregateFunctionInstruction*, DB::Arena*) @ 0xd26b20d in /opt/clickhouse/engine/clickhouse-common-static-20.4.2.9/usr/bin/clickhouse
2020.12.07 12:03:48.059858 [ 19244 ]  <Fatal> BaseDaemon: 4. DB::Aggregator::executeWithoutKeyImpl(char*&, unsigned long, DB::Aggregator::AggregateFunctionInstruction*, DB::Arena*) @ 0xd26b20d in /opt/clickhouse/engine/clickhouse-common-static-20.4.2.9/usr/bin/clickhouse
2020.12.07 12:03:48.059902 [ 19243 ]  <Fatal> BaseDaemon: 5. DB::Aggregator::executeOnBlock(std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > >, unsigned long, DB::AggregatedDataVariants&, std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >&, std::__1::vector<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >, std::__1::allocator<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> > > >&, bool&) @ 0xd281202 in /opt/clickhouse/engine/clickhouse-common-static-20.4.2.9/usr/bin/clickhouse
2020.12.07 12:03:48.059917 [ 19244 ]  <Fatal> BaseDaemon: 5. DB::Aggregator::executeOnBlock(std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > >, unsigned long, DB::AggregatedDataVariants&, std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >&, std::__1::vector<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >, std::__1::allocator<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> > > >&, bool&) @ 0xd281202 in /opt/clickhouse/engine/clickhouse-common-static-20.4.2.9/usr/bin/clickhouse

【问题讨论】：

SummingMT 不支持 SimpleA.F. github.com/ClickHouse/ClickHouse/issues/8052

【参考方案1】：

我在更新版本的 ClickHouse 中尝试了与您类似的场景，它适用于我：

CREATE TABLE target_test
    ( day date,
    hour UInt32,
    test_sum Float64 )
ENGINE = SummingMergeTree() PARTITION BY toRelativeWeekNum(toDateTime(day)) ORDER BY (day, hour)

CREATE MATERIALIZED VIEW target_test_mv TO target_test as
select
    toDate(event_time) as day,
    toHour(toDateTime(event_time)) as hour,
    sum(if (read_rows is null, 0,read_rows)) as test_sum
from system.query_log
group by day, hour
limit 10000

:) select sum(test_sum) from target_test;

SELECT sum(test_sum)
FROM target_test

┌─sum(test_sum)─┐
│         20661 │
└───────────────┘

请注意，我正在使用 query_log 进行测试，并且我已经删除了 SimpleAggregateFunction，因为 SummingMergeTree 负责这一点。

你可以尝试几件事：

尝试使用system.query_log 而不是您的events 表重现上述场景，以丢弃您的表有任何问题。 尝试更新版本的 ClickHouse（20.4.2.9 从 5 月开始） 如果您的方案仍然失败并且日志中存在分段错误，您可以open a case in the ClickHouse GitHub repo。

【讨论】：

顺便说一句，当我在 mv 上运行查询时，如果我在目标表上运行相同的查询，它不会导致 clickhouse 服务器崩溃