Hive的严格模式

Posted 2020-08-04 晨柳溪

在hive里面可以通过严格模式防止用户执行那些可能产生意想不到的不好的效果的查询,从而保护hive的集群。

用户可以通过 set hive.mapred.mode=strict 来设置严格模式，改成unstrict则为飞严格模式。  在严格模式下，用户在运行如下query的时候会报错。

1. 分区表的查询没有使用分区字段来限制。

select * from mart_catering.dim_shopca_reduction_ss limit 1000;

得到的错误是

CliDriver update main thread name to 4105daa5-e5a7-49d6-8e02-52c9184732f9
16/08/29 11:20:54 INFO CliDriver: CliDriver update main thread name to 4105daa5-e5a7-49d6-8e02-52c9184732f9

Logging initialized using configuration in jar:file:/opt/meituan/versions/mthive-0.13-package/lib/hive-common-0.13.1.jar!/hive-log4j.properties
FAILED: SemanticException [Error 10041]: No partition predicate found for Alias ‘dim_shopca_reduction_ss‘ Table ‘dim_shopca_reduction_ss‘
query fails

 

2. 使用了笛卡尔积

当用户写代码将表的别名写错的时候会引起笛卡尔积，例如

SELECT *
FROM origindb.promotion__campaign c
JOIN origindb.promotion__campaignex ce
ON c.id = c.id
limit 1000

得到的错误信息

CliDriver update main thread name to 9a962abc-afea-470f-9738-dceda72ff1fd
16/08/29 11:38:23 INFO CliDriver: CliDriver update main thread name to 9a962abc-afea-470f-9738-dceda72ff1fd

Logging initialized using configuration in jar:file:/opt/meituan/versions/mthive-0.13-package/lib/hive-common-0.13.1.jar!/hive-log4j.properties
FAILED: SemanticException [Error 10052]: In strict mode, cartesian product is not allowed. If you really want to perform the operation, set hive.mapred.mode=nonstrict
query fails

3. order by 的时候没有使用limit

SELECT *
FROM origindb.promotion__campaign 
order by id

得到的错误信息

CliDriver update main thread name to 5efafee6-1ce8-4985-9b98-c8f5ff88bccd
16/08/29 11:40:45 INFO CliDriver: CliDriver update main thread name to 5efafee6-1ce8-4985-9b98-c8f5ff88bccd

Logging initialized using configuration in jar:file:/opt/meituan/versions/mthive-0.13-package/lib/hive-common-0.13.1.jar!/hive-log4j.properties
FAILED: SemanticException 3:9 In strict mode, if ORDER BY is specified, LIMIT must also be specified. Error encountered near token ‘id‘
query fails

当使用的limit的时候错误消除。