Hive 编程入门 之 Join 的失效
Posted 有关SQL
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Hive 编程入门 之 Join 的失效相关的知识,希望对你有一定的参考价值。
环境:
Hive: 2.7.7
Oracle SQL Developer
Cloudera JDBC Driver
案例 - 1 : 非全等 Join 失效
select a.*, b.*
from default.employee a
inner join default.employee b on a.salary < b.salary
异常:
SQL 错误: [Cloudera][HiveJDBCDriver](500051) ERROR processing query/statement. Error Code: 40000, SQL state: TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Error while compiling statement: FAILED: SemanticException Cartesian products are disabled for safety reasons. If you know what you are doing, please sethive.strict.checks.cartesian.product to false and that hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get errors or incorrect results if you make a mistake while using some of the unsafe features.:17:16,
...(省去万字)
SemanticException Cartesian products are disabled for safety reasons. If you know what you are doing, please sethive.strict.checks.cartesian.product to false and that hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get errors or incorrect results if you make a mistake while using some of the unsafe features.), Query: select a.*, b.*
from default.employee a
inner join default.employee b on a.salary < b.salary.
为了保障整个集群的稳定性,类似非全等的 Join 是默认禁止的,原因在异常代码中给出了:
FAILED: SemanticException Cartesian products are disabled for safety reasons
这样就可以避免因为大数据的 Join 导致网络瘫痪。
如果清晰的知道,非全等 Join 不会造成影响,那么可以修改这些配置来启动非全等 join 的操作。
set hive.strict.checks.cartesian.product=false
set hive.mapred.mode=nonstrict
select a.*, b.*
from default.employee a
inner join default.employee b on a.salary < b.salary
最终非全等 Join 执行也顺利通过:
以上是关于Hive 编程入门 之 Join 的失效的主要内容,如果未能解决你的问题,请参考以下文章