union不支持orderByClauseclusterByClausedistributeByClausesortByClause或limitClause

Posted 2021-02-04 wqbin

union all 

union 

相同点 是 相当于上下拼接 上下两个拼接表必须字段保持一致

不同 union有去重效果，速度会更慢。

=============================================================================================================================

union all的子句里不支持orderByClause、clusterByClause、distributeByClause、sortByClause或limitClause
        解决：改造hql，去掉union all子查询里的orderByClause、clusterByClause、distributeByClause、sortByClause和limitClause语句
        示例：select t.id from (select dummy from default.dual limit 1 union all select dummy from default.dual limit 1)t;

                   去掉两个子查询里的limit 1;

create table lk3 (id string,nname string,grade int,goldUser int);

insert into lk3 values
(1,‘jack‘,300, 10 ),
(2,‘mach‘, 200, 10 ),
(3,‘lich‘, 100 ,10 ),
(4,‘rock‘, 1, 0 ),
(5,‘mick‘, 1 ,10 ),
(6,‘kight‘, 0 ,10 ),
(7,‘babaya‘, 0, 0 ),
(8,‘kano‘, 0, 10);

 select * from lk3;
+---------+------------+------------+---------------+--+
| lk3.id  | lk3.nname  | lk3.grade  | lk3.golduser  |
+---------+------------+------------+---------------+--+
| 1       | jack       | 300        | 10            |
| 2       | mach       | 200        | 10            |
| 3       | lich       | 100        | 10            |
| 4       | rock       | 1          | 0             |
| 5       | mick       | 1          | 10            |
| 6       | kight      | 0          | 10            |
| 7       | babaya     | 0          | 0             |
| 8       | kano       | 0          | 10            |
+---------+------------+------------+---------------+--+

报错代码：

0: jdbc:hive2://localhost:10000> SELECT * FROM lk3 where grade != 1 order by grade desc,goldUser
. . . . . . . . . . . . . . . .> union all
. . . . . . . . . . . . . . . .> select * from lk3 where grade != 1 order by grade desc,goldUser;
Error: Error while compiling statement: FAILED: ParseException line 3:63 Failed to recognize predicate ‘<EOF>‘.
Failed rule: ‘orderByClause clusterByClause distributeByClause sortByClause limitClause can only be applied to the whole union.‘ in statement (state=42000,code=40000)

正确处理

0: jdbc:hive2://localhost:10000> select * from (SELECT * FROM lk3 where grade != 1 order by grade desc,goldUser limit 1) a
. . . . . . . . . . . . . . . .> union all
. . . . . . . . . . . . . . . .> select * from lk3 where grade != 1 order by grade desc,goldUser limit 2,3;
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
+--------+-----------+-----------+--------------+--+
| u1.id  | u1.nname  | u1.grade  | u1.golduser  |
+--------+-----------+-----------+--------------+--+
| 2      | mach      | 200       | 10           |
| 3      | lich      | 100       | 10           |
| 7      | babaya    | 0         | 0            |
+--------+-----------+-----------+--------------+--+
3 rows selected (33.439 seconds)