在 hql 脚本中，我们使用 "!sh echo ---new line---" 来表示相同的 .想知道在 impala 中打印 impala 脚本中任何行的替代方法吗？

Posted 2023-03-22

【中文标题】在 hql 脚本中，我们使用 "!sh echo ---new line---" 来表示相同的 .想知道在 impala 中打印 impala 脚本中任何行的替代方法吗？

【英文标题】：in hql scripts we use "!sh echo ---new line---" for the same . Want to know the alternative for this in impala to print any line in impala scripts?

【发布时间】：2020-06-03 13:44:25

【问题描述】：

在 hql 脚本中，我们使用 "!sh echo ---new line---" 来表示相同的 . 想知道在 impala 中打印 impala 脚本中任何行的替代方法吗？

【问题讨论】：

我在答案中添加了一些更改。关于如何将输出打印到文件的所有内容。希望它有所帮助，如果它有用，请随时接受我的回答并投票，我真的很感激。

【参考方案1】：

您可以从 impala 脚本调用 shell 命令行。

作为其工作原理的示例。

script_impala.sql

-- set a variable containing the of the game
SET hivevar:game=Monopoly;
-- return the list of the game
SELECT list_price FROM fun.games WHERE name = '$hivevar:game';
-- return the prices of the game ate game shops
SELECT shop, price FROM fun.inventory WHERE game = '$hivevar:game';

shell hdfs dfs -ls /user;
shell ls -ltr;
shell echo I can echo from impala scripts;
shell cat hex_color.sql

$ impala-shell -f script_impala.sql

输出

Starting Impala Shell without Kerberos authentication
Connected to localhost.localdomain:21000
Server version: impalad version 2.10.0-cdh5.13.3 RELEASE (build 15a453e15865344e75ce0fc6c4c760696d50f626)
Variable GAME set to Monopoly
Query: -- return the list of the game
SELECT list_price FROM fun.games WHERE name = 'Monopoly'
Query submitted at: 2020-06-04 23:29:19 (Coordinator: http://localhost.localdomain:25000)
Query progress can be monitored at: http://localhost.localdomain:25000/query_plan?query_id=a94afc7556843cf7:8b19809d00000000
+------------+
| list_price |
+------------+
| 19.99      |
+------------+
Fetched 1 row(s) in 0.15s
Query: -- return the prices of the game ate game shops
SELECT shop, price FROM fun.inventory WHERE game = 'Monopoly'
Query submitted at: 2020-06-04 23:29:19 (Coordinator: http://localhost.localdomain:25000)
Query progress can be monitored at: http://localhost.localdomain:25000/query_plan?query_id=c64d1c8950e5e1f8:15d410cb00000000
+-----------+-------+
| shop      | price |
+-----------+-------+
| Dicey     | 17.99 |
| Board 'Em | 25.00 |
+-----------+-------+
Fetched 2 row(s) in 0.14s
Found 5 items
drwxrwxrwt   - mapred   hadoop              0 2019-04-29 18:19 /user/history
drwxrwxrwx   - hive     hive                0 2019-04-29 18:19 /user/hive
drwxr-xr-x   - hue      hue                 0 2019-11-25 10:19 /user/hue
drwxr-xr-x   - spark    spark               0 2019-04-29 18:19 /user/spark
drwxrwxrwx   - training supergroup          0 2020-05-28 11:33 /user/training
--------
Executed in 2.10s
total 56
-rw-rw-r-- 1 training training   61 Sep 25  2019 hex_color.sql
-rw-rw-r-- 1 training training  115 Sep 25  2019 color_from_rgb.sql~
-rw-rw-r-- 1 training training   58 Sep 25  2019 hex_color_impala.sql
-rwxr-xr-x 1 training training  449 Sep 25  2019 email_results.sh
-rw-rw-r-- 1 training training 1166 Sep 25  2019 zero_air_time.csv
-rw-r--r-- 1 training training  261 Sep 26  2019 change_background.sh~
-rwxr-xr-x 1 training training  262 Sep 26  2019 change_background.sh
-rw------- 1 training training 2966 Sep 26  2019 ChangeVMDesktopColor.txt
-rw------- 1 training training 3279 Sep 26  2019 Hive&ImpalaInScripts&Applications.txt~
-rw------- 1 training training 3278 Sep 26  2019 Hive&ImpalaInScripts&Applications.txt
-rw-rw-r-- 1 training training  449 Sep 26  2019 email_resuts.sh
-rw-rw-r-- 1 training training  120 Apr 27 11:38 color_from_rgb.sql
-rw-rw-r-- 1 training training  397 Jun  4 23:24 game_prices.sql~
-rw-rw-r-- 1 training training  395 Jun  4 23:29 game_prices.sql
--------
Executed in 0.00s
I can echo from impala scripts
--------
Executed in 0.00s
SELECT hex FROM wax.crayons WHERE color = '$hivevar:color'
--------
Executed in 0.01s

如你所见，你只需要写在你的脚本中

shell <command>;

您甚至可以调用 hdfs dfs 命令

shell hdfs dfs -<command>;

上面的例子将输出打印到控制台，如果你想将输出打印到一个文件，你必须执行以下操作将所有输出重定向到一个文件：

$ impala-shell -f script_impala.sql >> /home/..../my_file.csv

其他选项是仅重定向脚本内部的 echo 命令。这将如下所示： script_impala.sql

-- set a variable containing the of the game
SET hivevar:game=Monopoly;
-- return the list of the game
SELECT list_price FROM fun.games WHERE name = '$hivevar:game';
-- return the prices of the game ate game shops
SELECT shop, price FROM fun.inventory WHERE game = '$hivevar:game';

shell hdfs dfs -ls /user;
shell ls -ltr;
shell echo I can echo from impala scripts >> /home/..../my_file.csv;
shell echo How are you? >> /home/..../my_file.csv;
shell echo I am feeling very good, how about you? >> /home/..../my_file.csv;
shell cat hex_color.sql

不要忘记行尾的分号“;”。 希望对你有帮助。