用sqoop将mysql的数据导入到hive表

Posted 2020-11-19 学无止境

一、先将mysql一张表的数据用sqoop导入到hdfs

1.1、先在mysql中准备一张测试用的表

mysql> desc user_info;
+-----------+-------------+------+-----+---------+-------+
| Field     | Type        | Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+-------+
| id        | int(11)     | YES  |     | NULL    |       |
| user_name | varchar(50) | YES  |     | NULL    |       |
| age       | int(11)     | YES  |     | NULL    |       |
| address   | varchar(50) | YES  |     | NULL    |       |
+-----------+-------------+------+-----+---------+-------+
4 rows in set (0.14 sec)

mysql> select * from user_info;
+------+-----------+------+--------------------+
| id   | user_name | age  | address            |
+------+-----------+------+--------------------+
|    1 | zhangsan  |   20 | shenzhen nanshang  |
|    2 | lisi      |   20 | shenzhen futian    |
|    3 | wangwu    |   23 | shenzhen luohu     |
|    4 | cailiu    |   24 | shenzhen guangming |
|    5 | zhuqi     |   25 | shenzhen baoan     |
|    6 | houba     |   26 | shenzhen xili      |
|    7 | laojiu    |   27 | shenzhen yantian   |
+------+-----------+------+--------------------+
7 rows in set (0.00 sec)

mysql> 

下面 将 user_info表中的前100条数据导 导出来  只要id  user_name和 age 这3个字段，数据存在hdfs 目录 /tmp/sqoop/user_info下。