Hive Remote模式搭建

Posted 2020-10-07

一、实验环境

1.软件版本：apache-hive-2.3.0-bin.tar.gz、mysql-community-server-5.7.19

2.mysql JDBC驱动包：mysql-connector-java-5.1.44.tar.gz

3.mysql已经安装在hadoop5上

4..主机规划

hadoop3	Remote：client
hadoop4	Remote：client
hadoop5	Remote：server；mysql

二、基础配置

1.解压并移动hive

[[email protected] ~]# tar -zxf apache-hive-2.3.0-bin.tar.gz
[[email protected] ~]# cp -r apache-hive-2.3.0-bin /usr/local/hive

2.修改环境变量

[[email protected] ~]# vim /etc/profile
export HIVE_HOME=/usr/local/hive
export PATH=$HIVE_HOME/bin:$PATH
[[email protected] ~]# source /etc/profile

3.复制初始文件

[[email protected] ~]# cd /usr/local/hive/conf/
[[email protected] conf]# cp hive-env.sh.template hive-env.sh  
[[email protected] conf]# cp hive-default.xml.template hive-site.xml  
[[email protected] conf]# cp hive-log4j2.properties.template hive-log4j2.properties  
[[email protected] conf]# cp hive-exec-log4j2.properties.template hive-exec-log4j2.properties

4.修改hive-env.sh文件

[[email protected] conf]# vim hive-env.sh    #在最后添加
export JAVA_HOME=/usr/local/jdk
export HADOOP_HOME=/usr/local/hadoop
export HIVE_HOME=/usr/local/hive    
export HIVE_CONF_DIR=/usr/local/hive/conf

5.拷贝mysql的JDBC驱动包

[[email protected] ~]# tar -zxf mysql-connector-java-5.1.44.tar.gz 
[[email protected] ~]# cp mysql-connector-java-5.1.44/mysql-connector-java-5.1.44-bin.jar /usr/local/hive/lib/

6.在hdfs中创建一下目录，并授权，用于存储文件

hdfs dfs -mkdir -p /user/hive/warehouse
hdfs dfs -mkdir -p /user/hive/tmp
hdfs dfs -mkdir -p /user/hive/log
hdfs dfs -chmod -R 777 /user/hive/warehouse
hdfs dfs -chmod -R 777 /user/hive/tmp
hdfs dfs -chmod -R 777 /user/hive/log

7.在mysql中创建相关用户和库

mysql> create database metastore;
Query OK, 1 row affected (0.03 sec)
mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.26 sec)
mysql> grant all on local_hive.* to [email protected]‘%‘ identified by ‘hive123456‘;
Query OK, 0 rows affected, 1 warning (0.03 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

7.使用scp将hive拷贝到hadoop3和4上

[[email protected] ~]# scp -r /usr/local/hive [email protected]:/usr/local/
[[email protected] ~]# scp -r /usr/local/hive [email protected]:/usr/local/

三、修改配置文件

1.服务端hive-site.xml的配置（在hadoop5上）

<configuration>
<property>
    <name>hive.exec.scratchdir</name>
    <value>/user/hive/tmp</value>
</property>
<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
</property>
<property>
    <name>hive.querylog.location</name>
    <value>/user/hive/log</value>
</property>

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://hadoop5:3306/metastore?createDatabaseIfNotExist=true&amp;characterEncoding=UTF-8&amp;useSSL=false</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>hive123456</value>
</property>
</configuration>

2.客户端hive-site.xml的配置（在hadoop3和4上）

<configuration>
<property>
    <name>hive.metastore.uris</name>
    <value>thrift://hadoop5:9083</value>
</property>
<property>
    <name>hive.exec.scratchdir</name>
    <value>/user/hive/tmp</value>
</property>
<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
</property>
<property>
    <name>hive.querylog.location</name>
    <value>/user/hive/log</value>
</property>
<property>
    <name>hive.metastore.local</name>
    <value>false</value>
</property>
</configuration>

四、启动hive（两中方式）

1.直接启动

service：

[[email protected] ~]# hive --service metastore

client：

[[email protected] ~]# hive
hive> show databases;
OK
default
Time taken: 1.599 seconds, Fetched: 1 row(s)
hive> quit;

2.beeline方式

需要先在hadoop的core-site.xml中添加配置

<property>  
  <name>hadoop.proxyuser.root.groups</name>  
  <value>*</value>  
</property>  
<property>  
  <name>hadoop.proxyuser.root.hosts</name>  
  <value>*</value>  
</property>

service：

[[email protected] ~]# nohup hiveserver2 &
[[email protected] ~]# netstat -nptl | grep 10000
tcp        0      0 0.0.0.0:10000           0.0.0.0:*               LISTEN      3464/java

client：

[[email protected] ~]# beeline 
Beeline version 1.2.1.spark2 by Apache Hive
beeline> !connect jdbc:hive2://hadoop5:10000 hive hive123456
Connecting to jdbc:hive2://hadoop5:10000
17/09/21 09:47:31 INFO jdbc.Utils: Supplied authorities: hadoop5:10000
17/09/21 09:47:31 INFO jdbc.Utils: Resolved authority: hadoop5:10000
17/09/21 09:47:31 INFO jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://hadoop5:10000
Connected to: Apache Hive (version 2.3.0)
Driver: Hive JDBC (version 1.2.1.spark2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hadoop5:10000> show databases;
+----------------+--+
| database_name  |
+----------------+--+
| default        |
+----------------+--+
1 row selected (2.258 seconds)

本文出自 “lullaby” 博客，请务必保留此出处http://lullaby.blog.51cto.com/10815696/1967630