hive-3.1.1 安装详细步骤

Posted 2021-01-26 lihao7

1、由于hive的元数据存储在关系型数据库中，先安装mysql

mysql官网，选择社区版：https://dev.mysql.com/downloads/mysql/5.7.html#downloads

wget下载安装包：wget https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.24-linux-glibc2.12-x86_64.tar.gz

解压：tar -zxvf mysql-5.7.24-linux-glibc2.12-x86_64.tar.gz

创建data文件：mkdir data #路径为/usr/local/src/mysql/data

需要创建mysql用户和用户组：

groupadd mysql

useradd -r -g mysql mysql

将安装文件和data文件修改为mysql用户：

chown -R mysql:mysql /usr/local/src/mysql

chmod -R 777 data /usr/local/src/mysql/data

 

修改配置：

./mysql_ssl_rsa_setup --datadir=/usr/local/src/mysql/data

vi /etc/my.cnf 修改为：(修改datadir的存储位置)

basedir=/usr/local/src/mysql/mysql

datadir=/usr/local/src/mysql/data

port = 3306

socket=/tmp/mysql.sock

pid-file=/tmp/mysqld/mysqld.pid

character-set-server = utf8

log-error=/var/log/mysqld.log

 

创建/tmp/mysql.sock、/tmp/mysqld/mysqld.pid、/var/log/mysqld.log，并分别修改mysql用户权限：

cd /tmp

touch mysql.sock

chown mysql:mysql mysql.sock

chmod 755 mysql.sock

 

cd /tmp/mysqld/

touch mysqld.pid

chown -R mysql:mysql mysqld.pid

chmod 755 mysqld.pid

 

cd /var/log/

touch mysqld.log

chown -R mysql:mysql /var/log

chmod 755 mysqld.log

 

修改环境变量：vi ~/.bash_profile

修改为：PATH=$PATH:$HOME/bin:/usr/local/src/mysql/mysql/bin

然后source ~/.bash_profile

 

初始化：./mysqld --initialize --user=root --basedir=/usr/local/src/mysql/mysql --datadir=/usr/local/src/mysql/data --lc_messages_dir=/usr/local/src/mysql/mysql/share --lc_messages=en_US

 

进入mysql的bin目录下，./mysqld_safe --user=mysql启动mysql，在/var/log/mysqld.log中找生成的临时密码：

然后mysql -uroot -p输入临时密码，即进入mysql命令终端；

 

设置密码，安装完成：

use mysql

update user set host=‘%‘ where user=‘root‘;

GRANT ALL PRIVILEGES ON *.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘......‘ WITH GRANT OPTION;（或者是：grant all privileges on *.* to [email protected]‘%‘ identified by "新密码";）

FLUSH PRIVILEGES;

设置开机自动启动：

cp /usr/local/src/mysql/mysql/support-files/mysql.server /etc/init.d/mysql

chkconfig --list mysql

service mysql start

service mysql stop

 

2、安装hive

获取bin安装包：wget http://mirror.bit.edu.cn/apache/hive/hive-3.1.1/apache-hive-3.1.1-bin.tar.gz

tar -zxvf apache-hive-3.1.1-bin.tar.gz

ln -s apache-hive-3.1.1-bin hive

 

修改环境变量：/etc/profile，添加

#hive

export HIVE_HOME=/usr/local/src/hive/hive

export PATH=$PATH:$HIVE_HOME/bin

 

然后： source /etc/profile

hive --version

进入conf目录，进行设置：cd /usr/local/src/hive/hive/conf

cp hive-default.xml.template hive-site.xml

vi hive-site.xml，增加：

<configuration>

<property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:mysql://localhost:3306/hive</value>

</property>

<property>

<name>javax.jdo.option.ConnectionDriverName</name>

<value>com.mysql.jdbc.Driver</value>

</property>

<property>

<name>javax.jdo.option.ConnectionUserName</name>

<value>root</value>

</property>

<property>

<name>javax.jdo.option.ConnectionPassword</name>

<value>123456</value>

</property>

<property>

<name>hive.metastore.schema.verification</name>

<value>false</value>

</property>

</configuration>

 

进入mysql数据库，创建hive数据：

create database hive;

 

进入lib目录，cd ../lib，下载mysql的连接包：

wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.47/mysql-connector-java-5.1.47.jar

 

进入hive安装目录bin目录下：执行schematool -dbType mysql -initSchema

初始化成功：

[[email protected] bin]# schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/src/hive/apache-hive-3.1.1-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/src/hadoop-3.0.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://localhost:3306/hive
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: root
Thu Dec 13 22:35:42 CST 2018 WARN: Establishing SSL connection without server‘s identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn‘t set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to ‘false‘. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.mysql.sql
Thu Dec 13 22:35:43 CST 2018 WARN: Establishing SSL connection without server‘s identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn‘t set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to ‘false‘. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

hive自动创建的表：

mysql> show tables;
+-------------------------------+
| Tables_in_hive |
+-------------------------------+
| AUX_TABLE |
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| COMPACTION_QUEUE |
| COMPLETED_COMPACTIONS |
| COMPLETED_TXN_COMPONENTS |
| CTLGS |
| DATABASE_PARAMS |
| DBS |
| DB_PRIVS |
| DELEGATION_TOKENS |
| FUNCS |
| FUNC_RU |
| GLOBAL_PRIVS |
| HIVE_LOCKS |
| IDXS |
| INDEX_PARAMS |

........

 由于已经新建了环境变量，直接输入hive进入命令终端：

出现了大量的warn信息，是由于 MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn‘t set，在hive的hive-site.conf中加上useSSL=false，即不启用SSL连接就可以：

 

再连接就没有了：

 

 create database test;

然后在hdfs 上查看目录：

 后续研究下 hive-site的其他配置项的含义。