Hive基础

Posted 2021-01-06 xupccc

Hive基础

1、介绍

Hive是OLAP(online analyze process，在线分析处理)。通常称为数据仓库，简称数仓。内置很多分析函数，可进行海量数据的在线分析处理。hive构建在hadoop之上，使用hdfs作为进行存储，计算过程采用的是Mapreduce完成，本质上hive是对hadoop的mr的封装，通过原始的mr方式进行数据处理与分析，往往效率较低，而且具有相当的复杂度，学习曲线较长。hive常用传统的sql方式作为操作手段，极大的降低了学习曲线，毕竟大部分人对sql还是比较熟悉的。但在运行时，仍然要将sql进行翻译成mapreduce程序进行。

2、hive安装

下载hive的软件包，解压之后配置环境变量即可。

3、hive配置

hive中元数据存在关系型数据库中，默认是derby数据库，这里改成mysql数据库。hive的配置文件为conf/hive-site.xml目录下：

<configuration>
  ...
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://192.168.231.1:3306/big11_hive</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>Username to use against metastore database</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>root</value>
    <description>password to use against metastore database</description>
  </property>
  <property>
    <name>hive.server2.enable.doAs</name>
    <value>false</value>
    <description>
      Setting this property to true will have HiveServer2 execute
      Hive operations as the user making the calls to it.
    </description>
  </property>
  <property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>
    <description>
      Enforce metastore schema version consistency.
      True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic
            schema migration attempt. Users are required to manually migrate schema after Hive upgrade which ensures
            proper metastore schema migration. (Default)
      False: Warn if the version information stored in metastore doesn‘t match with one from in Hive jars.
    </description>
  </property>


  ...
</configuration>