Hadoop之Hue

Posted 2022-11-29 _TIM_

Hadoop User Experience

一个图形化界面，提供一个Web界面可以供我们查看执行我们其他的框架

Hue是一个开源的Apache Hadoop UI系统，最早是由Cloudera Desktop演化而来，由Cloudera贡献给开源社区，它是基于Python Web框架Django实现的。通过使用Hue我们可以在浏览器端的Web控制台上与hadoop集群进行交互来分析处理数据，例如操作HDFS上的数据，运行MapReduce Job等等。简单就理解就是hadoop可视化工具

Hue集成HDFS

##hdfs-site.xml
<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>


##core-site.xml
<property>
  <name>hadoop.proxyuser.hue.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.hue.groups</name>
  <value>*</value>
</property>

修改hue.ini

fs_defaultfs=hdfs://node01:8020
 
webhdfs_url=http://node01:50070/webhdfs/v1
 
hadoop_hdfs_home= /export/servers/hadoop-2.6.0-cdh5.14.0
 
hadoop_bin=/export/servers/hadoop-2.6.0-cdh5.14.0/bin
 
hadoop_conf_dir=/export/servers/hadoop-2.6.0-cdh5.14.0/etc/hadoop

配置Yarn

resourcemanager_host=hadoop-senior.ibeifeng.com
resourcemanager_port=8032
submit_to=True
resourcemanager_api_url=http://hadoop-senior.ibeifeng.com:8088
proxy_api_url=http://hadoop-senior.ibeifeng.com:8088
history_server_api_url=http://hadoop-senior.ibeifeng.com:19888

Hue集成HIve

#hive-site.xml
<property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
</property>

<property>
    <name>hive.server2.thrift.bind.host</name>
    <value>hadoop-senior.ibeifeng.com</value>
</property>

<property>
    <name>hive.metastore.uris</name>
    <value>thrift://hadoop-senior.ibeifeng.com:9083</value>
</property>

配置Hue

hive_server_host=hadoop-senior.ibeifeng.com
hive_server_port=10000
hive_conf_dir=/opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/conf
server_conn_timeout=120

Hue集成RDBMS

##大概在510行，这里需要配置sqlite，hue本身使用的是sqlite，然后再配置集成mysql
  [[databases]]
    # sqlite configuration.
    [[[sqlite]]]
      # Name to show in the UI.
      nice_name=SQLite

      # For SQLite, name defines the path to the database.
      name=/opt/app/hue-3.7.0-cdh5.3.6/desktop/desktop.db

      # Database backend to use.
      engine=sqlite

      # Database options to send to the server when connecting.
      # https://docs.djangoproject.com/en/1.4/ref/databases/
      ## options=

    # mysql, oracle, or postgresql configuration.
    [[[mysql]]]
      # Name to show in the UI.
      nice_name="My SQL DB"

      # For MySQL and PostgreSQL, name is the name of the database.
      # For Oracle, Name is instance of the Oracle server. For express edition
      # this is 'xe' by default.
      name=test

      # Database backend to use. This can be:
      # 1. mysql
      # 2. postgresql
      # 3. oracle
      engine=mysql

      # IP or hostname of the database to connect to.
      host=hadoop-senior.ibeifeng.com

      # Port the database server is listening to. Defaults are:
      # 1. MySQL: 3306
      # 2. PostgreSQL: 5432
      # 3. Oracle Express Edition: 1521
      port=3306

      # Username to authenticate with when connecting to the database.
      user=root

      # Password matching the username to authenticate with when
      # connecting to the database.
      password=123456

      # Database options to send to the server when connecting.
      # https://docs.djangoproject.com/en/1.4/ref/databases/
      ## options=