Apache Drill 与 Kerberos

Posted 2023-02-16

【中文标题】Apache Drill 与 Kerberos

【英文标题】：Apache Drill with Kerberos

【发布时间】：2015-07-14 17:56:00

【问题描述】：

有谁知道如何使用 Apache Drill 启用 kerberos？是否可以。我似乎找不到任何关于它的文档，或者任何问题/答案与它的信息有关。我目前正在运行一个 CDH 集群。

我在尝试将 HDFS 与 Drill 一起使用时遇到此错误：

Error: PERMISSION ERROR: SIMPLE authentication is not enabled.  
Available:[TOKEN, KERBEROS]

【参考方案1】：

目前不支持/测试/记录 HDFS + Kerberos 集成。对此票进行投票以跟踪它何时可用：

https://issues.apache.org/jira/browse/DRILL-3584

【参考方案2】：

Drill 团队没有提供任何关于如何启用 kerberos 的文档，他们也没有使用 Drill 测试过 kerberos。钻探工程。确实相信它应该有效。

【讨论】：

是的，这就是我的怀疑。对于 HDFS 连接，它最终将是必要的。

【参考方案3】：

In order to gain access onto the cluster once Kerberized, you must configure certain files in order to gain access.
Make an HDFS Superuser account as indicated in this Cloudera doc. On the Main Node, run 

•sudo kadmin.local 
In addition, add an 'hdfs' principal with this command

•addprinc hdfs@LOCALDOMAIN   -- Where localdomain is the principal name

In order to enable authentication with Kerberos, we also need to copy the file hadoop-yarn-api.jar into Drill's class path. Example given below

•cp /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/client/hadoop-yarn-api.jar ~/apache-drill/jars/

The above step and the three following must be performed on each node of the cluster that an Apache Drill is installed.

Next, Drill's conf/core-site.xml file should be edited to contain the following snippet of xml. You might have to copy this file from /etc/hadoop/conf.cloudera.yarn/core-site.xml, etc or a similar path. 

<property>
  <name>hadoop.security.authentication</name>
  <value>kerberos</value>
</property>

After this step, you will also need to add the following xml snippet below to the drill core-site.xml file. In this instance, hdfs/_HOST@LOCALDOMAIN is my principal property. The property can be found on the hdfs-site.xml file

<property>
  <name>dfs.namenode.kerberos.principal</name>
  <value>hdfs/_HOST@LOCALDOMAIN</value>
</property>

All that is left to do is create an 'hdfs' Kerberos ticket for the user that we're logged into

•kinit hdfs   -- hdfs is the super user
Then start up each of the drillbits

•/opt/apachedrillfolder/bin/Drillbit.sh start
So now, Drill has both the configuration and the authority to use our kerberized HDFS store. Give it a shot by opening up a Drill prompt (drill-conf) and trying a query