关于hadoop登陆kerberos时设置环境变量问题的思考
Posted 风雨咒之无上密籍
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了关于hadoop登陆kerberos时设置环境变量问题的思考相关的知识,希望对你有一定的参考价值。
中心思想,设置kerberos环境变量时,发现JDK源码当中的一个问题,故描述如下。
在平时的使用中,如果hadoop集群配置kerberos认证的话,使用java访问hdfs或者hive时,需要先进行认证登陆,之后才可以访问。登陆代码大致如下:
package demo.kerberos; import java.io.IOException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.security.UserGroupInformation; public class KerberosLoginTest { public static void main(String[] args) throws IOException { // 设置kerberos相关登陆信息 // 方法1:设置krb5.conf配置文件 System.setProperty("java.security.krb5.conf", "/home/eabour/hadoop/conf/krb5.conf"); // 方法2:设置kerberos环境变量 // System.setProperty("java.security.krb5.realm", "HADOOP.COM"); // System.setProperty("java.security.krb5.kdc", "kdc.server.com"); // 开启登陆调试日志 System.setProperty("sun.security.krb5.debug", "true"); Configuration conf = new Configuration(); conf.addResource(new Path("/home/eabour/hadoop/conf/core-site.xml")); conf.addResource(new Path("/home/eabour/hadoop/conf/hdfs-site.xml")); UserGroupInformation.setConfiguration(conf); // 登陆kerberos UserGroupInformation.loginUserFromKeytab("[email protected]", "/home/eabour/test.keytab"); // TODO // 访问HDFS } }
设置krb5.conf和core-site.xml、hdfs-site.xml相关大数据平台的配置文件,用来初始化相关配置信息。使用krb5.conf登陆没有问题,不管设置多个kdc server还是单个,jdk中的相关模块都会解析出来,jdk源码位置为openjdkjdksrcshareclassessunsecuritykrb5,其中openjdk的源码地址为http://hg.openjdk.java.net/jdk7u/jdk7u60/jdk/file/33c1eee28403/src/share/classes,其他版本的类似,解析类为sun.security.krb5.Config,部分代码如下:
/** * Private constructor - can not be instantiated externally. */ private Config() throws KrbException { /* * If either one system property is specified, we throw exception. */ String tmp = getProperty("java.security.krb5.kdc"); if (tmp != null) { // The user can specify a list of kdc hosts separated by ":" defaultKDC = tmp.replace(‘:‘, ‘ ‘); } else { defaultKDC = null; } defaultRealm = getProperty("java.security.krb5.realm"); if ((defaultKDC == null && defaultRealm != null) || (defaultRealm == null && defaultKDC != null)) { throw new KrbException ("System property java.security.krb5.kdc and " + "java.security.krb5.realm both must be set or " + "neither must be set."); } // Always read the Kerberos configuration file try { Vector<String> configFile; String fileName = getJavaFileName(); if (fileName != null) { configFile = loadConfigFile(fileName); stanzaTable = parseStanzaTable(configFile); if (DEBUG) { System.out.println("Loaded from Java config"); } } else { boolean found = false; if (isMacosLionOrBetter()) { try { stanzaTable = SCDynamicStoreConfig.getConfig(); if (DEBUG) { System.out.println("Loaded from SCDynamicStoreConfig"); } found = true; } catch (IOException ioe) { // OK. Will go on with file } } if (!found) { fileName = getNativeFileName(); configFile = loadConfigFile(fileName); stanzaTable = parseStanzaTable(configFile); if (DEBUG) { System.out.println("Loaded from native config"); } } } } catch (IOException ioe) { // No krb5.conf, no problem. We‘ll use DNS or system property etc. } }
从代码可以看出,先从环境变量中获取java.security.krb5.kdc和java.security.krb5.realm,然后在读取配置文件java.security.krb5.conf。如果同时设置java.security.krb5.kdc、java.security.krb5.realm和java.security.krb5.conf,在登陆的时候,会使用java.security.krb5.kdc,除非登陆的realm与defaultRealm不一致,从代码中可以看出来。
那么,我要说的问题来了,在某些场景下,没有设置java.security.krb5.conf变量,而使用java.security.krb5.kdc、java.security.krb5.realm来登陆。在1个或多个kdc server情况下,这样的设置没有问题。我们知道kdc server的默认端口为88,使用的时UDP协议,当然可以为TCP协议,服务端口也可以修改,这时候大致为:
kdc=kdc.server.com:88
或者
kdc=kdc.server.com:1088
krb5.conf文件配置
[realms] HADOOP.COM = { kdc = kdc.server.com:1088 }
此时,设置环境变量:
System.setProperty("java.security.krb5.kdc", "kdc.server.com:1088");
那么,真正的问题就来了,执行登陆的话,会出现如下报错:
Caused by: GSSException: No valid credentials provided (Mechanism level: ICMP Port Unreachable) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:775) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) … 76 more Caused by: java.net.PortUnreachableException: ICMP Port Unreachable at java.net.PlainDatagramSocketImpl.receive0(Native Method) at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:143) at java.net.DatagramSocket.receive(DatagramSocket.java:812) at sun.security.krb5.internal.UDPClient.receive(NetClient.java:206) at sun.security.krb5.KdcComm$KdcCommunication.run(KdcComm.java:411) at sun.security.krb5.KdcComm$KdcCommunication.run(KdcComm.java:364) at java.security.AccessController.doPrivileged(Native Method) at sun.security.krb5.KdcComm.send(KdcComm.java:348) at sun.security.krb5.KdcComm.sendIfPossible(KdcComm.java:253) at sun.security.krb5.KdcComm.send(KdcComm.java:229) at sun.security.krb5.KdcComm.send(KdcComm.java:200) at sun.security.krb5.KrbTgsReq.send(KrbTgsReq.java:254) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:269) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:302) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:120) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693)
为什么会报错呢?因为在解析环境变量时,解析出问题了,现在,再来看看解析代码:
/** * Private constructor - can not be instantiated externally. */ private Config() throws KrbException { /* * If either one system property is specified, we throw exception. */ String tmp = getProperty("java.security.krb5.kdc"); if (tmp != null) { // The user can specify a list of kdc hosts separated by ":" defaultKDC = tmp.replace(‘:‘, ‘ ‘); } else { defaultKDC = null; } defaultRealm = getProperty("java.security.krb5.realm"); if ((defaultKDC == null && defaultRealm != null) || (defaultRealm == null && defaultKDC != null)) { throw new KrbException ("System property java.security.krb5.kdc and " + "java.security.krb5.realm both must be set or " + "neither must be set."); }
请看红色黄底的代码 defaultKDC = tmp.replace(‘:‘, ‘ ‘) ,对,就是这句代码的问题,他将kdc.server.com:1088分割为kdc.server.com和1088了,认为时两个kdc server。我的心崩溃呀,为什么要用冒号来分割多个server的配置,如果在使用默认端口的话,这样也没问题,但是,如果kdc修改了端口,这种通过环境变量设置kdc server的方式就没法用了。
到最后,原来时jdk源码的问题,正常的途径怕是设置不了了。但是,不是不能修改了,我们可以用反射来修改Config的defaultKDC的值,虽然说defaultKDC为final String ,到那时它是private final String defaultKDC;,所以还是可以修改的。
以上就是我分析的问题,在实际项目中是真实遇到过,因为涉及JDK底层代码,所以请大家来参详一下。
附:
1.kerberos配置文件设置:https://www.ibm.com/support/knowledgecenter/zh/SSAW57_9.0.0/com.ibm.websphere.nd.multiplatform.doc/ae/tsec_kerb_create_conf.html
以上是关于关于hadoop登陆kerberos时设置环境变量问题的思考的主要内容,如果未能解决你的问题,请参考以下文章