Jvm加载jar包的顺序

Posted 2020-10-11 起名什么的...

   使用-XX:+TraceClassPaths或者在服务器上执行jinfo时，都能得到classpath包含的jar包，例如：

java.class.path = local/aaa/lib/spring-data-redis-1.8.3.RELEASE.jar:/usr/local/aaa/lib/spring-tx-4.3.8.RELEASE.jar:/usr/local/aaa/lib/spring-jdbc-4.3.7.RELEASE.jar:/usr/local/aaa/lib/classmate-1.3.1.jar:/usr/local/aaa/lib/javax.servlet-api-3.1.0.jar:/usr/local/aaa/lib/mongodb-driver-3.4.2.jar:/usr/local/aaa/lib/xml-apis-2.0.2.jar:/usr/local/aaa/lib/ufc-api-utils-2.0.0.jar:/usr/local/aaa/lib/log4j-over-slf4j-1.7.25.jar:/usr/local/aaa/lib/tomcat-embed-websocket-8.5.14.jar:...

   这些jar的顺序不同的机器总是不一样的，平时没有问题，所以也没有细想过，这些jar包的顺序为什么会不一样的。    在之前排查的一个问题 的结尾还留了一个问题，为什么有的机器会加载正确的类，有的就是错的。因为这一段在上线一个项目，灰度公测阶段，所以拖了些天，穿插着看了看加载相关的一些hotspot代码，以前也没看过，就边猜测边看了。因为是穿插着看，怕今天看的明天就忘了，所以博客结尾连接中流水账记录了我看的过程。

   总之，经过一番查代码，最终在rt.jar中找到了JarFile这个类，代码在jdk的src.zip包里的。    用 https://github.com/saaavsaaa/warn-report/blob/master/src/main/java/report/btrace/JarBTrace.java 脚本跟踪了一下：

---

   正常的服务器脚本的输出：

---

   异常的服务器上脚本输出：

---

   这个问题在多台服务器上都出现了，所以搜集多台的输出后可以确认这个规律，其实在结尾连接中代码记录里也可以看出，如果有多个同名的类只会加载其中第一个。在不出问题的服务器上commons-codec-1.10.jar是在http-1.1.0.jar之前加载的，而出问题的服务器正好相反。

---

   /usr/lib/jvm/java-8-oracle/jre/lib/rt.jar!/sun/misc/URLClassPath.class中的private ArrayList path有所有加载的jar的URL。这个类里还有每个jar的loader：ArrayList<URLClassPath.Loader> loaders，jar路径和jarLoader的映射：HashMap<String, URLClassPath.Loader> lmap。lmap通过private synchronized URLClassPath.Loader getLoader(int var1)方法从Stack urls中弹出的jar来构造loader并加入映射的。有两个URLClassLoader加载不同的URLClassPath实例，一个只有jre/lib下的几个jar，一个有所有的。不过这都不重要，这些jar都是从classpath加载的。于是我在几个服务器上用jinfo输出了java.class.path，并对比了一下发现正常服务器和服务器中，这两个jar的顺序果然是不一样的。那么我估计问题是出现jvm对java.class.path赋值的前面了，或许是操作系统什么的。

   于是决定看看java.class.path初始化的情况，代码在/home/aaa/Github/hotspot/src/share/vm/runtime/arguments.cpp:_java_class_path = new SystemProperty("java.class.path", "", true)。SystemProperty的构造在/home/aaa/Github/hotspot/src/share/vm/runtime/arguments.hpp：

---

 // Constructor
 SystemProperty(const char* key, const char* value, bool writeable) {
   if (key == NULL) {
     _key = NULL;
   } else {
     _key = AllocateHeap(strlen(key)+1, mtInternal);
     strcpy(_key, key);
   }
   if (value == NULL) {
     _value = NULL;
   } else {
     _value = AllocateHeap(strlen(value)+1, mtInternal);
     ba(_value, value);
   }
   _next = NULL;
   _writeable = writeable;
 }
};

---

   构造里似乎没什么关系，然后回到cpp仔细看了下，发现：

    // following are JVMTI agent writeable properties.
    // Properties values are set to NULL and they are
    // os specific they are initialized in os::init_system_properties_values().
     
    // Set OS specific system properties values
    os::init_system_properties_values();

   这个方法的定义在os.hpp中，不过实现是不同的系统不一样，所以并没有在对应的cpp中，我是linux系统，所以我去找了 /home/aaa/Github/hotspot/src/os/linux/vm/os_linux.cpp，不过也没有我想要的。arguments.cpp中只找到了对endorsed，ext，bootclasspath的目录文件读取。

---

   然后找到了-XX:+TraceClassPaths参数的输出位置：    [classpath: ...]的调用链：

/home/aaa/Github/hotspot/src/share/vm/runtime/thread.cpp:
// Parse arguments
jint parse_result = Arguments::parse(args);

     // Parse JAVA_TOOL_OPTIONS environment variable (if present)
     jint result = parse_java_tool_options_environment_variable(&scp, &scp_assembly_required);
     if (result != JNI_OK) {
       return result;
     }

     // Parse JavaVMInitArgs structure passed in
     result = parse_each_vm_init_arg(args, &scp, &scp_assembly_required, Flag::COMMAND_LINE);
     if (result != JNI_OK) {
       return result;
     }

     // Parse _JAVA_OPTIONS environment variable (if present) (mimics classic VM)
     result = parse_java_options_environment_variable(&scp, &scp_assembly_required);
     if (result != JNI_OK) {
       return result;
     }

    parse_java_tool_options_environment_variable和parse_java_options_environment_variable都有调用parse_each_vm_init_arg，这里提一句_JAVA_OPTIONS会覆盖JAVA_TOOL_OPTIONS的同key配置。

    parse_each_vm_init_arg这方法一进来就看到这么一段略坑，不过也无所谓了响。。。

    if (!match_option(option, "-Djava.class.path", &tail) &&
        !match_option(option, "-Dsun.java.command", &tail) &&
        !match_option(option, "-Dsun.java.launcher", &tail)) {

        // add all jvm options to the jvm_args string. This string
        // is used later to set the java.vm.args PerfData string constant.
        // the -Djava.class.path and the -Dsun.java.command options are
        // omitted from jvm_args string as each have their own PerfData
        // string constant object.
        build_jvm_args(option->optionString);
    }

  这就是打印上面那句的地方：  

     jint Arguments::parse_each_vm_init_arg
     Arguments::fix_appclasspath():
       if (!PrintSharedArchiveAndExit) {
         ClassLoader::trace_class_path(tty, "[classpath: ", _java_class_path->value());
       }

   然后[Bootstrap loader class path=/usr/lib/jvm/java-8-oracle/jre/lib/resources.jar:/usr/lib/jvm/java-8-oracle/jre/lib/rt.jar:/usr/lib/jvm/java-8-oracle/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jsse.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jce.jar:/usr/lib/jvm/java-8-oracle/jre/lib/charsets.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jfr.jar:/usr/lib/jvm/java-8-oracle/jre/classes]这一句输出:

    /home/aaa/Github/hotspot/src/share/vm/runtime/init.cpp
    jint init_globals()

    /home/aaa/Github/hotspot/src/share/vm/classfile/classLoader.cpp
    void classLoader_init() {
      ClassLoader::initialize();
    }

    void ClassLoader::initialize()

    void ClassLoader::setup_bootstrap_search_path()

   大体上代码都在这一部分了，可以看出代码中并没有对加载顺序有所改变，然后比较重要的线索就是加载方法用的是opendir和readdir函数，在ios同事的协助下，终于确定了，问题就是jar的加载顺序问题，而这个顺序实际上是由文件系统决定的，linux内部是用inode来指示文件的。

   于是写了个简单的c方法来验证，打印inode号：https://github.com/saaavsaaa/warn-report/blob/master/src/main/java/report/main.c

  d_off:4066763678044974396 d_name: commons-codec-1.10.jar
  d_off:4317940688349827723 d_name: commons-lang-2.5.jar
  d_off:4319988583919237504 d_name: jul-to-slf4j-1.7.25.jar
  d_off:4356646752930279233 d_name: spring-aop-4.3.8.RELEASE.jar
  d_off:4579877846802590742 d_name: tomcat-embed-core-8.5.14.jar
  d_off:4731608207059974753 d_name: groovy-2.4.3.jar
  d_off:4771541818858258978 d_name: jpush-client-3.2.9.jar
  d_off:4817159055427520488 d_name: httpcore-4.3.jar
  d_off:5037058976412869958 d_name: rocketmq-common-3.2.6.jar
  d_off:5112026581585883935 d_name: xmlParserAPIs-2.6.2.jar
  d_off:5223887631628650300 d_name: commons-fileupload-1.2.2.jar
  d_off:5270962929984509613 d_name: logback-core-1.1.11.jar
  d_off:5295594039709144434 d_name: jackson-core-2.8.8.jar
  d_off:5313324231349868064 d_name: .
  d_off:5369671282559728007 d_name: spring-webmvc-4.3.8.RELEASE.jar
  d_off:5480616600783493234 d_name: xalan-2.6.0.jar
  d_off:5598132018198955916 d_name: unbescape-1.1.0.RELEASE.jar
  d_off:5620136379821311472 d_name: spring-boot-starter-aop-1.5.3.RELEASE.jar
  d_off:5639942392021544761 d_name: mybatis-3.4.4.jar
  d_off:5688537857783605585 d_name: validation-api-1.1.0.Final.jar
  d_off:5689351964973998306 d_name: json-lib-2.4-jdk15.jar
  d_off:5708471019688158398 d_name: tagsoup-0.9.7.jar
  d_off:5716046632217600256 d_name: xom-1.0b3.jar
  d_off:5736731302988875630 d_name: p2p-repository-1.0-SNAPSHOT.jar
  d_off:5770969695350360533 d_name: ognl-3.0.8.jar
  d_off:5946256519116188501 d_name: jackson-annotations-2.8.0.jar
  d_off:6011005565981751131 d_name: jxl-2.6.jar
  d_off:6121401373763401899 d_name: spring-data-mongodb-1.10.3.RELEASE.jar
  d_off:6155887434083949861 d_name: icu4j-2.6.1.jar
  d_off:6203694621577639938 d_name: jboss-logging-3.3.0.Final.jar
  d_off:6241670413890043636 d_name: jaxme-api-0.3.jar
  d_off:6317802240634228683 d_name: thymeleaf-2.1.5.RELEASE.jar
  d_off:6362118033392674189 d_name: logback-classic-1.1.11.jar
  d_off:6391821784225315730 d_name: snakeyaml-1.17.jar
  d_off:6447926989912518869 d_name: javassist-3.16.1-GA.jar
  d_off:6586996539318953387 d_name: spring-boot-1.5.3.RELEASE.jar
  d_off:6682174700565688505 d_name: mybatis-spring-1.3.1.jar
  d_off:6858079168157560275 d_name: http-1.1.0.jar

   对照前面正常服务器的截图可以发现，顺序和这是一样的。而出错的服务器：

  d_off:7500897893766328572 d_name: http-1.1.0.jar
  d_off:7630237192571233449 d_name: jaxen-1.1-beta-4.jar
  d_off:7846439931967980783 d_name: xom-1.0b3.jar
  d_off:7986273690820996399 d_name: jxl-2.6.jar
  d_off:8013065173263952359 d_name: spring-boot-starter-thymeleaf-1.5.3.RELEASE.jar
  d_off:8231450206996036007 d_name: spring-context-support-4.3.8.RELEASE.jar
  d_off:8471297127500795042 d_name: jpush-client-3.2.9.jar
  d_off:8635726305307688944 d_name: commons-codec-1.10.jar

   

    同样可以对照上面错误服务器的截图，而且一般情况下，修改了文件名，再改回来，或者从新上传一个，这个编号依然还是这个，所以在出问题的服务器上问题稳定复现。那么，这个问题终于是可以告一段落了，原因搞清楚了，心理就有底了。解决什么的就随便了，我就随便把名字改成zzzhttp然后问题就在两个包都存在的情况下解决了。

  d_off:8635726305307688944 d_name: commons-codec-1.10.jar
  d_off:8710228832141418589 d_name: xercesImpl-2.6.2.jar
  d_off:8722513994996448409 d_name: android-json-0.0.20131108.vaadin1.jar
  d_off:8754081964290049159 d_name: ufc-api-utils-2.0.0.jar
  d_off:8830796801498266528 d_name: httpcore-4.3.jar
  d_off:8854152647200610772 d_name: tomcat-jdbc-8.5.11.jar
  d_off:8971846312288780129 d_name: spring-tx-4.3.8.RELEASE.jar
  d_off:8986236906371055996 d_name: bcprov-jdk15-1.45.jar
  d_off:8988385379950226997 d_name: mysql-connector-java-5.1.30.jar
  d_off:8994464230154278010 d_name: p2p-common-1.0-SNAPSHOT.jar
  d_off:9012703293696799571 d_name: mybatis-spring-boot-starter-1.3.0.jar
  d_off:9057592519440006836 d_name: zzzhttp-1.1.0.jar

---

　　流水账记录：https://saaavsaaa.github.io/aaa/Java_Class_Path.html

 

微信公众号：