记一次Maven优化过程，减少到分钟级别
Posted 风清扬逍遥子
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了记一次Maven优化过程，减少到分钟级别相关的知识，希望对你有一定的参考价值。
以前在工作中，生产开发中出现了一些问题，比较严重，构建的时候居然要1个小时以上，没法子只能排查，尝试改maven源码的方式进行优化，和大佬研究一番后验证没问题发出来作为分享！！
1、问题重现
Jenkins使用的maven版本3.8.5（我没想明白为啥要用这么新的，给我idea都换最新的了）
版本：apache-maven3.8.5
现象重现下：
当我把项目拉到本地，用maven去reimport的时候，发现内存非常高，并且cpu也被吃满了，于是我用jvm的调试工具看了下，发现内存不断涨，开发经常说本地配置了20G都不够，真头疼！
我因为本地配的是8G，这种就直接往上飙升，不带释放的，打了线程快照发现：
"main" #1 prio=5 os_prio=0 tid=0x000001ea3dcba000 nid=0x46c4 runnable [0x00000044ccef9000]
java.lang.Thread.State: RUNNABLE
at java.util.Collections$UnmodifiableMap.get(Collections.java:1456)
at java.util.AbstractMap.equals(AbstractMap.java:495)
at java.util.Collections$UnmodifiableMap.equals(Collections.java:1493)
at java.util.Objects.equals(Objects.java:59)
at org.eclipse.aether.artifact.AbstractArtifact.equals(AbstractArtifact.java:203)
at org.eclipse.aether.internal.impl.collect.DataPool$ConstraintKey.equals(DataPool.java:314)
at java.util.HashMap.getNode(HashMap.java:573)
at java.util.HashMap.get(HashMap.java:558)
at org.eclipse.aether.internal.impl.collect.DataPool.getConstraint(DataPool.java:155)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.cachedResolveRangeResult(DefaultDependencyCollector.java:616)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:392)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:506)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:362)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:349)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:254)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies(DefaultRepositorySystem.java:284)
at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:170)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:243)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:147)
at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:339)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:293)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:211)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:165)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:157)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:121)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:127)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:294)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:960)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:196)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
截图就很容易发现一个问题，这四个方法一直在循环；应该是这出现问题，但是是什么问题，暂时不清楚。
现象有了，我们一探究竟？
2、探路开始
先下载maven的源码：
下载地址：Index of /dist/maven/maven-3/3.8.5/source
下载下来后用idea打开，这里注意下idea的版本，要2022版本及以上可以
为了保证版本一致，也要下载下maven3.8.5的发行版（配置在idea和本地电脑的环境变量里）
下载release的地址：Maven – Download Apache Maven
下载下来后配置好环境变量（这里就不演示怎么配置了），先用idea打开maven源码，等待自己编译好后，可以斗把地主。
根据上面堆栈的截图，发现有个类DefaultDependencyCollector的4个方法一直在循环，doRecurse->process->processDependency->processDependency(重载)
接下来我们要用idea新建一个远程debug的application，记得端口改成监听8000，因为要用到mvnDebug工具，端口是8000。
然后这个时候突然发现DefaultDependencyCollector是在 maven-resolver-impl-1.6.3.jar这个依赖里的，所以我顺便把这个jar也下载下来了。
下载地址：Index of /dist/maven/resolver
同样，idea打开这个项目：
那么既然都到这里了，其实上面的maven源码就暂时可以不需要了，就先用maven-resolver-impl 这个源码进行调试，同样这里设定个remote debug：
调试开始，我首先打几个debug在那四个卡的方法上，于是进到我的项目里，执行下命令：
mvnDebug dependency:tree -Dverbose 其中-Dverbose 将当前所有的依赖关系都展示出来，包括来自不同处的依赖项。
回车后，此时进到maven-resolver-impl项目里，debug开启走起：
断点进来了，跟着走看看到底在干啥？显示几个参数
args：暂时说不好，但是知道意思
results：解析的结果
dependencies：项目解析的pom依赖
repositories：远程仓库的地址
depSelector：看样子是个选择器，来遍历每个dependency的坐标依赖深度的
depManager：依赖管理器，负责迭代深度用的
depTraverser：依赖的遍历器
verFilter：更不知道什么玩意
于是，把上面所有的参数带进入process后，发现要遍历所有的pom里的dependency坐标
@SuppressWarnings("checkstyle:parameternumber")
private void process(final Args args, Results results, List<Dependency> dependencies,
List<RemoteRepository> repositories, DependencySelector depSelector,
DependencyManager depManager, DependencyTraverser depTraverser, VersionFilter verFilter)
for (Dependency dependency : dependencies)
processDependency(args, results, repositories, depSelector, depManager, depTraverser, verFilter,
dependency);
具体的debug过程就不写了，蛮复杂的，有兴趣找我要源码，我注释写在里面了...
3、原因剖析（重点）
debug后我打了个日志，发现解析slf4j依赖的次数，高达9w次，这么恐怖的吗？
说下maven主要解析的思路：
debug后发现maven在解析依赖的时候，会解析到exclude这个结点，如果有1个项目A，引用了B和C的业务jar包，而业务jar包中出现exclude坐标，那么maven会把这个业务jar包中所有的内部依赖，全部都去排除这个jar，而不断的循环中出现了Spring的依赖在不断的解析，大概率是出现业务包里一旦排除了类似某些包，是在spring全家桶里的（springboot、springcloud等），譬如slf4j，logback等，那么依赖的B和C的业务jar中依赖全部都要排除，那么此时需要的spring依赖，maven会认为这些都不一样，会不断的放到迭代器里去递归解析，进而不断的解析里面的依赖关系，这就非常耗时，也就说明上面四个方法根本出不来的原因了，但是最后生成的spring相关的jar是一样的。
举个简单例子：B业务的pom依赖了spring，springcloud等，mybatis，其他第三方的，，包括自己业务代码....
而A项目中出现B的坐标，并且exclude slf4j的依赖，当去解析B这个依赖的时候，会去把B的pom全部解析一遍，并且里面的每个坐标，都剔除掉slf4j这个依赖，假设某个依赖spring-boot-starter里有这个jar（只是举例子），那么在去build里面的所有spring依赖，都会被认为是不一样的，正常来说只要发现了一个spring依赖（比如spring-core），就应该放在缓存里作为key，后面从缓存取，如果取到了就不去遍历，跳出循环，而真实情况是，B业务排除了slf4j，C业务排除了log4j，就会导致两个去遍历的时候，根据artifactId和exclude解析，认为的spring不会认为是你排除后，要得到的是同一个spring（但是实际上是一个spring），这个机制就是这样，自然每个都会去解析，从而递归去得到包，而最后的包，都是spring的包，没有不一样。
那么优化的思路就是针对自己的业务进行优化：遍历坐标，如果遇到我们的artifactId的坐标，就会认为不进行排除exclude，不走了exclude逻辑，就不会去每个业务包里都去执行exclude逻辑，最终的效果是会多出一些包，这些包原本应该是被exclude掉的，通过maven手动处理就行。
Maven本身机制没有问题，只是场景需要吧。
4、发包
接下来改好了源码后，我要打包，这里又是坑涉及到apache的rat证书，因为apache有一定的规范不让打包，不多解释了（mvn clean install -DskipTests -Drat.skip=true -Dcheckstyle.skip=true）
这个时候我就发布了一个包，验证后，本地内存4g不到就可以解析了，环境上从1h起步，到几分钟的优化，如果war中多出了一些包，用这个方法排除下就行：
<!-- 剔除无用的依赖 -->
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>3.2.3</version>
<configuration>
<packagingExcludes>
WEB-INF/lib/slf4j-simple-1.7.28.jar
</packagingExcludes>
</configuration>
</plugin>很久之前搞了一晚上的调试，自己研究研究后写出来大家分享下调试过程，记得感兴趣可以了解下！！
以上是关于记一次Maven优化过程，减少到分钟级别的主要内容，如果未能解决你的问题，请参考以下文章