热修复 笔记 第三部分 优化篇

Posted 2022-11-30 xzj_2013

热修复的pre-verify问题原理分析

上一篇实战的时候遗留了一个问题
在android4.4运行时出现了这样的一个崩溃：

 java.lang.IllegalAccessError: Class ref in pre-verified class resolved to unexpected implementation

那么这是个什么错误呢？
直接翻译：一个被标为pre-verify的class引用了一个ref类，这个ref类被发现不是期待的实现方式

问题原因的简单介绍

回顾下上篇的修复原理，以及我们的实战，我们是将Utils以及调用它的Activity放在同一个dex下，且只调用了同一个Dex下的类，这也就是问题出现的原因。

如果MainActivity类中只引用了：Utils类。当打包dex时， MainActivity与Utils都在classes.dex中，则MainActivity类被标记为 CLASS_ISPREVERIFIED。

如果使用补丁包中的Utils类取代出现bug的Utils，则会导致MainActivity与其引用的Utils不在同一个Dex，但MainActivity已经被打上标记，此时出现冲突。导致校验失败！

问题原因的源码分析：
我们去看一下异常抛出的位置以及如何调用到这个位置：
通过字符串搜索，我们在/dalvik/vm/oo/Resolve.cpp这个类中发现了这个异常的输出,就在dvmResolveClass方法中：

/*
36 * Find the class corresponding to "classIdx", which maps to a class name
37 * string.  It might be in the same DEX file as "referrer", in a different
38 * DEX file, generated by a class loader, or generated by the VM (e.g.
39 * array classes).
40 *
41 * Because the DexTypeId is associated with the referring class' DEX file,
42 * we may have to resolve the same class more than once if it's referred
43 * to from classes in multiple DEX files.  This is a necessary property for
44 * DEX files associated with different class loaders.
45 *
46 * We cache a copy of the lookup in the DexFile's "resolved class" table,
47 * so future references to "classIdx" are faster.
48 *
49 * Note that "referrer" may be in the process of being linked.
50 *
51 * Traditional VMs might do access checks here, but in Dalvik the class
52 * "constant pool" is shared between all classes in the DEX file.  We rely
53 * on the verifier to do the checks for us.
54 *
55 * Does not initialize the class.
56 *
57 * "fromUnverifiedConstant" should only be set if this call is the direct
58 * result of executing a "const-class" or "instance-of" instruction, which
59 * use class constants not resolved by the bytecode verifier.
60 *
61 * Returns NULL with an exception raised on failure.
62 */
63ClassObject* dvmResolveClass(const ClassObject* referrer, u4 classIdx,
64    bool fromUnverifiedConstant)
65
66    DvmDex* pDvmDex = referrer->pDvmDex;
67    ClassObject* resClass;
68    const char* className;
69
70    /*
71     * Check the table first -- this gets called from the other "resolve"
72     * methods.
73     */
74    resClass = dvmDexGetResolvedClass(pDvmDex, classIdx);
75    if (resClass != NULL)
76        return resClass;
77
78    LOGVV("--- resolving class %u (referrer=%s cl=%p)",
79        classIdx, referrer->descriptor, referrer->classLoader);
80
81    /*
82     * Class hasn't been loaded yet, or is in the process of being loaded
83     * and initialized now.  Try to get a copy.  If we find one, put the
84     * pointer in the DexTypeId.  There isn't a race condition here --
85     * 32-bit writes are guaranteed atomic on all target platforms.  Worst
86     * case we have two threads storing the same value.
87     *
88     * If this is an array class, we'll generate it here.
89     */
90    className = dexStringByTypeIdx(pDvmDex->pDexFile, classIdx);
91    if (className[0] != '\\0' && className[1] == '\\0') 
92        /* primitive type */
93        resClass = dvmFindPrimitiveClass(className[0]);
94     else 
95        resClass = dvmFindClassNoInit(className, referrer->classLoader);
96    
97
98    if (resClass != NULL) 
99        /*
100         * If the referrer was pre-verified, the resolved class must come
101         * from the same DEX or from a bootstrap class.  The pre-verifier
102         * makes assumptions that could be invalidated by a wacky class
103         * loader.  (See the notes at the top of oo/Class.c.)
104         *
105         * The verifier does *not* fail a class for using a const-class
106         * or instance-of instruction referring to an unresolveable class,
107         * because the result of the instruction is simply a Class object
108         * or boolean -- there's no need to resolve the class object during
109         * verification.  Instance field and virtual method accesses can
110         * break dangerously if we get the wrong class, but const-class and
111         * instance-of are only interesting at execution time.  So, if we
112         * we got here as part of executing one of the "unverified class"
113         * instructions, we skip the additional check.
114         *
115         * Ditto for class references from annotations and exception
116         * handler lists.
117         */
118        if (!fromUnverifiedConstant &&
119            IS_CLASS_FLAG_SET(referrer, CLASS_ISPREVERIFIED))
120        
121            ClassObject* resClassCheck = resClass;
122            if (dvmIsArrayClass(resClassCheck))
123                resClassCheck = resClassCheck->elementClass;
124
125            if (referrer->pDvmDex != resClassCheck->pDvmDex &&
126                resClassCheck->classLoader != NULL)
127            
128                ALOGW("Class resolved by unexpected DEX:"
129                     " %s(%p):%p ref [%s] %s(%p):%p",
130                    referrer->descriptor, referrer->classLoader,
131                    referrer->pDvmDex,
132                    resClass->descriptor, resClassCheck->descriptor,
133                    resClassCheck->classLoader, resClassCheck->pDvmDex);
134                ALOGW("(%s had used a different %s during pre-verification)",
135                    referrer->descriptor, resClass->descriptor);
136                dvmThrowIllegalAccessError(
137                    "Class ref in pre-verified class resolved to unexpected "
138                    "implementation");
139                return NULL;
140            
141        
142
143        LOGVV("##### +ResolveClass(%s): referrer=%s dex=%p ldr=%p ref=%d",
144            resClass->descriptor, referrer->descriptor, referrer->pDvmDex,
145            referrer->classLoader, classIdx);
146
147        /*
148         * Add what we found to the list so we can skip the class search
149         * next time through.
150         *
151         * TODO: should we be doing this when fromUnverifiedConstant==true?
152         * (see comments at top of oo/Class.c)
153         */
154        dvmDexSetResolvedClass(pDvmDex, classIdx, resClass);
155     else 
156        /* not found, exception should be raised */
157        LOGVV("Class not found: %s",
158            dexStringByTypeIdx(pDvmDex->pDexFile, classIdx));
159        assert(dvmCheckException(dvmThreadSelf()));
160    
161
162    return resClass;
163

那我们继续跟踪dvmResolveClass的调用

通过搜索，我们发现这个方法的调用都是在实例化的时候调用的；

HANDLE_OPCODE(OP_NEW_INSTANCE /*vAA, class@BBBB*/)
   
       ClassObject* clazz;
       Object* newObj;

       EXPORT_PC();

       vdst = INST_AA(inst);
       ref = FETCH(1);
       ILOGV("|new-instance v%d,class@0x%04x", vdst, ref);
       clazz = dvmDexGetResolvedClass(methodClassDex, ref);
       if (clazz == NULL) 
           clazz = dvmResolveClass(curMethod->clazz, ref, false);
           if (clazz == NULL)
               GOTO_exceptionThrown();
       

通过跟踪我们也会发现解释器执行到new-instance时，会触发，最终会调用到dvmResolvedClass方法；

然后我们看异常，也是在调用Utils的类 就是MainActivity.class中抛出的
那么我们跟踪MainActivity.class的初始化：
具体的应用加载的流程：
可以参考老罗的博客Android应用程序启动过程源代码分析
那我们先去看ActivityThread的源码
很快我们就发现了：

继续往下看

这里我们发现了activity的初始化

看完了整个流程，我们回归dvmResolvedClass方法
referrer是curMethod->clazz , 首先在dvmDexGetResolvedClass方法中判断是否解析过该类，很明显，该类是首次加载，所以返回结果为空，然后调用dvmFindClassNoInit方法用classloader去查找类，因为patch.dex已经在之前反射注入到了elements中，所以此时resClass不为空，此时检查MainActivity是否被打上了CLASS_ISPREVERIFIED，此时先给出结果，肯定是打上了的，进而转入到
if (referrer->pDvmDex != resClassCheck->pDvmDex && resClassCheck->classLoader != NULL)
这行代码翻译过来就是MainActivity所在的dex和Test所在的dex不是同一个且Test的类加载器不为空的情况下，就会抛出异常 “Class ref in pre-verified class resolved to unexpected implementation”，现在大家都应该清楚了这个异常具体的来源。

发现问题原因了，如何解决？
盗用腾讯bugly的一张图

当三个条件均满足时，会抛出异常，解决方案大致上有以下四种。

• 修改fromUnverfiedConstant=true
  需要通过 native hook 拦截系统方法，更改方法的入口参数，将 fromUnverifiedConstant 统一改为 true，风险大，几乎无人采用。
• 禁止dexopt过程打上CLASS_ISPREVERIFIED标记
  Q-zone方案突破了此限制，但是损失了性能。
• 补丁类与引用类放在同一个dex中
  Tinker等全量合成方案突破了此限制。
• 使dvmDexGetResolvedClass返回不为null，直接返回
  QFix的方案，可参考这篇文章QFix探索之路—手Q热补丁轻量级方案

各个方案都有各自的优缺点。
上一篇实战我们使用的就是Q-zone方案
那么我们就分析Q-zone方案原理是在每个类的构造方法中加入一行代码,保证Hack.class在单独的dex中，选择在构造函数中进行可以不增加方法数。如下：

public class Test 
  public Test() 
    System.out.println(Hack.class);
  

我们从源码的角度看一下，为什么加入了这行代码，每个插入的类中都不会打上CLASS_ISPREVERIFIED了。
dexopt的过程是分为verify+optimize两个步骤进行的，对于每个类的verify+optimize方法是在verifyAndOptimizeClass方法中进行的，源码位置在：
/dalvik/vm/analysis/DexPrepare.cpp

/*
 * Verify and/or optimize a specific class.
 */
 static void verifyAndOptimizeClass(DexFile* pDexFile, ClassObject* clazz,
 const DexClassDef* pClassDef, bool doVerify, bool doOpt)

    const char* classDescriptor;
    bool verified = false;

    if (clazz->pDvmDex->pDexFile != pDexFile) 
    /*
     * The current DEX file defined a class that is also present in the
     * bootstrap class path.  The class loader favored the bootstrap
     * version, which means that we have a pointer to a class that is
     * (a) not the one we want to examine, and (b) mapped read-only,
     * so we will seg fault if we try to rewrite instructions inside it.
     */
     ALOGD("DexOpt: not verifying/optimizing '%s': multiple definitions",
         clazz->descriptor);
        return;
      
    classDescriptor = dexStringByTypeIdx(pDexFile, pClassDef->classIdx);
    /*
     * First, try to verify it.
     */
    if (doVerify) 
       if (dvmVerifyClass(clazz)) 
           /*
            * Set the "is preverified" flag in the DexClassDef.  We
            * do it here, rather than in the ClassObject structure,
            * because the DexClassDef is part of the odex file.
            */
         assert((clazz->accessFlags & JAVA_FLAGS_MASK) ==           
         pClassDef->accessFlags);
         ((DexClassDef*)pClassDef)->accessFlags |= CLASS_ISPREVERIFIED;
          verified = true; 
           else 
          		// TODO: log when in verbose mode
          		ALOGV("DexOpt: '%s' failed verification", classDescriptor);
           
     
 if (doOpt) 
      bool needVerify = (gDvm.dexOptMode == OPTIMIZE_MODE_VERIFIED ||
       gDvm.dexOptMode == OPTIMIZE_MODE_FULL);
        if (!verified && needVerify) 
 			 ALOGV("DexOpt: not optimizing '%s': not verified",  classDescriptor);
		  else 
		 		 dvmOptimizeClass(clazz, false);
				/* set the flag whether or not we actually changed anything */
		      ((DexClassDef*)pClassDef)->accessFlags |= CLASS_ISOPTIMIZED;
 		
  

很清晰，dvmVerifyClass如果校验通过了，该clazz就会被打上CLASS_ISPREVERIFIED标记。接下来我们主要看dvmVerifyClass方法都干了什么。源码位置：/dalvik/vm/analysis/DexVerify.cpp

/*
 * Verify a class.
 *
 * By the time we get here, the value of gDvm.classVerifyMode should already
 * have been factored in.  If you want to call into the verifier even
 * though verification is disabled, that's your business.
 *
 * Returns "true" on success.
 */
bool dvmVerifyClass(ClassObject* clazz)

    int i;

    if (dvmIsClassVerified(clazz)) 
        ALOGD("Ignoring duplicate verify attempt on %s", clazz->descriptor);
        return true;
    

    for (i = 0; i < clazz->directMethodCount; i++) 
        if (!verifyMethod(&clazz->directMethods[i])) 
            LOG_VFY("Verifier rejected class %s", clazz->descriptor);
            return false;
        
    
    for (i = 0; i < clazz->virtualMethodCount; i++) 
        if (!verifyMethod(&clazz->virtualMethods[i])) 
            LOG_VFY("Verifier rejected class %s", clazz->descriptor);
            return false;
        
    

    return true;

在verifyMethod中会对Method的各个字段进行验证，篇幅原因，不进行逐层源码追踪了，在verifyMethod方法中，会调用dvmVerifyCodeFlow方法，接着调用doCodeVerification，会具体分析每一条指令，执行必要的解析及验证。对于每一条指令，是调用verifyInstruction方法来验证的。verifyInstruction方法的源码位置：/dalvik/vm/CodeVerify.cpp。
在verifyInstruction中，注意这段代码。

为什么要关注OP_CONST_CLASS,因为我们插入的System.out.println(Hack.class);会生成const-class的dalvik指令，可以通过dexdump或者反编译apk来查看，此时会触发dvmOptResolveClass的调用。dvmOptResolveClass函数会去查找Hack.class,由于我们的dex没有Hack.class,肯定查不到，抛异常返回，此时这个类的dvmVerifyClass过程会返回false，这个类也就没有打上CLASS_ISPREVERIFIED，而verified为false，导致也不会进行optimize过程。

值得说明的是如果类没有打上CLASS_ISPREVERIFIED，那么verify+optimize都会在类第一次加载时dvmInitClass中进行，正常情况下每个类的verify+optimize只会在安装时dexopt中进行一次，verify过程非常重，会对类的所有方法的所有指令都进行校验，如果短时间内，大量的类进行verify，耗时是比较严重的，尤其在应用刚启动的时候，有可能造成白屏；
这也是我们为什么说Qzone的修改方案很损失性能的原因；

至于我们如何插入System.out.println(Hack.class),我们可以采用ASM进行实现。实现过程注意两点：

• Application不要插入Hack.class，因为application的构造函数执行时，我们还没有注入hack.apk
• 在注入patch.dex前注入hack.apk,否则会找不到类

实战修复CLASS_ISPREVERIFIED

我使用Qzone 禁止dexopt过程打上CLASS_ISPREVERIFIED标记的方案实现修复CLASS_ISPREVERIFIED问题；
根据上面说的原理，CLASS_ISPREVERIFIED标签被打上的原因，是因为MainActivity只引用了当前Dex包内的class而导致，那么我们只需要在该Dex内除了Application的所有class中都引用一个在另外一个Dex的class即可避免打上CLASS_ISPREVERIFIED标签，这个就是Qzone的修复方案。