CRC32 参数逆向工程可以访问多个示例

Posted 2023-02-19

【中文标题】CRC32 参数逆向工程可以访问多个示例

【英文标题】：CRC32 Parameters Reverse Engineering having access to multiple examples

【发布时间】：2019-08-19 20:42:25

【问题描述】：

我必须找出如何重现用于专有数据库文件的 CRC32 算法，该文件由许多 128 字节的“块”组成，每个块都是一条记录。我知道对于每条记录，字节 1-4 是 CRC32 校验和，接下来的 35 个字节似乎无关紧要，因为我可以轻松更改它们，而无需应用程序告诉我 CRC 校验失败。因此，我希望找出用于计算后者的多项式和其他参数。下面是一个例子。

文字版：

00 27 AE 3B 9F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 08 41 41 41 41 41 41 41 41 
19 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 
42 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00

如果我们只取我们无法更改的字节，打破记录，我们会得到：

41 08 41 41 41 41 41 41 41 41 19 42 42 42 42 42 42 42 42 42 42 42 42 42 42 
42 42 42 42 42 42 42 42 42 42 42 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00

上面的 CRC32 是 27 AE 3B 9F

真实记录示例 1.1，与上述仅相差一个字节（CRC 为 BC D4 84 FB）：

41 08 41 41 41 41 41 41 41 41 19 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 
42 42 42 42 42 42 42 42 42 43 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00

真实记录示例 2（输出 CRC 为 3B 6A D1 AF）：

41 07 41 41 41 41 41 41 41 00 19 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 
42 42 42 42 42 42 42 42 42 42 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00

真实记录示例 3（输出 CRC 为 0B 54 CC 09）：

41 01 31 00 00 00 00 00 00 00 03 41 73 61 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00

真实记录示例 4（输出 CRC 为 12 91 EA 8E）：

41 B4 A8 D0 02 46 00 B4 A8 00 03 52 4D 31 03 53 54 50 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FF 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 25 00 00 00 00 00 00 00 
00 00 A3 05 00 00 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 
00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

真实记录示例 5（输出 CRC 为 8A 68 00 3B）：

41 B4 A8 D0 02 46 00 B4 A8 01 03 52 4D 31 03 53 54 50 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FF 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 25 00 00 00 00 00 00 00 
00 00 A3 05 00 00 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 
00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 64 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

最后两条记录只有一个字节不同。通过使用@rcgldr 指定的方法，我能够获得 0x9902539d 的最终 Xor 值，并且我可以成功地更改数据而不会出现应用程序抱怨。我运行了一些代码来为应用程序上的每个实体/文件找到这些最终的异或值，并且在所有这些上都成功了，但是能够找到单个 crc 参数集将是一个很好的补充。

编辑：添加了另外两个示例记录

编辑 2：添加了一个与第一个字节相比仅与一个字节不同的示例

编辑 3：添加了另外两个大小不同的示例，它们来自应用程序中的另一种类型的记录。也删除了部分问题，因为它变得无关紧要

【问题讨论】：

如果你有公式，输入和输出值，你可以使用SMT求解器找出公式中的未知系数。

我不太确定我是否遵循了对这两个记录进行异或运算的方法，我可以在任何地方看到这一点吗？感谢您的回答，顺便说一句，我会继续研究如何同时完成这项工作。

这将有助于发布更多示例记录，尤其是两条记录之间的差异很小。如果您可以找到除了数据中的 crc 和一个字节之外的两条相同的记录，那么对这两条记录进行异或操作将简化对 CRC 进行逆向工程的尝试，因为有一些方法可以加快生成测试用例 crc 的速度是很多零，或者最好的情况是数据中的单个非零字节。

谢谢，刚刚又添加了两条示例记录。

0x19 和 0x03 分别只是属性的大小，所以对于前两条记录，我们可以将该值解析为由 25 个 B 组成的字符串，最后一个示例只是转换为“AsA ”。这些只是您在读取此文件的应用程序上看到的值。

【参考方案1】：

异或 1.0 和 1.1 的结果是：

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00

对两个 crcs 进行异或运算得到 ​​p>

9b 7a bf 64

假设存储的crc为“little endian”，则计算出的crc为

0x64bf7a9b

通过对两条记录进行异或，由于异或，初始值和最终异或值被抵消，这允许仅根据数据确定crc多项式，假设初始值= 0并且最终异或值= 0。利用这一点，我尝试了一些常见的crc多项式，确定crc多项式是

0x104C11DB7 or ignoring the msb: 0x04C11DB7

使用您在评论中链接到的网站：

http://www.sunshine2k.de/coding/javascript/crc/crc_js.html

参数为：

crc32
custom
input:  not reflected
result: not reflected
polynomial: 0x04C11DB7
initial value: 0x0
final xor value: 0x0

如果数据的大小始终相同，则可以使用初始值或最终 xor 值或两者的组合来调整 crc，使其与示例中显示的实际 crc 匹配，但这是最简单的使用最终 xor 来匹配示例，因为它只需要使用示例之一计算 crc，假设初始值 = 0 并且最终 xor 值 = 0，然后将计算出的 crc 与示例 crc 中的实际 crc 进行异或计算特定长度数据的最终异或值。

因此，对于第一个示例中的数据大小，最终的异或值 0x189B52BC 将生成与示例匹配的 crc。这些是 crc 计算器的参数。

crc32
custom
input:  not reflected
result: not reflected
polynomial: 0x04C11DB7
initial value: 0x0
final xor value: 0x189B52BC

这些参数与您发布的所有第一个示例相匹配。再次注意，crc 是“小端”存储的，最高有效字节在前。

如果数据大小是可变的，则需要一个初始值（并且可能同时使用初始值和最终异或值）。一旦知道多项式，就可以执行“反向”CRC 来找到初始值，或者可以使用蛮力搜索。我使用快速 crc 计算器对初始值进行了蛮力搜索（因为我还没有“反向”CRC 程序），它似乎适用于任何数据大小，至少基于新示例你补充说。这些参数适用于上述所有示例，包括您添加的新参数：

crc32
custom
input:  not reflected
result: not reflected
polynomial: 0x04C11DB7
initial value: 0xc704dd7b
final xor value: 0x0

0xc704dd7b的初始值是使用ff ff ff ff的数据模式生成的crc，初始值= 0，最终xor值= 0。与在数据前加前缀ff ff ff相同ff。

【讨论】：

使用 link 我将多项式更改为 0x6485409B 并没有运气。然后我在文件上手动添加了一条记录，它说无效的 CRC

@marcoscaramalho - 我更新了我的答案。您可以尝试其他示例来验证我是否使用了正确的值。

您好，我可以确认它有效，谢谢！！但如果您愿意回答，我有一个问题。我没有按照最后一步计算最终的异或值，你到底是用什么异或得到的？提前致谢。

谢谢@rcgldr 我现在明白了，我写了一个小程序来为我工作，因为我注意到对于应用程序中的其他文件，它们使用不同的最终异或值。这个应用程序很大，我需要一些方法来自动化这个过程。再次感谢您的帮助，我学到了很多。

@marcoscaramalho - 我进行了蛮力搜索以找到初始值 0xc704dd7b，最终异或值 = 0x0，它应该适用于任何数据大小。这应该适用于任何数据大小。我更新了我的答案。