接口测试过程中如何快速进行 diff 测试？

Posted 2021-04-24 发现bug

一、背景

接口测试过程中，当接口版本发生变更时，如何快速进行 diff 测试？本文使用Node.js、Python、Java三种语言来讲解如何快速的进行接口diff测试.

二、原理

通过对接口返回结果添加断言来判断不同版本接口返回值是否相同，结合测试框架动态生成测试用例（测试数据参数化）来批量对比接口返回值。

三、方法

3.1 Node.js

测试过程中常用命令：

# 创建node.js项目
npm init

# 安装依赖包
npm install request mocha chai mochawesome --save

# 执行测试用例，并指定报告为mochawesome，报告默认存放到当前项目 mochawesome-report/ 目录中
mocha test\batchdiff.test.js --reporter mochawesome

3.2 Python
断言使用unittest的assertDictEqual方法.

测试过程中常用命令：

# 安装依赖包

pip install requests pytest pytest-html
# 执行测试用例，并生成html报告，报告指定存放到当前项目 static/ 目录中

pytest test\test_batchdiff.py --html=static\jsondiff.html

3.3 Java
断言使用junit的Assert.assertEquals方法.

本地测试项目为maven项目，maven依赖包如下：

<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.11</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>6.14.3</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>com.alibaba</groupId>
  <artifactId>fastjson</artifactId>
  <version>1.2.49</version>
</dependency>
<dependency>
  <groupId>com.github.kevinsawicki</groupId>
  <artifactId>http-request</artifactId>
  <version>6.0</version>
</dependency>
<dependency>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-report-plugin</artifactId>
  <version>2.22.0</version>
</dependency>

maven命令：

# 运行测试用例
mvn test

# 运行测试用例，生成html报告，报告默认存放到项目的 target/site 目录中
mvn  surefire-report:report 

四、代码示例

4.1 Node.js diff测试示例

源代码

var assert = require('chai').assert;
var request = require('request');

describe("Node.js版本批量接口diff", function () {
  this.timeout(0);
  [
    // diff apis
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/demo",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/demo"
    },
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/hello",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/hello"
    },
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/user",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/user"
    },
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/1/list",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/1/list"
    },
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/2/list",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/2/list"
    },
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/3/list",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/3/list"
    },
  ].forEach(function (kv) {
    it(kv.newapi + " vs " + kv.oldapi, function (done) {
      request.get(kv.newapi, {
          json: true
        },
        function (err, res, body) {
          request.get(kv.oldapi, {
              json: true
            },
            function (oldapiError, oldapiRes, oldapiBody) {
              assert.deepEqual(body, oldapiBody);
              done();
            }
          );
        }
      );
    });
  });});

4.2 Python diff测试示例

源代码

# -*- coding:utf-8 -*-
import unittest
import requests
from parameterized import parameterized, param

diffApis = [
    # diff apis
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/demo",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/demo"
    },
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/hello",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/hello"
    },
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/user",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/user"
    },
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/1/list",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/1/list"
    },
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/2/list",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/2/list"
    },
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/3/list",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/3/list"
    },]

class TestParameterized(unittest.TestCase):
    @parameterized.expand(
        [param("diff test",kv) for kv in diffApis])
    def test_batchdiff(self, _, kv):
        self.maxDiff = None
        oldRes = requests.get(kv["oldapi"])
        newRes = requests.get(kv["newapi"])
        self.assertDictEqual(oldRes.json(), newRes.json())

if __name__ == '__main__':
    unittest.main(verbosity=2)

4.3 Java diff测试示例

源代码

package com.test.apitest;

import com.alibaba.fastjson.JSON;
import org.junit.Assert;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import com.github.kevinsawicki.http.HttpRequest;

public class BatchDiffTest {
    @DataProvider
    public Object[][] data() {
        Object[][] diffApis = new Object[][]{
         {
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/demo",
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/demo"
         },
         {
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/hello",
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/hello"
         },
         {
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/user",
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/user"
         },
         {
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/1/list",
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/1/list"
         },
         {
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/2/list",
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/2/list"
         },
         {
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/3/list",
           "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/3/list"
         },
        };
        return diffApis;
    }

    @Test(dataProvider = "data")
    public void test_apiDiff(String oldApiUrl, String newApiUrl) {
        String oldJsonString = HttpRequest.get(oldApiUrl)
                .accept("application/json")
                .body();
        String newJsonString = HttpRequest.get(newApiUrl)
                .accept("application/json")
                .body();
        Object oldJsonVo = JSON.parseObject(oldJsonString);
        Object newJsonVo = JSON.parseObject(newJsonString);
        Assert.assertEquals(oldJsonVo, newJsonVo);
    }}

五、运行结果

六、Node.js、Python、Java对比效果分析

1. 命令行日志显示的对比结果

   当接口返回结果不一致时：
   1) mocha+chai断言在日志中显示效果最好，日志输出json字段级别不一致的地方；
   2) python+unittest断言在日志中显示的是python dict对象，不太友好；
   3) java+junit仅在日志中显示两个对比的对象不一致，需要借助IDE对比返回值字段级别不一致的地方；

2. html报告

   与命令行输出日志相似，当接口返回值不一致时：
   1) Node.js体系的mochawesome报告最好看，最直观，可以看到json字段级别不一致的地方；
   2) Python体系的pytest-html报告diff测试结果不友好，不能明显看出json字段级别不一致的地方；
   3) Java体系的Surefire Report不能看出json不一致的地方；

3. 中文支持

   Node.js、Java语言输出日志、html报告中中文字符正常显示为中文字符，python输出日志、html报告中中文字符显示为中文字符对应的Unicode编码。