第一次失败后开玩笑停止测试套件
Posted
技术标签:
【中文标题】第一次失败后开玩笑停止测试套件【英文标题】：Jest stop test suite after first fail 【发布时间】：2018-12-17 09:37:18 【问题描述】：我正在使用Jest 进行测试。
我想要的是在该测试套件中的测试失败时停止执行当前测试套件。
--bail option 不是我需要的，因为它会在一个测试套件失败后停止其他测试套件。
【问题讨论】：
这是一个很好的问题，我很惊讶 Jest 似乎没有针对这种情况的指导。我已经提交了ticket on their GitHub repo。 @DanDascalescu 我认为这更多是关于抛出错误的钩子。正如 OP 询问的那样，如果测试失败，我们该怎么办？ 【参考方案1】：我做了一些杂碎，但它对我有用。
stopOnFirstFailed.js:
/**
* This is a realisation of "stop on first failed" with Jest
* @type globalFailure: boolean
*/
module.exports =
globalFailure: false
;
// Injects to jasmine.Spec for checking "status === failed"
!function (OriginalSpec)
function PatchedSpec(attrs)
OriginalSpec.apply(this, arguments);
if (attrs && attrs.id)
let status = undefined;
Object.defineProperty(this.result, 'status',
get: function ()
return status;
,
set: function (newValue)
if (newValue === 'failed') module.exports.globalFailure = true;
status = newValue;
,
)
PatchedSpec.prototype = Object.create(OriginalSpec.prototype,
constructor:
value: PatchedSpec,
enumerable: false,
writable: true,
configurable: true
);
jasmine.Spec = PatchedSpec;
(jasmine.Spec);
// Injects to "test" function for disabling that tasks
test = ((testOrig) => function ()
let fn = arguments[1];
arguments[1] = () =>
return module.exports.globalFailure ? new Promise((res, rej) => rej('globalFailure is TRUE')) : fn();
;
testOrig.apply(this, arguments);
)(test);
在所有测试之前导入该文件（在第一个 test(...) 之前），例如我的 index.test.js：
require('./core/stopOnFirstFailed'); // before all tests
test(..., ()=>...);
...
当第一个错误发生时，该代码将所有接下来的测试failed 标记为globalFailure is TRUE。
如果您想排除failing，例如。您可以这样做一些清理测试：
const stopOnFirstFailed = require('../core/stopOnFirstFailed');
describe('some protected group', () =>
beforeAll(() =>
stopOnFirstFailed.globalFailure = false
);
test(..., ()=>...);
...
它从failing 中排除整个组。
使用 Node 8.9.1 和 Jest 23.6.0 测试
【讨论】：
不适用于node v10.15.3 和jest v24.9.0。我得到TypeError: Class constructor Spec cannot be invoked without 'new' 指向function PatchedSpec(attrs) 行。为什么你们的 cmets 会提到 Jasmine？
我什至没有尝试过，因为它看起来很糟糕......从整个测试套件中基本上只在 Jest 中进行 process.exit() 有那么复杂吗？！【参考方案2】：
我有连续且复杂的测试场景，如果该套件的其中一个测试失败，则没有必要继续测试套件。但是我没有设法将它们标记为已跳过，因此它们显示为已通过。
我的测试套件示例：
describe('Test scenario 1', () =>
test('that item can be created', async () =>
expect(true).toBe(false)
)
test('that item can be deleted', async () =>
...
)
...
我改成如下：
let hasTestFailed = false
const sequentialTest = (name, action) =>
test(name, async () =>
if(hasTestFailed)
console.warn(`[skipped]: $name`)
else
try
await action()
catch (error)
hasTestFailed = true
throw error
)
describe('Test scenario 1', () =>
sequentialTest('that item can be created', async () =>
expect(true).toBe(false)
)
sequentialTest('that item can be deleted', async () =>
...
)
如果第一个测试失败，下一个测试将不会运行，但它们将获得通过状态。
报告将如下所示：
测试场景 1 > 可以创建该项目 - 失败 测试场景 1 > 可以删除该项目 - 通过这并不理想，但在我的情况下可以接受，因为我只想在报告中看到失败的测试。
【讨论】：
【参考方案3】：感谢this comment on github，我能够使用自定义testEnvironment 解决此问题。为此，jest-circus 需要通过 npm/yarn 安装。
值得注意的是jest will set jest-circus to the default runner with jest v27。
首先需要调整 jest 配置：
jest.config.js
module.exports =
rootDir: ".",
testRunner: "jest-circus/runner",
testEnvironment: "<rootDir>/NodeEnvironmentFailFast.js",
然后你需要实现一个自定义环境，上面的配置已经引用了这个环境：
NodeEnvironmentFailFast.js
const NodeEnvironment = require("jest-environment-node")
class NodeEnvironmentFailFast extends NodeEnvironment
failedDescribeMap =
registeredEventHandler = []
async setup()
await super.setup()
this.global.testEnvironment = this
registerTestEventHandler(registeredEventHandler)
this.registeredEventHandler.push(registeredEventHandler)
async executeTestEventHandlers(event, state)
for (let handler of this.registeredEventHandler)
await handler(event, state)
async handleTestEvent(event, state)
await this.executeTestEventHandlers(event, state)
switch (event.name)
case "hook_failure":
const describeBlockName = event.hook.parent.name
this.failedDescribeMap[describeBlockName] = true
// hook errors are not displayed if tests are skipped, so display them manually
console.error(`ERROR: $describeBlockName > $event.hook.type\n\n`, event.error, "\n")
break
case "test_fn_failure":
this.failedDescribeMap[event.test.parent.name] = true
break
case "test_start":
if (this.failedDescribeMap[event.test.parent.name])
event.test.mode = "skip"
break
if (super.handleTestEvent)
super.handleTestEvent(event, state)
module.exports = NodeEnvironmentFailFast
注意
我添加了 registerTestEventHandler 功能，这对于快速失败功能来说不是必需的，但我认为它非常有用，尤其是如果您之前使用过 jasmine.getEnv() 并且它可以与 async/await 一起使用！
您可以像这样在您的测试中注册自定义处理程序inside（例如beforeAll hook）：
// testEnvironment is globally available (see above NodeEnvironmentFailFast.setup)
testEnvironment.registerTestEventHandler(async (event) =>
if (event.name === "test_fn_failure")
await takeScreenshot()
)
当一个test 失败时，将跳过同一describe 中的其他test 语句。这也适用于嵌套的 describe 块，但 describe 块必须具有不同的名称。
执行以下测试：
describe("TestJest 3 ", () =>
describe("TestJest 2 ", () =>
describe("TestJest 1", () =>
beforeAll(() => expect(1).toBe(2))
test("1", () => )
test("1.1", () => )
test("1.2", () => )
)
test("2", () => expect(1).toBe(2))
test("2.1", () => )
test("2.2", () => )
)
test("3", () => )
test("3.1", () => expect(1).toBe(2))
test("3.2", () => )
)
将产生以下日志：
FAIL suites/test-jest.spec.js
TestJest 3
✓ 3
✕ 3.1 (1 ms)
○ skipped 3.2
TestJest 2
✕ 2
○ skipped 2.1
○ skipped 2.2
TestJest 1
○ skipped 1
○ skipped 1.1
○ skipped 1.2
● TestJest 3 › TestJest 2 › TestJest 1 › 1
expect(received).toBe(expected) // Object.is equality
Expected: 2
Received: 1
2 | describe("TestJest 2 ", () =>
3 | describe("TestJest 1", () =>
> 4 | beforeAll(() => expect(1).toBe(2))
| ^
5 | test("1", () => )
6 | test("1.1", () => )
7 | test("1.2", () => )
at suites/test-jest.spec.js:4:33
● TestJest 3 › TestJest 2 › TestJest 1 › 1.1
expect(received).toBe(expected) // Object.is equality
Expected: 2
Received: 1
2 | describe("TestJest 2 ", () =>
3 | describe("TestJest 1", () =>
> 4 | beforeAll(() => expect(1).toBe(2))
| ^
5 | test("1", () => )
6 | test("1.1", () => )
7 | test("1.2", () => )
at suites/test-jest.spec.js:4:33
● TestJest 3 › TestJest 2 › TestJest 1 › 1.2
expect(received).toBe(expected) // Object.is equality
Expected: 2
Received: 1
2 | describe("TestJest 2 ", () =>
3 | describe("TestJest 1", () =>
> 4 | beforeAll(() => expect(1).toBe(2))
| ^
5 | test("1", () => )
6 | test("1.1", () => )
7 | test("1.2", () => )
at suites/test-jest.spec.js:4:33
● TestJest 3 › TestJest 2 › 2
expect(received).toBe(expected) // Object.is equality
Expected: 2
Received: 1
8 | )
9 |
> 10 | test("2", () => expect(1).toBe(2))
| ^
11 | test("2.1", () => )
12 | test("2.2", () => )
13 | )
at Object.<anonymous> (suites/test-jest.spec.js:10:31)
● TestJest 3 › 3.1
expect(received).toBe(expected) // Object.is equality
Expected: 2
Received: 1
14 |
15 | test("3", () => )
> 16 | test("3.1", () => expect(1).toBe(2))
| ^
17 | test("3.2", () => )
18 | )
19 |
at Object.<anonymous> (suites/test-jest.spec.js:16:31)
Test Suites: 1 failed, 1 total
Tests: 2 failed, 6 skipped, 1 passed, 9 total
Snapshots: 0 total
Time: 0.638 s, estimated 1 s
【讨论】：
【参考方案4】：这是我的solution -- 如果有重大缺点，请告诉我，就我的目的而言，它似乎按预期工作
我只有一个***描述块，出于我的目的，我希望在一个测试失败时整个测试文件都失败
export class FailEarly
msg: string | undefined;
failed: boolean = false;
jestIt: jest.It;
constructor(jestIt: jest.It)
this.jestIt = jestIt;
test = (name: string, fn: jest.EmptyFunction, timeout?: number) =>
const failEarlyFn = async () =>
if (this.failed)
throw new Error(`failEarly: $this.msg`);
try
await fn();
catch (error)
this.msg = name;
this.failed = true;
throw error;
;
this.jestIt(name, failEarlyFn, timeout);
;
给我一个上下文（类属性）来存储 global-esq 变量
const failEarlyTestRunner = new FailEarly(global.it);
const test = failEarlyTestRunner.test;
const it = failEarlyTestRunner.test;
用我的类方法重载test 和it 函数（从而访问类属性）
describe('my stuff', () =>
it('passes', async () =>
expect(1).toStrictEqual(1);
)
test('it fails', async () =>
expect(1).toStrictEqual(2);
)
it('is skipped', async () =>
expect(1).toStrictEqual(1);
)
)
结果：
my stuff
✓ can create a sector (2 ms)
✕ it fails (2 ms)
✕ is skipped (1 ms)
● my stuff › it fails
expect(received).toStrictEqual(expected) // deep equality
Expected: 2
Received: 1
> ### | expect(1).toStrictEqual(2);
| ^
### | );
● my stuff › is skipped
failEarly: it fails
69 | const failEarlyFn = async () =>
70 | if (this.failed)
> 71 | throw new Error(`failEarly: $this.msg`);
| ^
72 |
73 |
74 | try
每个跳过的测试都失败，并带有一个错误，指示上游失败的测试
正如其他人指出的那样——你必须用--runInBand 标志开玩笑
希望这对某人有所帮助-如果有有意义的缺点或更好的方法，请发表评论；我总是乐于学习
【讨论】：
【参考方案5】：破解 global.jasmine.currentEnv_.fail 对我有用。
describe('Name of the group', () =>
beforeAll(() =>
global.__CASE_FAILED__= false
global.jasmine.currentEnv_.fail = new Proxy(global.jasmine.currentEnv_.fail,
apply(target, that, args)
global.__CASE__FAILED__ = true
// you also can record the failed info...
target.apply(that, args)
)
)
afterAll(async () =>
if(global.__CASE_FAILED__)
console.log("there are some case failed");
// TODO ...
)
it("should xxxx", async () =>
// TODO ...
expect(false).toBe(true)
)
);
【讨论】：
以上是关于第一次失败后开玩笑停止测试套件的主要内容，如果未能解决你的问题，请参考以下文章
Xcode UI 测试 - 当测试套件中的任何给定测试失败时停止测试？