Adapt BDD in scrum team

Posted 2021-04-25 胡予妈

Motivation

Hybris commerce is acquired by SAP since 2013. My team is working on SAP commerce On-Premise version since 2015, and we deliver new features quarterly using SCRUM as process framework for agile development. Like all others SCRUM team, our PO defines the requirement with priority, developers pick tasks including UT/IT/E2E, Arch helps design and technical problems, Release Master helps CI (continuous integration), and I as QE work starting from scratch of development to ensure quality in whole SDLC. (shift left testing).

As you see, we have UT which have >60% code coverage, have IT which just cover DAO layer (data access object), also have E2E which covers more in detail even for bugs. However, most of time, we still run regression test manually. Why? firstly, most of UT test nothing, secondly E2E runs for a long long time, and so on.

Now, we are transiting to cloud and rebuilding a new web application, the delivery cycle is changed from quarterly to bi-weekly. it means faster time to market over perfect software. Everyone puts strong focus on automation testing in cloud. so do we. However, can we tolerate useless UT? tolerate long time execution time? NO, now the chance is coming:

• No one likes manual test bi-weekly

• No legacy code to stumble

• Coding using new technologies without fixed thoughts

• No delivery pressures

• TDD is trained just now

• All roles including manager support change to improve quality

and there is an article which touched me deeply - 没有TDD/ATDD，你的持续交付可能是持续找死.

Test-Driven Development (TDD) focuses on the “inside-out” perspective, meaning we create tests from a developer’s perspective. 
Behavior Driven Development or BDD is an extension to Test Driven Development. it focuses on the “outside-in” perspective, meaning we test behaviors which are related to business outcomes, which can be understood by everyone.
TDD point out several key advantages:

• you have very high test coverage

• you can refactor fearlessly

• your code was working a minute ago

• you reduce unnecessary code

• you think about the requirements before diving into hacking coding

• your code has self-documented acceptance/demo-tests

• you know when your code is done

Let's take this chance for better automation testing.

Step by Step 

Talking about automation, Test Pyramid is a commonplace topic. Generally (Test Pyramid is only a way of thinking about different kinds of automated tests, it depends on the test context), Test Pyramid is a model that tells you how many tests to write at each layer. Since unit tests are the fastest and cheapest , we should have a lot of them. E2E tests sit at the top of the pyramid. These tests are flaky and take a long time to run, so we should have fewer test cases implemented at this level. No doubt, automation is a big and complex topic, like splitting a big feature into small user stories, we will take improving UT as our first step of automation.

However, where do we start?

Following the beaten track - developer writes productive code firstly, then writes UT, it may still result in useless UT like On-Premise version. and image that if productive code is done, do you have passion or motivation to write test code? No. in this case, UT is just a task without any value, then code coverage means nothing. It is not what we expected.

Jumping to TDD directly? No, TDD changes the traditional behavior, it is very difficult. and if pushing TDD from top to bottom, TDD is reduced to a form, a process. Team may not gain benefits, instead, team may struggle with the process. Also when using TDD, it is inevitable to encounter many difficulties, only when team is willing to put TDD into practice, then team will try their best to tackle impediment.

First Phase - what to test & how to test

first phase - making the tasks (1 task for productive code and 1 task for UT under 1 user story) picked in parallel. In fact, it is agile methodology. ideally, all user stories, even all sub tasks should be in-depending. we aim to complete the highest priority user story together, developers can collaborate and be backup with each other.

During this phase, 2 developers working on 1 user story sit together actively in order to facilitate communication. they discuss requirement, service interface and also implementation. Due to the UT framework for Angular is Jasmine, which is a BDD behavior-driven development framework for testing javascript code. so I work on UT test cases at the beginning of development.

Let's take 1 simple feature as an example - notification preference.

Please see the UI design below:

Requirement:

The selected channels will be used to send notification like stock notification and so on.

Please see the first version for designed UT tests for this angular component based on requirement and productive code (we will see whether the cases are correct or not later):

Notice that I also go through productive code to see how to test.

1. title should be displayed (due to title is a property in angular component and bound to template)

2. notification channel cannot be shown if no response is returned, should show busy indicator.

3. notification channels can be shown with correct name, value if there are channels

4. notification channel can be checked/unchecked correctly

5. notification channel can be multiple clicked correctly

Please see the first version of UT. To be honest, I copied a part of file as an example here in our first phase.

it('should be able to show page title', () => { let title: string;  component.title$ .subscribe(value => { title = value; }) .unsubscribe(); let h3: htmlElement; h3 = fixture.nativeElement.querySelector('h3'); expect(title).toEqual('Test title'); expect(h3.textContent).toContain(title); });

Could you see the problems in this UT?  Don't think the problems by yourself, Developers have experience to write productive code and UT together, what's their feelings? Let's brainstorm. 

I organized the first brainstorm; we discuss the following outstanding questions bases on the results:

• who will determine css name due to we get element by css

• whether we should call lifecycle hooks manually

• whether we should verify every element in UI

• whether we should know the properties/private methods of component

• is there any common issues we can take consideration in first UT version?

• whether the NGRX will affect the UT

Everyone presents opinion because they have first-hand experience, they want to improve.

After heated discussion, we come to an agreement about what we should test in UT:

and also introduce a new attribute for test only:

The most important thing is that we work out our general test rule. what we should test keep changing, but rule cannot change. 

We need to make sure that the entire development efforts are directed and focused towards meeting the requirements. In order to avoid any kind of a “requirement – miss” defect, the entire development team has to align them to understand the requirement.

After the brainstorm, I have a little concern about the additional attribute for test only, I asked 1 developer what's the feeling about it. The developer thought the attribute is so good.

Please notice here, if there are some different from core team, put yourself into their shoes, why did they not do it like this, what's the reason behind it?

The arch from core team reviewed our code and commented that we should not introduce the additional attribute, I argued that additional attribute will make the UI testing more resilient to change. if there is a UI layout change, we don't need to change UI testing.

However, for this case, it is taken as a breaking change, if UT can test the breaking change for minor version, it is better. because we provide complete component implementation with UI that customer can build upon, customize styles, so our test should guard us and make sure, our changes are not breaking customer's customizations, and the style is released as lib, it's even more valid, because customers can target those css classes and changing them would mean a breaking change, so unit tests should guard us and fail when someone will change the css class of some element

so, I accepted their decision.

Second Phase - when to test

Let's start our second phase - Modify the first version of UT ONLY. (depended core productive code is changed all the time - core 1.0 version is not release yet; our productive code will be changed accordingly)

why did we only plan UT?

• Based on the first version of productive code and UT, we know how to write UT even if the productive code is out of date. It is not same with staring from UT without productive code. Developers will think it is feasible.

• Force developers to think about the requirements before diving into hacking coding

• Let developers feel UT cases stable event if the productive code is not stable

We refactor the UT to the following cases:

1. it should show channels and notes. 
   → ensure UI key elements here, which present the requirement. 
   properties binding of component is not our purpose, we don't know which properties will be designed in component, that's implementation details, not what we care about.

2. it should show spinner when loading.
   → it is also requirement for intermediate state of action

3. it should be able to disable a channel when GET loading
   → based on NGRX, we should test intermediate state - loading for every action

4. it should be able to disable a channel when UPDATE loading
   → based on NGRX, we should test intermediate state - loading for every action

5. no case for check/uncheck
   → based on NGRX, check/uncheck will only dispatch an action without return, it can be covered in case 3 and 4.

Please see the updated UT below:

 it('should show channels', () => {    fixture.detectChanges();    expect(el.query(By.css('.pref-header'))).toBeTruthy();    expect(el.query(By.css('.pref-note'))).toBeTruthy();    expect(      el.queryAll(By.css('.form-check-input')).length ===        notificationPreference.length    ).toBeTruthy();    expect(      el.queryAll(By.css('.pref-channel')).length ===        notificationPreference.length    ).toBeTruthy(); });

Compared with previous version:

1. simpler without unnecessary code

2. can see the requirement behind it

3. enhance the requirement - should disable checkbox when response is not returned

4. simulate end user behavior, every action starts from click event and so on, even for observable object.

I also asked some developers, what's their feeling for updating UT.  Feedback is positive, change is obvious.

Currently, we don't know whether it reduce the defects (not release, so no customer use), however, obviously it helps requirement in detail, UT is not useless anymore, and also improve the efficiency of writing UT. 

Third Phase - continue improving

During third phase - we continue decouple component implementation.

for example:

1. Component with a subcomponent inside, whether we should mock a test host/test child to trigger the component test

2. How to design a component, whether we should include all business logic in standalone component - it is a topic from test point of view

We follow the same way - organize a brainstorm to discuss what we should do.

All decisions come from team.

If there are other problems, we will continue discussing and improving.

Fourth Phase - enhance code coverage together with E2E

 to be continue... (my next goal)

How is QE involved 

UT is not only developer's responsibility, but also the whole team's responsibility.

When changing to new technologies, cross-function team should be real cross-function, for QE:

1. QE should learn development technology together with developers for better case design
   We can see technology - NGRX will impact test, even for UT

2. QE should learn UT deeply to drive TDD forward
   Due to developer will learn development technology deeply, less time to learn UT technology, QE should take this responsibility. Once developers meet issues in UT, there must be a person who can resolve the issues to drive TDD forward. 

3. QE should be a change agent to lead the change.

4. QE should be a coordinator to find points to organize discussion, and continue improving.

5. QE should be an interface to communicate with stakeholder to reduce risk. 

Conclusion

Leading change is not easy, should step by step. The final way may be not a best way, but it is decided by team, accepted by team, executed by team. Let's keep moving.