Back to the UI testing framework

So, we started working on the UI testing framework long long ago. But we stopped since the team was gone. But once (almost) everyone had returned, we stormed unto the UI testing automation system with all the resources and efforts we had, reconsidering each step until we saw a big smile on all the developer’s faces.

This post is about what we learned once we got back on track, and whoever has the time, it’s interesting to see what had changed since then.

A short reminder: The goal is to create a reliable, sustainable, and clear testing framework that can also replace the APP documentation. Code you can present in-front un-alfa-bits (None code-readers. In Hebrew it works better) and still, they will have a good understanding of the test and APP behaviors.

Clear Short and readable

• The tests need to be as short as possible, so we hid all the configuration, settings, and common mocks.
• Creating an action library with all the re-used operations. Keeping a clear and simple “IntelliSense” object hierarchy to navigate inside the action library.
• Creating selectors mapping objects with all the site selectors. Once more, Having a clear hierarchy to navigate inside the selectors.
• Creating dynamic selectors like getXBy, but using them only within the other selectors. Keeping the tests with well defined, descriptive DRY code.
• Encapsulation of all the local actions inside declarative private helper functions. Keeping the tests clear, short, and describable as stories.
• Moving the local helper functions and other side data into an encapsulated location, making the tests clear and readable.
• Write tests from the most general into edge cases.
• Used the nice “given / when / then” syntax.
• Keep the mock data as thin as possible, returning only stuff we needed for most of the tests. If some test needed something specific, they can extend the default mock with the relevant information.

Different types of tests:

• E2E: Without any mock, starting with the sign-in page and doing a full flow. Testing usually just some combinations of the flow, or sensitive/critical scenarios.
• Integration test: See how the SPA components work together without depending on external resources, like the server.
• Component test: Check and document all the options and possibilities inside a section of the SPA.
• Unit test: As it sounds. test the smallest public peace of code, without any external interactions.
• visual regression tools: Using a visual comparison tool like Applitools. Seeing that the SPA layout looks as it supposed.

No side dependencies

Besides keeping the “Different type of test” mentioned above:

• The test starts on the exact page, with no pre-navigations. Unless it’s part of the test.
• Return mocks that already contain all the information you need for the test.
• Checked the element on the closest testable place. Preventing the influence of other parts in between:
  • Check form data by the data it creates. Not by screen visual cues.
  • Check the visual layout by static data, not by filling forms.
  • If the two above bind. Have one more type of test to verify the binding between the two.

No redundancy

• Selectors: we don’t use selectors inside the code, but use items from the “selectors objects”. Some of the selectors are just CSS selectors, and some have more complicated methods.
• Common actions: If a set of steps is done more than once, we move it as a method inside the actions folder.
• Tests overlapping: This is harder to measure, but as written on the “no side dependencies”, we try to define exactly what we are testing and do only that.
• Keep all the mock data as thin as possible. Organize it inside a well-defined location: Mocks in the mock folder and Stubs on their messy place.

Stable and smooth

• No flickery tests: Yup, flickery tests are not an inevitable fate. It can be avoided and must be.
• Easy to execute and review: There is enough effort in writing and maintaining the test. So it’s worth doing everything possible to make the execution and interpretation as simple as possible.
• Search the available existing tools: Most of your wishlist has already been encountered by someone else. So look for existing tools. There are some amazing toys out there.

Collaborate

• Start with one developer and escalate until everyone steps on the other toes.
• Refactor.
• Some more refactoring.
• And then refactor even more.

The nice part about tests is that you don’t need to worry about breaking them without noticing. So refactor, and then refactor some more.

Example of a test suite:

If you can read it without JS knowledge, then we won 🙂

describe('Test table visualisation from settings form', () => {
  const testLifecycle: TestLifecycle = new TestLifecycle();
  const { visualizationIcons } = selectors.report;
  const settingsDropdowns = selectors.report.visualizationSettings.dropdowns;
  const htmlSelect = action.common.input.HTMLSelectBox;

  beforeAll(async () => {
    await testLifecycle.requestInterception.setRequestListeners([
      mocks.reportPreview(preview),
    ]);
  });

  beforeEach(async () => {
    // given
    await action.report.setBuilderTest(ReportType.SQL);
    await _displayTableVisualization();
  });

  test('Check font size: expect to be 16px', async () => {
    // given
    const fontSize = '16px';
    await argusBrowser.click(visualizationIcons.settings);

    // when
    await htmlSelect(settingsDropdowns.fontSize, fontSize);
    const lastRequest = await action.report.save(testLifecycle, 'Delete Me');

    // then
    expect(lastRequest.vizConf.table.fontSize).toBe(fontSize);
  });

  test('Check decimals: expect to be 3', async () => {
    // given
    const decimals = '3';
    await argusBrowser.click(visualizationIcons.settings);

    // when
    await htmlSelect(settingsDropdowns.decimals, decimals);
    const lastRequest = await action.report.save(testLifecycle, 'Delete Me');

    // then
    expect(lastRequest.vizConf.table.decimals).toBe(decimals);
  });

  test('Check number format: expect to be eu', async () => {
    // given
    const format = 'de-DE';
    await argusBrowser.click(visualizationIcons.settings);

    // when
    await htmlSelect(settingsDropdowns.formatting, format);
    const lastRequest = await action.report.save(testLifecycle, 'Delete Me');

    // then
    expect(lastRequest.vizConf.table.separator).toBe(format);
  });


  async function _displayTableVisualization() {
    await action.report.compute(testLifecycle.requestInterception);
  }
});