DCB Admin Automated Testing

Overview of testing strategy

The initial aims of the DCB Admin Automated Testing Strategy are as follows:

• To write basic tests to detect unexpected changes or behaviour in DCB Admin, so that the core user experience can be protected.

• To test basic actions within DCB Admin, with a view to testing more complex ones in future work.

• To make the DCB Admin codebase more resilient, providing ‘guardrails’ that complement the existing code review process and provide additional protection against unintentional, unexpected or incorrect changes to DCB Admin functionality.

• To prevent regression where possible, and to provide a method of detecting instances of regression when they do occur, so that they can be promptly fixed.

  • Regression in this context is defined as a feature or other part of the DCB Admin UX that breaks or experiences previously unseen unexpected behaviour after a change has been made.

  • Example 1: Changes to the page titles within DCB Admin are pushed, but contain unintentional mistakes. Cypress tests detect this and flag the unexpected behaviour before it even reaches the user.

  • Example 2: Changes to the DataGrids within DCB Admin are pushed and unintentionally break the ability to search. Cypress tests detect this unexpected action, and alert us to the regression so that it can be fixed.

What we are testing presently

• The correct rendering of the core DCB Admin pages: agencies, patron requests, groups, hostlmss, sourceBibs and locations - do these pages load as they should, with the data and UI elements we expect?

• The search functionality on the aforementioned DCB Admin pages: can a user find what they’re looking for, and does searching return the expected results?

  • This principle should be extended to filtering and sorting. It is somewhat more complicated on pages that use server-side searching, sorting and filtering, but it is still possible to test.

• The creation of new groups, and whether new groups are displayed (and displayed with the correct information) on the groups page after their creation.

  • This is an example of an action test: there are many other actions that could be tested within DCB Admin, such as uploading mappings and adding agencies to groups.

  • Action tests should have two main phases:

    • Did the action complete successfully (i.e. were the appropriate requests sent / responses received)?

    • Was the result correctly shown to the user? (i.e. if a new group is created, does it show up in the grid?)

• Login and logout within DCB Admin, whether core UI elements render as they should, and whether they change and render correctly when a user is logged in / logged out.

What we may seek to test in the future

• Other page functionality, such as sorting and filtering - particularly when this shifts to being done on the server-side for other pages.

• The home and mappings pages, in a similar vein to how the other pages have been tested.

• The process of uploading mappings, and whether the page updates correctly once uploaded or displays the appropriate error messages.

• The Details page and its variants, and their corresponding actions (closing accordions etc).

• The 404 and unauthorised pages.

• The content of the header, footer and sidebar (particularly release information).

• Other functionality as it is added, to prevent future regression and improve reliability.

Other future considerations

• Ideally, these tests should be run every time frontend changes are pushed to DCB-Admin. This can be done locally for now, but would work particularly well in Continuous Integration, as there would be no need to rely on developers remembering to run the tests before pushing.

  • Cypress Cloud is a potential solution to this problem.

• While code coverage is a useful tool, a manual look at the most common and most important user paths through DCB Admin would be equally useful in determining where to focus our testing efforts.

• New DCB Admin work should have accompanying tests written / updated for it going forward to take full advantage of our new approach.

• We should consider writing Jest unit tests for greater code coverage and to build resilience (as previously discussed). When we do this, we should adopt the strategy of manual mocking used in ERM/Folio, to help us insulate against API or component changes in future.

• Higher-level actions in Cypress (such as login, etc) should be refactored either into commands or interactors (possibly using these, which are also compatible with Jest integration tests) to promote reusability and consistency throughout our testing approach.

  • We can use before() and after() in our e2e.js file to set up / tear down after tests.

How we are handling network requests and responses

At present, we have decided to mock our network responses, rather than using genuine server responses. This approach was chosen for the following reasons:

• No need to make any code changes, or seed data.

• No stress placed on our DCB dev server.

• Control over response bodies, status and headers allows us to test a wider range of scenarios in future.

• Fast response times, with the ability to add delay to simulate bad network conditions.

• Suited to JSON APIs, as responses can be easily stubbed with fixtures.

However, there are also some trade-offs:

• No test coverage on server endpoints.

• No cast-iron guarantee that the stubbed responses 100% match the server responses.

As such, we may want to introduce tests using genuine server responses in the future, but use them sparingly and only around critical paths, as recommended in the official docs.

An example pathway of a test using this approach:

• Test navigates to a certain point in DCB Admin at which a request is sent (i.e. navigating to a new page, or creating a new group).

• The request is sent as it normally would be (if it isn’t, the test will fail).

• Our tests use cy.intercept() to intercept the request at the network layer, before it can reach DCB Admin.

• Our stubbed response kicks in (usually from a specified fixture) to provide the response.

• Assuming the response is as expected, the test continues.

Why Cypress was chosen for this work

With Cypress, we can write end-to-end (E2E) tests that navigate the application just as a user would. This is extremely helpful when testing the overall user experience, as opposed to component testing, which has a much narrower focus. E2E integration tests allow us to look at the broader picture of how users may use our apps, enabling us to test actions, paths and journeys within DCB Admin.

Cypress has numerous useful features for testing. It takes snapshots as our tests run, meaning we can identify exactly what the user is seeing at any point. This means that when tests do fail, we can see exactly what happened, which is very useful when trying to fix it. Debugging Cypress tests is also very straightforward, and can be done through common tools such as Chrome DevTools - this also aids in speeding up our investigations.

Finally, Cypress gives us extremely powerful network traffic control tools. These empower us to stub requests and responses, and simulate application behaviour in a range of network conditions, all without having to send actual requests to the server. All of these features are also available through Continuous Integration, meaning that while we may start with a local Cypress setup, there is the potential to run Cypress tests through Gitlab and unlock a range of benefits - not least spotting potentially UX-breaking code before it even hits code review.

Configuration / setup within DCB Admin

Cypress is currently configured locally within DCB Admin. This means that you may have to run npm install when you attempt to run the Cypress tests for the first time. The configuration is as follows:

• A /cypress directory has been created. It contains the following sub-directories.

  • /downloads - for files downloaded during a test. Not yet used.

  • /e2e - this is where all of our E2E integration tests go. All tests follow the format exampleTest.cy.ts.

  • /fixtures - for Cypress fixtures.

  • /screenshots - Cypress automatically saves screenshots it takes from the tests to this location. These are not to be committed to Git, but are sometimes useful for evidencing test failures or diagnosing an issue. Add this directory to your .gitignore file.

  • /support - contains configuration files for Cypress such as e2e.ts, which manages config for E2E tests, and commands.ts, which is where Cypress custom commands are registered.

    • An example of a custom command is cy.login() - as we generally have to login at the start of most tests, refactoring the necessary code into a command makes sense.

  • /utils - holds any required utility functions.

• Also located within this directory is the Cypress tsconfig.json file, which regulates the behaviour of the TypeScript compiler when dealing with Cypress files.

• At the root of the project, cypress.config.ts manages global Cypress configuration settings.

• Before doing anything with Cypress, you will need to create a cypress.env.json file at the root of the project (same level as package.json) and add the NEXTAUTH_SECRET from .env like so:

  {
      "NEXTAUTH_SECRET": "jkdhfkjsahfdjkhsdkjf73"
  }       

  This provides the NextAuth secret to Cypress, which enables it to create the NextAuth cookies necessary for simulating login. Any future Cypress environment variables can go in this file, and the former CYPRESS_USER and CYPRESS_PW variables can be removed from .env, as DCB-930 has changed the method for simulating login.

Preparing and writing tests

Refer to https://docs.cypress.io/guides/end-to-end-testing/writing-your-first-end-to-end-test for a useful in-depth tutorial (but remember we use TypeScript)- below is a quick summary.

• Create your test file. The filename should be yourName.cy.ts.

• All Cypress tests start with a describe statement, like so: describe('My First DCB Admin Test', () => {}

  • This should be a concise test name. You will outline your test cases within this describe block.

• Outline your individual test cases like so: it('should add this example', () => {}.

  • This should be a clear description of the action or behaviour you are testing - i.e. what should happen, or what should be possible. See the existing tests and the ERM tests for examples.

• We typically prefer to stub responses instead of using genuine server responses in most of our tests. There is a good official discussion and explanation here , and there is a case that certain critical pathways should test genuine server responses: however, at these early stages we are using stubs for simplicity and because they have no effect on the actual server.

• Cypress has excellent documentation - make use of it when writing your tests. Many common questions can be answered by reading it.

  • For questions around best practice, see here. Cypress docs also include many highly useful examples.

Running tests locally

To run the Cypress tests locally, you have two options. You should first ensure that DCB Admin is running.

• npx cypress open will open the Cypress application, assuming you have it installed on your machine. This will allow you to choose which tests you would like to run through the Cypress GUI.

• You can also use

  npx cypress run --spec "cypress/e2e/example-test.cy.ts"

  to specify either an individual test, or a set of tests contained within a certain directory.

  • Note that this will run headlessly (in the command line) by default. To force it to run through the GUI, you must add the --headed flag to your command.

  • To run all tests within a certain directory, use the following:

    • npx cypress run --spec "cypress/e2e"

For more information on running Cypress through the command line, please refer to the documentation.

Running coverage reports

Our Cypress setup has code coverage reporting configured automatically through the cypress/code-coverage plugin. This will generate a code coverage report and save it to the coverage directory. However, if you want to see a summary of the code coverage after the tests have been run, run the following command.

npx nyc report --reporter=text-summary

For a more detailed overview, reference the documentation here.

Examples

• Refer to groups.cy.ts for an example of a test for a page with searching and an additional action (the ‘new group’ modal).

• agencies.cy.ts is a useful template test with lots of informative comments for every testing step.

• patronRequests.cy.ts is a good example of a test for a ‘standard’ page - i.e. one with no additional actions.

Real-World Examples

These examples are useful for understanding the wider context of Cypress testing.

• The Cypress real-world app: this is the definitive official Cypress example application.

• This example of Cypress with Next.js - while focused on component testing, it also has several useful E2E examples.

Debugging

Cypress' documentation has an excellent guide to debugging that can be found here. The standard Developer Tools for your browser can be used in operation with Cypress, making debugging considerably easier.

Keeping the tests up-to-date

To keep our tests updated, a few key principles should be followed.

• When a general set of test data is defined, it should be added to the relevant fixtures - these presently use data taken from dcb-dev.

• Before pushing anything to code review, run the relevant tests.

• When a page covered by tests is changed, the relevant tests should always be run and, if necessary, updated to test any new UI elements or functionality.

• We should continually strive for greater test coverage. As such, we should ensure that all new features or pages have corresponding Cypress tests - this will ensure our code coverage isn’t reduced when new features are added.