End-to-End Testing

Learning Objectives

What are end-to-end tests?

End-to-end tests are at the opposite end of the spectrum to unit tests. Unit tests take the smallest piece of an application (usually functions) and ensure it works as expected. End-to-end tests on the other hand test the application in its entirety, ensuring all its components (UI, database, server etc.) work together. They essentially mimic an actual user interacting with your application. In the middle we have integration tests, which test sub-systems within your application.

Testing pyramid: unit tests make up the base of the pyramid; integration tests in the middle; end-to-end tests at the top

Why are end-to-end tests beneficial?

Application codebases can be very large and complicated. Even if we are confident that are code is suitably modular, it's always possible that some change we make is going to have some unintended consequence we can't foresee. Thus, before we release any changes to our users, we want to ensure that they won't break any of the features our application offers.

One way to do this is to make a big list of all your application's features and, before reach release, have an employee manually test each one of these features to ensure they all work. This takes many hours and is extremely tedious.

A better way is, rather than maintaining a list of features, maintain a list of end-to-end tests which test those features. A good end-to-end test should step through a process your customers will want to use (e.g. logging in) and make sure everything works as designed.

The advantage of end-to-end tests is therefore that

What makes end-to-end testing challenging?

The downside of end-to-end tests is that they are typically much harder to create and run than unit tests.

Firstly, end-to-end tests, by their very nature, involve all the components of the app. Unlike a unit test - where a single function is loaded into a file - end-to-end tests involve creating a fully-functioning application. Modern applications can be very complicated and so setting up the environment to run your end-to-end tests can also be complicated. End-to-end also often involve seeding a database and then rolling-back changes after each test (to ensure test-independence).

Secondly, end-to-end are much more computationally expensive than unit tests. Because the entire app needs to be fired up, the tests require more computational power and take longer to run.

Thirdly, end-to-end tests can be harder to debug. When a unit test fails, it's usually because an error was thrown or because a value didn't equal the expectation, and the developer need only consider a small section of code. On the other hand, any part of the application failing can cause an end-to-end test to fail, so the faulty code has many more places to hide.

Assignment

In groups, review the testing pyramid. Research and discuss the differences between unit, integration and end-to-end tests. What are the advantages and disadvantages of each?