Today we look at the Testing stage of the SDLC.
The input to the Testing stage is untested software.
The purpose of testing software is to ensure the quality of the system is as high as possible by removing:
Think about how important testing is for self-driving cars!
Tests are based on 2 key principles:
The earlier in the SDLC we test, the sooner we find issues and avoid shipping flawed software.
The "V" model (also known as the Verification and Validation model) associates a testing phase to each SDLC stage.
Static testing focusses on the early stages of the SDLC, before code is written. It aims to remove errors which could lead to defects in software code. Examples of static testing include:
Static testing is done via reviews and walkthroughs.
Dynamic testing focusses on testing the code as it is executing. It aims to detect defects. Examples include:
We can further breakdown dynamic testing into:
Black-box testing - here we treat the system as a black box and only concern ourselves with the inputs and output e.g. system testing, integration testing, user acceptance testing
White-box testing - here we are aware of the inner workings of our code e.g. unit tests
| Black Box | White Box |
|---|---|
|
|
| Black box testing asks WHAT inputs produce WHAT outputs | White box testing asks HOW the internal logic of the system works |
Experience-based testing - key stakeholders identify areas of the system that, in their experience, are likely to be yield defects e.g. areas of high complexity, areas which previously caused issues.
The International Software Testing Qualifications Board describes the following 7 principles of testing.
This video explains each one in detail.
Dynamic testing is concerned with 4 test types:
Unit Tests isolate a section of code and verify its correctness. A unit may be an individual function, method, procedure, module, or object.
Unit tests are run by developers
Integration tests determine if independently developed units of software work correctly when they are connected to each other. For example calling this endpoint /users/x023/photos should return a JSON string with an array of urls. This is testing that the server correctly interacts with the database and returns the correct values.
Integration tests are run by developers or testers
Examples of system testing are end-to-end tests such as Cypress tests) with assertions of state or outputs. Other examples of system tests include non-functional tests such as load/performance tests, security penetration tests.
System tests are run by testers
The purpose of acceptance tests are to evaluate the system’s compliance with the business requirements and assess whether it is acceptable for delivery. You built a perfect bridge, but did you meet the business requirements?! The Use Cases and User Stories developed during the Requirements Analysis stage help define user acceptance tests.
Acceptance tests are run by the end-users, supported by business analysts
During each stage of dynamic testing we must ensure that the system meets the non-functional requirements defined in the Requirements Analysis stage of the SDLC. Non-functional testing includes:
This video explains how a load testing tool works.
One of the seven principles of testing is exhaustive testing is impossible. So how should we go about testing our applications? The answer is to use equivalence classes. Equivalence partioning divides the input data to a software unit into partitions of equivalent data from which test cases can be derived.
Imagine we have an input field that is designed to take the number of tickets you can book for a concert. One individual can buy up to 10 tickets. We want to test this. For example
What emerges are four classes or partions of tests that are created by the different ranges of valid and invalid possible inputs.
We only need one test in each class because, testing that 6 is OK would be equivalent to testing 7 is OK. They are both valid amounts, the result of the test would be the same or equivalent. There is no point in testing in this class more than once because the results are equal or equivalent. Same for the out of range values, we tested with 11, but would expect the same results for 50, 100, 1000 etc
Boundary testing is the process of testing the extreme ends or boundaries of input values.
For example, imagine we are writing an application which alerts individuals between 28 and 40 to come for their COVID vaccination. Both 28 and 40 will be considered as boundary values.
The assumption of boundary value analysis is that boundary values are most likely to cause an error.
Regression testing ensures that unchanged areas of the system have not been affected by new changes. Running the regression test suite is just as important as creating and running new tests!
A test plan typically includes:
The output of the Testing phase is fully tested software.
Look for job vacancies for software testers - what kind of skills do they require?
What are the advantages and disadvantages of outsourcing testing to an external organisation?
Update your portfolio to evidence different kinds of tests e.g. unit, system, performance
Example job spec for a tester at Tesla
Fireside Chat: DevOps at Amazon with Ken Exner, GM of AWS Developer Tools - AWS Online Tech Talks