You have implemented the testing pyramid! In the SDLC the development phase is followed by testing and after testing comes deployment. As a senior engineer responsible for a product or service you'll want developers to run those tests every time they make changes and definitely run them before deploying those changes. How could you be sure that happened?
You could ask nicely. It might be a better option to systematize the running of tests so it becomes impossible to deploy without the tests being run on your codebase. Then you can deploy with greater confidence. In this session we are going to look at an industry wide practice of building integrated deployment pipelines.
A continuous integration (CI) pipeline automates the process of deploying software. Often the pipeline connects to version control, and will automate a deployment when code is pushed to the master branch in version control. That deployment process usually involves a number of stages, for example, building the app, testing it, updating version numbers etc and finally deploying it.
In this diagram of a pipeline you can see 3 GitHub repos. Look for a mobile app, and a backend service. All 3 repos when they are pushed to will trigger a series of automated stages including the "System_Tests" stage. If the tests pass, then the code is deployed into a production environment.
You should have a project from our last session with each type of test in it. Our mission is to create a pipeline that will connect to our github, pull the new code from the repo, run the tests and deploy it to Heroku.
If your project is not in version control put it into version control.
Have your project set up in such a way that you can run the unit tests with a single command. For example:
npm run test:unit
Do the same for the integration tests and system (end-2-end) tests.
There are lots of different CI/CD pipeline tools to choose from. What follows are instructions to set up GOCD, other providers will follow similar steps, but you will have to read the documentation for those tools to figure out how to replicate these steps.
We are going to set up a 'New Pipeline' connect that pipeline to our GitHub repo, then add a couple of stages to our pipeline. Stage one will run all the tests, stage two will release our code to Heroku.
Click new pipeline and complete the wizard.
test-unit lets start with the most simple jobOn GOCD each job on each stage is designed to run in parallel. It starts when new commits are added to the master branch of our repo. The pipeline will fetch the latest code from the GitHub repo using git pull so from here in a fresh environment we will have to run the following 2 tasks in our job:
npm installnpm run test:unitOnce this is created you should be able to trigger the pipeline when you push to the master branch on GitHub. Can you add parallel jobs for the integration and end-2-end tests?
With all our tests passing we are ready to deploy. We will use a stage to do this, and leverage the environmental variables to keep our secrets secure. We will use Heroku as a deployment destination.
Can we take a moment to consider what is going on with our pipeline.
In the diagram above you can see yourself pushing code to GitHub. GOCD is listening to your GitHub repo, when the HEAD moves forward (when you push or merge to master branch) the pipeline is triggered. GOCD server selects an available agent to run your stages on. The agent clones your repo, then runs each Stage which contains the Jobs which are made up of Tasks or commands to run. Jobs run in parallel which is why we have to start each job with npm install - there are ways to cache this step, but for now just be simple.
Usually when we deploy to Heroku it is from our computer. Now we need to deploy from a remote computer. This is part of the knowledge and skill of building and maintaining pipelines. You need remote computers to do things for you. In this instance we need a remote computer, our GOCD agent, that will be Ephemeral, to push our new changes to Heroku.
sh -c git push --force https://heroku:$HEROKU_API_KEY@git.heroku.com/multiverse-kanban.git HEAD:master
This command from our agent will push our latest repo to Heroku.
sh invokes the Unix shell interpreter-c this flag indicates that the following command should be executedgit push this is executing the push command within git (git needs to be installed on the agent)--force we have to use this because the local git working directory will be clean as all we have done is clone the repo, we've not made any changes but want to push to a remote anyway$HEROKU_API_KEYYou can generate a HEROKU_API_KEY on Heroku. Goto your Account Settings scroll down and generate or copy your API Key. Keep this safe. Goto your stage, job and task in GOCD and select the ENVIRONMENT VARIABLES tab, add a Secure Variable paste in your API key and SAVE the task. See below.
You should now have a Continuous deployment pipeline set up and working. There might be issues, please work through these with your coach. All of us will be triggering pipelines and one single agent is going to struggle cloning and running multiple npm installs and tests. You can help. You can run an agent on your computer. If you join that agent to the GOCD server then together we should have the combined computing bandwidth to support everyone's pipelines. Below are instructions to run an agent on your computer and register it as an agent for GOCD to use.
docker run -d -e AGENT_AUTO_REGISTER_KEY=$AGENT_AUTO_REGISTER_KEY -e GO_SERVER_URL=https://gocd.multiverse-coaches.io/go bmordan/gocd-agent
Your coach will share with you the $AGENT_AUTO_REGISTER_KEY you need to register your agent. When your agent is running have a look at the docker container id i.e. a2aac941631f
CONTAINER ID NAMES CREATED ago STATUS
a2aac941631f reverent_banach 4 minutes ago ago Up 4 minutes
Then find your agent on https://gocd.multiverse-coaches.io/go/agents and enable it by ticking the checkbox next to your agent and then clicking on ENABLE.
How does your team deploy the projects that you work on? Which CI/CD platform do they use. You might want to look into setting up a simple pipeline using the platform your team uses. Being familiar with the pipelines in your workplace might influence your design choices and the way you structure your features and solutions in your final workplace project.