This tutorial describes the CI/CD procedure from the user/developer point of view.
The tutorial contains 5 sections:
1. Storing your Code in Git
2. Working with Different Branches
3. Creating Docker Images for your Services and Storing them in Container Registry
4. Deploying your Application as a Set of Services
5. Controlling Versions of your Application
A diagram illustrating the process can be found here
1. Storing your Code in Git.
Say, for example, that there is an application that consists of certain services. These services are written in different languages and they communicate with each other. Each service has its own repository and is independent from the other service in terms of its code. But the application requires all of them to be up and running in order to provide full functionality.
- Structure:
Generic repo -- service1 (NodeJS)
service2 (Python)
service3 (Go)
In this case, we are gathering some set of the services (where every service has its own git repository) under a single repository called Generic Repo. We are using git “Submodule” functionality. Once the submodule is added to Generic repo with git submodule add $service_url , it has its latest “main” branch commit checked out in Generic repo. This is the basic structure of the projects.
2. Working with Different Branches
Each service is continuously updated by one or more developers. For this reason, it is beneficial to implement branches separation. At this point, in order to describe the approach, it is advisable to introduce the idea of having three environments for our application: dev, stage and prod. We deploy to these envs differently.
2.1. main (master) branch: This branch is used to build images for testing and passing QA on stage environment, and then deployed on prod. Developers shlould strictly not be working on this branch. The best practice would be to merge DEVELOP branch into MAIN(MASTER) branch.
2.2. develop: This branch is used to trigger deployment of your application into dev environment. Developers create “merge requests” to this branch.
2.3. issue-#: This is the branch that developers are working on. The name of the branch should correlate with the issue number in GitLab.
3. Creating Docker Images for your Services and Storing them in Container Registry.
As part of the CI/CD process, docker images are created. A docker image represents the code with its functionality that is ready to be deployed on one or more of the three environments.
Once “merge requests” from issue-# to DEVELOP branch is approved and merged, it triggers the CI/CD process. In the service repository > CI/CD > Pipelines, you can check the status of the pipeline. It contains logs of thedocker buildand docker push commands. Merging to develop ALWAYS builds an image with the tag “latest”. The second stage of CI/CD is deploying this image to DEV environment so that developers can see their changes on kubernetes cluster.
After developers have checked the developer environment, they create a merge request from DEVELOP to MAIN branch. Once the merge request is approved and merged, the following actions are triggered:
- MAIN branch has new commit
- CI/CD process is triggered
- new commit = docker image tag
- The image with the new tag is pushed to container registry.
However, in contrast with merging to “DEVELOP” branch, merge to MAIN does not deploy the image to dev/stage/prod. It just builds and pushes the image to container registry.
4. Deploying your application as a set of services.
Once the code has been checked out in dev environment, then merged to MAIN branch, and you have received your image with the new tag, you can proceed to deploy stage/prod envs. As I mentioned before, the application represents the set of services. So each service has its own latest commit in MAIN branch, its own Docker Image, and it’s unique URL that is added as submodule to the Generic Repo.
In order to deploy the set of services, developers perform the following actions:
4.1. Clone generic repo recursively with git clone --recursive $generic_repo_url
4.2. Go to Generic repo/Service
4.3. Execute git checkout main
4.4. Go back to Generic Repo
4.5. Commit and push
4.6. Create a tag in generic repo with version number.
This sets the latest MAIN branch commit of the service in Generic Repo. Remember, that our service image has the tag that equals to latest MAIN branch commit and we are passing this commit as a service version to our generic repo. So the result would look like:
Generic repo -- service1:1234qwer (NodeJS) (docker image - cr.io/service1:1234qwer)
(TAG = 1.0) service2:2345wert (Python) (docker image - cr.io/service2:2345wert)
service3:3456erty (Go) (docker image - cr.io/service3:3456erty)
Once the process is completed, go to Gerenic Repo > CI/CD > Pipelines, and press the deploy stage/prod button.
5. Controlling versions of your application.
As we see from the description above,the set of services with different commits is gathered under a tag that equals to 1.0. So, if you would like to make an update in a single service, for example, follow these procedures:
5.1. Create an issue on gitlab
5.2. Create a branch issue-# in the service that you want to update (issue-# – should be set according to the issue number in gitlab)
5.3. After the code is fixed, merge issue-# to DEVELOP branch (this will trigger deployment to dev environment)
5.4. Check if the updated code is working on dev environment and if so, merge DEVELOP to MAIN
5.5. Go to Generic Repo/Service, execute git checkout main && git pull
5.6. Go to Generic Repo, commit and push the changes
5.7. Tag the Generic Repo with the new version (e.g. – 1.1, 2.0, etc.)
Say, for example, we were fixing NodeJS code.
In this instance:
Generic repo -- service1:1234qwer (NodeJS)
(TAG = 1.0) service2:2345wert (Python)
service3:3456erty (Go)
will be replaced with this:
Generic repo -- service1:6789yuio (NodeJS)
(TAG = 1.1) service2:2345wert (Python)
service3:3456erty (Go)
And after creating a new tag, the new version can be deployed on stage/prod in Gerenic Repo > CI/CD > Pipelines.