DevOps/ Automation

Automating Jenkins & Docker with Kubernetes

Integrating Kubernetes with Docker & Jenkins

7 min readOct 12, 2020

In the previous articles, I have demonstrated the process of integrating Jenkins and Docker together to create DevOps pipelines. If you haven't checked that, click the link below:

DevOps Automation using Jenkins

Integrating Jenkins, Docker and Github to create CI/CD pipeline

medium.com

In this article, we are going to increase the level of automation by one more level by integrating Container Orchestration Technology with the existing pipeline. Here I am going to demonstrate this by using Kubernetes.

Why Do we Need Kubernetes?

The short falls of using the previous pipelines were:

Auto Scaling: What if the clients exponentially increase and we have to scale or increase the number of container where our application is running? Manually going and launching container is not a good practice even we can have one monitoring software to look for the traffic and accordingly scale out and scale in. This will again add pain to our head. So we need one unified tool to handle this.
Load Balancing: Let’s say after scaling if our program is not so intelligent to distribute the traffic to other containers auto scaling is of no use then, hence we need a software which also look for load balancing.
Service Discovery: Let’s say we scaled out some container for our new clients and we even have load balancing for some existing containers, now how will the load balance program will come to know the IP of the newly created container? In this dynamic world even our IP is not constant, so we even need a program to have capability for service discovery.
Reverse Proxy: Let’s say we have auto scaled, we have balanced the load, we have service discovery but we cannot give too many IP address to client to connect to our application, no one does that, it’s a very bad practice. So, we need a program which takes all the IP of the containers and give us one unique IP.
Fault Tolerance: Here fault tolerance means once the container with our application is launched, we don’t want our clients to face any down time. Hence we need any system which can have a fault tolerance capability. So, this program have to continuously monitor the container running and if it fails it should automatically launch the new container.

Kubernetes, being Orchestration Service, takes care of all these altogether.

Getting Started

Objectives:

JOB#1: Pull the Github repo automatically when some developers push repo to Github.

JOB#2:

By looking at the code or program file, Jenkins should automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed )
Expose your pod so that testing team could perform the testing on the pod.
Make the data to remain persistent ( If server collects some data like logs, other user information )

JOB#3: Test your app if it is working or not.

JOB#4: If app is not working , then send email to developer with error messages and redeploy the application after code is being edited by the developer.

Assumptions:

First of all, we are assuming that Docker, Git, MiniKube and Jenkins with the Git Plugin are installed in the system.

1. JOB-1:

This is the system in which the developer works. We are considering it to be Windows. Here, Git is installed and authenticated. First, we create a new repository in GitHub. We copy the clone URL of the repository and clone it in the Desktop using git clone <repo url>.

In the github, I am having 3 files.

index.html : Which have the website code
testing.yml : Which will set up out testing environment
k8s-copy.sh : Which will copy our index.html inside the pod.

NOTE: Pod is where our container will reside i.e. pod is not equal to container, pod is a wrapper around one or more than one container which we want to live in correlation or symbiosis.

JOB-2:

Here we will check the extension of the code file and accordingly launch the deployment code inside the testing.yml file.

if sudo ls ./*.html
then
sudo kubectl create -f /k8s-jenkins/files/testing.yml

sleep 15
sudo sh /root/k8s-copy.sh
elif sudo ls ./*.py
then
sudo kubectl create -f /k8s-jenkins/files/testing.yml

sleep 15
sudo sh /root/k8s-copy.sh
elif sudo ls ./*.php
then
sudo kubectl create -f /k8s-jenkins/files/testing.yml

sleep 15
sudo sh /root/k8s-copy.shfi

Here we have used sleep of 15 seconds because while creating the testing.yml file depending upon the internet speed it downloads the image if you don’t have one as specified inside the testing.yml file. So that we will wait for some seconds before going on the next job of testing because there we will copy the code to the created persistent volume.

for podname in $(kubectl get pods -l env=testing -o custom-columns=:metadata.name | tr -d '\n'); do kubectl cp /k8s-jenkins/files/index.html "${podname}":/usr/local/apache2/htdocs; done

This is the code for k8s-copy.sh. This file will basically check for all the pods labeled with env as testing and copy the code into it. After copying it into the pod it will be automatically hosted as we have copied it in the document root or the folder from where the pod configured with web-server is reading files and its saved in the PVC hence it is permanent.

Here we will first retrieve the port number from the service and store it in the port variable. Then we will check if it’s 200 we will notify that the website is running successfully, we can even send email to developer. This, I have done in my previous article, you can refer it for sending automated mail.

JOB4:

Here we will create deployment.yml file from the testing.yml automatically using some linux command line tricks and once it’s launched we will automatically open it in the firefox browser.

sudo cp /k8s-jenkins/files/testing.yml /k8s-jenkins/files/deployment.yml
sudo sed -i 's/testing/production/g' /k8s-jenkins/files/deployment.yml
sudo sed -i 's/myweb/mywebprod/g' /k8s-jenkins/files/deployment.yml
sudo sed -i 's/claimName: mywebprod-pv-claim/claimName: myweb-pv-claim/g' /k8s-jenkins/files/deployment.yml

sudo kubectl create -f /k8s-jenkins/files/deployment.yml

port=$(sudo kubectl get service -l env=production | cut -d " " -f 15 | cut -d ":" -f 2 | cut -d "/" -f 1 | tr -d '\n')

export DISPLAY=:0
sleep 10

sudo cp /k8s-jenkins/files/testing.yml /k8s-jenkins/files/deployment.yml

This command will copy the testing.yml code into deployment.yml.

sudo sed -i ‘s/testing/production/g’ /k8s-jenkins/files/deployment.yml

sudo sed -i ‘s/myweb/mywebprod/g’ /k8s-jenkins/files/deployment.yml

sudo sed -i ‘s/claimName: mywebprod-pv-claim/claimName: myweb-pv-claim/g’ /k8s-jenkins/files/deployment.yml

This command sed -i ‘s/<find>/<replace>/g’ will find and replace the snippet, so first we have changed the env label to production and then we changed name and then to save our resources and make it more robust we have attached the same PVC which we used in the testing, this will help us in multiple ways like, save resources of creating a new PVC then if we make any changes in the testing it will automatically updated in the production or deployment. So initially we changed every name so we need to again change the claimName to the previous one as it’s already created and we need not have to create the other one.

Finally using port=$(sudo kubectl get service -l env=production | cut -d “ “ -f 15 | cut -d “:” -f 2 | cut -d “/” -f 1 | tr -d ‘\n’) we got the port number of the pod which has deployed our application in production or deployment environment.

Now, to launch it in the browser we need to first set the DISPLAY variable otherwise it will give error. So we used export DISPLAY=:0 to set the variable, this tells the system to use the local browser to display. Then we waited for 10 seconds to download and create the necessary deployment configuration. Then we launched the website, it will automatically pop on your screen.