Waseem Shaikh
cicd

Simple CICD Pipeline Project – Todo Application

This is very basic CICD Pipeline Project for a ToDo Application. Follow the steps below to set up your Continuous Integration and Continuous Deployment pipeline.

Prerequisites

 

Ensure you have the following prerequisites before proceeding:

  1. EC2 Machine: Medium Size preferred. (4/GB RAM and 2 Core Processor)
  2. Docker Setup on EC2 Machine.
  3. Jenkins Setup: You can either manually install Jenkins or use a Jenkins image.
  4. GitHub Account: You need a GitHub account to automate the pipeline.

For more updates and projects, visit:

Architecture Diagram

 

Architecture Diagram

Step 01 - Create EC2 Machine

 

EC2 Setup Step 1 EC2 Setup Step 2 EC2 Setup Step 3 EC2 Setup Step 4

Step 02 - Connect EC2 Machine

 

Connect to your EC2 Ubuntu Machine using SSH. Make sure you're in the root user or use 'sudo'.

$ sudo ssh -i "key.pem" ubuntu@<ip-address>

EC2 Setup Step 6

Step 03 - Install Docker on EC2 Machine

 

Install Docker on your EC2 machine. Use the following commands:

Set up Docker's apt repository.

 

$ sudo apt-get update
$ sudo apt-get install ca-certificates curl
$ sudo install -m 0755 -d /etc/apt/keyrings
$ sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
$ sudo chmod a+r /etc/apt/keyrings/docker.asc
$ echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
$ sudo apt-get update

# Install Docker packages.
$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

EC2 Setup Step 7.1

Step 04 - Installation of Jenkins

 

Install Jenkins either manually or using container-based installation. Here's the container-based installation command:

$ docker run -p 8080:8080 -p 50000:50000 -d \
-v jenkins_home:/var/jenkins_home \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $(which docker):/usr/bin/docker jenkins/jenkins:lts

EC2 Setup Step 7.2

After running the above command, Jenkins container will be created and started. Access Jenkins using the IP address of your EC2 machine followed by port 8080.

EC2 Setup Step 8

Now you can type blow ip address if your ec2 machine and check whether your Jenkins is accessible or not But make sure that the port should be added on your EC2 Security Group

git-ec6

git-ec6

 

$ localhost:8080

$ <ip-address>:porty

git-ec6

 

Now, In order to get the jenkins password. you have to login or access the jenkins container

$ docker exec -it <container-id> bash

$ docker exec -it 98760878abree bash

git-ec6

 

Step 04 - Jenkins Pipeline Setup

 

Now that we have setup our EC2 server, Docker, and Jenkins, let's create a simple Jenkins pipeline.

Before creating the pipeline, ensure your Git repository is up to date. Use the following links to update:

Log in to Jenkins using the password generated during setup.

EC2 Setup Step 13

Choose 'Pipeline' and 'Pipeline Script from SCM', using the Jenkinsfile provided in your Git repository. EC2 Setup Step 14

EC2 Setup Step 15

EC2 Setup Step 16

After setting up the pipleline and connected to git repo. now lets test the below code and see your our pipeline works or not

First create the Jenkinsfile in your git repo and add the below code in Jenkinsfile

pipeline {
agent any
stages {
stage("Code checkout") {
steps {
git branch: 'main', credentialsId: 'github-id', url: 'https://github.com/waseemuddin/simple-cicd-project01.git'
}
}
}
}

EC2 Setup Step 17

Now Jenkinsfile has been updated. Goto pileline and press the "Build Now" to check the pipeline fetch git repo and shows the stage view.

EC2 Setup Step gif01

EC2 Setup Step 18

Step 05 - Jenkins Set Webhook - Automate the Pipeline Process

 

To automate the Jenkins pipeline, we'll set up a Git webhook. Navigate to your repository settings and add a webhook with the Jenkins server's IP and port.

EC2 Setup Step 19

Step 06 - Final Step - Run Complete Code

 

In the final step, we create a complete pipeline from code checkout to container creation.

The updated code is available in the Jenkinsfile. Ensure you have an empty repository on your Docker Hub account for image storage.

pipeline {
agent any

stages {
stage("Code checkout") { # Git Repo Check-out Step
steps {
git branch: 'main', credentialsId: 'github-id', url: 'https://github.com/waseemuddin/simple-cicd-project01.git'
}
}
stage("image build") { # Image Build Step and make sure you have to create the empty repository on your Docker-Hub Account
steps {
sh 'docker image build -t waseem63/mydockerapp:v$BUILD_ID .'
sh 'docker image tag waseem63/mydockerapp:v$BUILD_ID waseem63/mydockerapp:latest'
}
}
stage("Image Push") { # Image Push Step and connect to your docker-hub account. Also remove the image so the storage can be maintain
steps {
withCredentials([usernamePassword(credentialsId: 'docker-hub-id', passwordVariable: 'PASS', usernameVariable: 'USER')]) {

sh "echo $PASS | docker login -u $USER --password-stdin"
sh 'docker push waseem63/mydockerapp:v$BUILD_ID'
sh 'docker push waseem63/mydockerapp:latest'
sh 'docker rmi waseem63/mydockerapp:v$BUILD_ID waseem63/mydockerapp:latest'
}
}

}
stage("Container Creating") { # last step is to fetch the lastet or recent image and create docker container automatically.
steps {
sh 'docker run -itd --name todoapp -p 3000:3000 waseem63/mydockerapp:latest'

}
}

}
}

EC2 Setup Step 21

Now, any push to the repository will trigger the Jenkins pipeline automatically.

Congratulations! You have successfully set up a CI/CD pipeline for your Todo Application.

Create Your First Git Repository And Push to GitHub Account

Create Your First Git Repository And Push to GitHub Account

GitHub is a popular platform for hosting and collaborating on code repositories. Whether you are new to coding or an experienced developer, knowing how to create a repository and push your code to GitHub is a crucial skill.

In this step-by-step Lab, I will guide you through the process of creating your first GitHub repository and pushing your code to your GitHub Account. By the end of this Lab, you will have the necessary skills to get started with version control, code hosting, and collaboration on GitHub. I will try my best to cover topics such as GitHub basics, repository creation, and pushing code to GitHub.

Youtube Video: https://youtu.be/pHLb3GMyNhI

GitHub Repo :https://github.com/waseemuddin/Git-for-beginners.git

Github Diagram

git-status

Step 01 - Make Directory/Folder on your local system windows or ubuntu

$ mkdir Git-for-begineers
$ cd Git-for-begineers
$ git status

git-status

Step 02 - create some files into the git local directory/folder

$ mkdir imges
$ mkdir js
$ mkdir css
$ mkdir project
$ touch css/front.css
$ touch js/mysim.js
$ touch index.php

Step 03 - After editing and adding completed, run the below commands

$ git init
$ git status
$ git add .
$ git commit

git-init

git-status

git-add

git-commit

Our local repository files are commited and now its time to push into remote repository

git-local

To push your code to remote repository, first you have to create the empty reposity on your github account

Step 04 - Create Empty Repo on your Github Account

Got your github accunt profile and create new repository

git-local

git-local

Step 05 - Push Local Repo to Remote Repo

Now its time to push your local repository to your remote repository means your Github account

git-local

git-local

git-local

git-local

As you see all local repository files/folders are now push to remote repository

git-local

Kubedoom DevOps Project

Kube DOOM Project for Kubernetes learning purposes

 

Kill Kubernetes pods using Id's Doom!

 

The next level of chaos engineering is here! Kill pods inside your Kubernetes cluster by shooting them in Doom!

Youtube Video: https://youtu.be/pHLb3GMyNhI

GitHub Repo : https://github.com/waseemuddin/kubedoom-project-devops

This repo contains about the Kube Doom project for learning and implementing the Kubernetes and killing the pods inside your kubernetes cluster.

The credit goes to iiDKx & storax/kubedoom.

The following steps are involved to implement the KubeDoom Project

Prerequisites

 

This is project can be implemented on your local machine (VM), AWS Cloud or any other cloud provider which suites you.

kubedoom Diagram

podskilling

 

Step 1: Update VM/Ubuntu

 

After installation of VM (ubuntu), please make sure your sytstem is updated

#Basic command to make sure all are update
sudo apt update
sudo apt-get update
#Check IP
ifconfig
ip --brief addr show
#Check firewall
sudo ufw status
sudo ufw allow 22/tcp
#swapoff
sudo -i
swapoff - a

Step 2: Steup Kubernetes Cluster (Kind)

 

Running Kubedoom inside Kubernetes

Install kind master and worker node

See the example in the /manifest directory. You can quickly test it using kind. Create a cluster with the example config from this repository:

$ kind create cluster --config kind-config.yaml
Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.25.3) 🖼
✓ Preparing nodes 📦 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
✓ Joining worker nodes 🚜
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind
Not sure what to do next? 😅 Check out https://kind.sigs.k8s.io/docs/user/quick-start/

Kind Master and worker node

 

Set your Kube context with: kubectl cluster-info --context kind-kind

This will spin up a 2 node cluster inside docker, with port 5900 exposed from the worker node. Then run kubedoom inside the cluster by applying the manifest provided in this repository. The Kubernetes-Manifests-Files directory holds Kubernetes manifests for deploying your application on cluster.

$ kubectl apply -k manifest/
namespace/kubedoom created
deployment.apps/kubedoom created
serviceaccount/kubedoom created
clusterrolebinding.rbac.authorization.k8s.io/kubedoom created

kubedoom namespace

 

Step 3: Docker Build

 

Build the image with docker build --build-arg=TARGETARCH=amd64 . -t kubedoom while in this directory. Then run:

kubedoom namespace

 

#docker run command and and attached pods
$ docker run -p5801:5800 \
-e NAMESPACE=default \
--net=host \
-v ~/.kube:/root/.kube \
--rm -it --name kubedoom \
kubedoom:latest

Optionally, if you set -e NAMESPACE={your namespace} you can limit Kubedoom to deleting pods in a single namespace

kubedoom namespace

 

Step 4: Docker Build

 

Deploy some Nginx pods to your cluster:

kubectl apply -f k8s/nginx-deployment.yaml

You can refresh the dashboard as you kill the pods from Kubedoom and expand upon this to track other metrics and applications in your cluster.

kubedoom namespace

 

Step 5: Attaching a VNC Client

 

Now start a VNC viewer and connect to localhost:5900. The password is idbehold:

$ vncviewer viewer localhost:5901

You should now see DOOM! Now if you want to get the job done quickly enter the cheat idspispopd and walk through the wall on your right. You should be greeted by your pods as little pink monsters. Press CTRL to fire. If the pistol is not your thing, cheat with idkfa and press 5 for a nice surprise. Pause the game with ESCiddqd for god mode.

Cheat codes found here: https://doom.fandom.com/wiki/Doom_Cheat_Codes

kubedoom namespace

 

kubedoom namespace

 

To connect run:
$ vncviewer viewer localhost:5900

Step 6: Kubedoom demo

 

podskilling

 

kubedoom namespace

 

Create a dashboard in Grafana to monitor the Nginx containers. To do this open Grafana at http://localhost:3000 and login. Make sure you have your data source set to your Prometheus pod from the previous step. From the left hand menu, create a new dashboard and add a panel. Select the panels dropdown menu, select Inspect and then select Panel JSON. Here you will be able to delete the current JSON and replace it with JSON from the grafana folder ./grafana/nginx-panel.json. Save and apply this and you should be able to see the CPU usage of the current deployed pods.

Step 7: Deploying Prometheus

 

Create a monitoring namespace to keep things tidy.

kubectl create namespace monitoring

Deploy Prometheus to scrape and store metrics for your cluster with:

kubectl apply -f k8s/prometheus.yaml -n monitoring

List the Prometheus pod name and IP address.

$ kubectl get pods -o wide -n monitoring
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
prometheus-deployment-75cff7d89f-w422q 1/1 Running 1 (15m ago) 25m 10.244.1.3 kind-worker <none> <none>

In a separate terminal run the below command to port-forward and you'll be able to access Prometheus on http://localhost:8080:

kubectl port-forward -n monitoring prometheus-deployment-75cff7d89f-w422q 8080:9090

To run in background:

kubectl port-forward -n monitoring prometheus-deployment-75cff7d89f-w422q 8080:9090 &

Prometheus graph display command.

rate(container_cpu_usage_seconds_total{namespace="default", container="nginx"}[30s]) * 100

Step 8: Deploying Grafana

 

Deploy Grafana to graph our metrics from Prometheus with:

kubectl apply -f k8s/grafana.yaml -n monitoring

In another terminal, list the pod names and port-forward from one of the pods:

$ kubectl get pods -n monitoring
NAME READY STATUS RESTARTS AGE
grafana-5469c64c7d-ddz4r 1/1 Running 1 (20m ago) 30m
grafana-5469c64c7d-xdlmw 1/1 Running 1 (20m ago) 30m
prometheus-deployment-75cff7d89f-w422q 1/1 Running 1 (20m ago) 30m

$ kubectl port-forward -n monitoring grafana-5469c64c7d-ddz4r 3000
Forwarding from 127.0.0.1:3000 -> 3000
Forwarding from [::1]:3000 -> 3000

Grafana should now be reachable at http://localhost:3000.

Log in with username admin and password admin.

Once you are logged in, you will need to go to the Settings (gear icon bottom left) and edit the Data sources.

Change the URL in the settings from http://prometheus-service.monitoring.svc:8080 to http://<Prometheus Pod IP>:9090. Use the below command to get the Prometheus pod IP.

$ kubectl get pods -o wide -n monitoring
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
prometheus-deployment-75cff7d89f-w422q 1/1 Running 1 (15m ago) 25m 10.244.1.3 kind-worker <none> <none>

So I would enter http://10.244.1.3:9090 as my data source. Save and test this.

Kubedoom requires a service account with permissions to list all pods and delete them and uses kubectl 1.25.3.

Running Locally

 

In order to run locally you will need to

  1. Run the kubedoom container
  2. Attach a VNC client to the appropriate port (5901)

With Podman

 

Run kubedoom:latest with podman locally:

$ podman run -it -p5901:5900/tcp \
-v ~/.kube:/tmp/.kube --security-opt label=disable \
--env "KUBECONFIG=/tmp/.kube/config" --name kubedoom
kubedoom:latest

Killing namespaces

 

Kubedoom now also supports killing namespaces in case you have too many of them. Simply set the -mode flag to namespaces:

$ docker run -p5901:5900 \
--net=host \
-v ~/.kube:/root/.kube \
--rm -it --name kubedoom \
kubedoom:latest \
-mode namespaces

Building Kubedoom

 

The repository contains a Dockerfile to build the kubedoom image. You have to specify your systems architecture as the TARGETARCH build argument. For example amd64 or arm64.

$ docker build --build-arg=TARGETARCH=amd64 -t kubedoom .

To change the default VNC password, use --build-arg=VNCPASSWORD=differentpw.