Mastering DevSecOps: Deploying .NET Application on Jenkins, Docker, and Kubernetes
PermalinkIntroduction
In this article, we will explore the deployment of a .NET-based application, a scenario frequently encountered by many organizations. Our deployment process will leverage Jenkins as the CI/CD tool and will involve deploying the application in Docker containers within a Kubernetes cluster.
This article aims to provide a detailed guide on the deployment process and includes comprehensive metrics, such as the CPU performance of the instance where the project is launched.
PermalinkStep 1: Clone the GitHub Repository
Clone the repository containing the DotNet Application deployment scripts
git clone https://github.com/Saurabh-DevOpsVoyager77/DotNet-App-Deployment-DevSecOps.git
PermalinkStep 2: Building an Infra using terraform
We'll build a simple infrastructure using Terraform. The infrastructure includes a user data script that installs Jenkins, Docker, and Trivy and starts a SonarQube container on port 9000. We'll run the Terraform commands to initialize, validate, plan, and apply the infrastructure configuration.
Run Terraform Commands:
cd Instance_Terraform/
terraform init
terraform validate
terraform plan
terraform apply --auto-approve
Using the script provided below, we will install Jenkins, Docker, Trivy, and other necessary packages. Additionally, the script will start the SonarQube container. Please note that if you have followed the previous steps, this script may have already been executed.
#!/bin/bash
sudo apt update -y
sudo touch /etc/apt/keyrings/adoptium.asc
sudo wget -O /etc/apt/keyrings/adoptium.asc https://packages.adoptium.net/artifactory/api/gpg/key/public
echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | sudo tee /etc/apt/sources.list.d/adoptium.list
sudo apt update -y
sudo apt install temurin-17-jdk -y
/usr/bin/java --version
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \
/usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update -y
sudo apt-get install jenkins -y
sudo systemctl start jenkins
sudo systemctl status jenkins
#Install docker
sudo apt install docker.io -y
sudo usermod -aG docker ubuntu
newgrp docker
sudo chmod 777 /var/run/docker.sock
docker version
#SonarQube Container
docker run -d --name sonar -p 9000:9000 sonarqube:lts-community
# Install Trivy
sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y
Go to the AWS Dashboard in the EC2 service, where you can see the Instance.
PermalinkStep 3: Set Up SonarQube and Jenkins
PermalinkA. SonarQube
Copy the public IP of your server/machine.
Open your browser and navigate to
<public-ip>:9000
.
- When the SonarQube window opens, log in with the initial credentials:
Username: admin
Password: admin
Update your password with the New Password.
You will see the welcome window of SonarQube.
PermalinkB. Jenkins
Open your browser and navigate to
<public-ip>:8080
.Connect to your EC2 instance
Run the following commands:
sudo su
cat /var/lib/jenkins/secrets/initialAdminPassword
Copy the output (this is your Jenkins initial admin password).
Paste the password into Jenkins to unlock it.
Install the suggested plugins.
Set up your Jenkins user.
Successfully you have entered into the Jenkins dashboard!
PermalinkStep 4: CI/CD Pipeline
PermalinkA. Installation of Plugins
Go to the Jenkins Dashboard -> Manage Jenkins -> Plugins and Install the Following Plugins:
Eclipse Temurin Installer: Install without restart
SonarQube Scanner: Install without restart
OWASP Dependency-Check Plugin
Download Docker-related Plugins: Docker, Docker Commons, Docker Pipeline, Docker API
Kubernetes
Kubernetes CLI
Kubernetes Client API
Kubernetes Pipeline DevOps Steps
PermalinkB. Add Credentials for SonarQube and Docker
I. SonarQube Credentials Setup:
Go to publicip:9000.
Log in with your username and password.
Click on Administration โ Security โ Users โ Token โ Generate Token.
Set
token_name
asSonar-token
.
- Copy the Token.
II. Configure Jenkins:
Copy the generated token.
Go to your Jenkins dashboard.
Navigate to Manage Jenkins โ Credentials โ System.
Click on Global โ Add Credentials.
Select Secret text from the dropdown.
Set the Secret as your token.
Set the ID as
jenkins
.Click on Create.
III. Setup projects in SonarQube for Jenkins
Go to your SonarQube server.
Click on projects and in the name field type
DotNet-Deployment-DevSecOps
.Click on set up. Then you can see the screen like below
Click on the above option i.e Locally.
Click on Generate.
Click on continue.
SonarQube Project for Jenkins is setup now.
Now, go to Dashboard โ Manage Jenkins โ System
IV. Setup Docker Credentials
Go to your Jenkins dashboard.
Navigate to Manage Jenkins โ Manage Credentials.
Click on Global โ Add Credentials.
Provide your DockerHub username and password.
Set the ID as docker.
Click on Create.
PermalinkC. Setup Tools for Jenkins
I.Add JDK:
Go to Manage Jenkins โ Tools.
Click on Add JDK and select the installer from adoptium.net.
Choose JDK version 17.0.8.1+1 and name it as jdk17.
II. Add Docker:
Click on Add Docker.
Set the name as
docker
.Choose the installer option to download from docker.com.
III. Add SonarQube Scanner:
Add Sonar Scanner.
Name it as
sonar-scanner/SonarQubeScanner.
IV. Add OWASP Dependency-Check:
Add Dependency Check.
Name it as
DP-Check
.Select the installer option to install from github.com.
- Click Apply & Save.
PermalinkD. Configure Global Settings for SonarQube and Setup Webhooks
I. Configure Global Settings:
Go to Manage Jenkins โ System โ Add SonarQube Servers.
Set the name as sonar-server.
Set the server URL as http://public_ip:9000.
Set the server authentication token as jenkins(It is created in sonarqube security configuration).
II. Setup Webhooks in SonarQube:
Go to Administration โ Configuration โ Webhooks.
Create a webhook.
Click on Create.
Enter the URL as http://jenkins-public-ip:8080/sonarqube-webhook/.
PermalinkE. Running the Pipeline
Create a New Pipeline:
Go to New Item โ Select Pipeline.
Name it as
DotNet-pipeline
Paste the below pipeline like this
pipeline {
agent any
tools {
jdk 'jdk 17'
// Assuming 'sonar-scanner' tool configuration is correctly set up in Jenkins Global Tool Configuration
}
environment {
SCANNER_HOME = tool 'sonar-scanner'
}
stages {
stage('Clean Workspace') {
steps {
cleanWs()
}
}
stage('Checkout From Git') {
steps {
git branch: 'main', url: 'https://github.com/Saurabh-DevOpsVoyager77/DotNet-App-Deployment-DevSecOps.git'
}
}
stage('Sonarqube Analysis') {
steps {
withSonarQubeEnv('sonar-server') {
sh """$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName='Dotnet-Webapp' \
-Dsonar.projectKey='Dotnet-Webapp'"""
}
}
}
stage('Quality Gate') {
steps {
script {
waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
}
}
}
stage('TRIVY File Scan') {
steps {
sh "trivy fs . > trivy-fs_report.txt"
}
}
stage('OWASP Dependency Check') {
steps {
dependencyCheck additionalArguments: '--scan ./ --format XML', odcInstallation: 'DP-Check'
archiveArtifacts artifacts: '**/dependency-check-report.xml', allowEmptyArchive: true
}
}
stage('Docker Build & Tag') {
steps {
script {
withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
sh "make image"
}
}
}
}
stage('TRIVY Image Scan') {
steps {
sh "trivy image nacromancer858/dotnet-monitoring:latest > trivy.txt"
}
}
stage('Docker Push') {
steps {
script {
withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
sh "make push"
}
}
}
stage("Deploy to container"){
steps{
sh "docker run -d --name dotnet -p 5000:5000 nacromancer858/dotnet-monitoring:latest"
}
}
}
}
}
Now to see the report, you can go to SonarQube Server and go to Projects.
Also, after running the above pipeline you can see your application on http:<PublicIP>:5000
PermalinkStep 5: Setup Kubernetes (K8s)
Go to cloned repo and then
cd k8s_Instances/
Run
terraform init
and thenterraform apply --auto-approve
It will create 2 instance One with the named Master and other with Worker.
By default I have provided scripts into it. It will install all dependencies.
LogIn into the Worker Node
Run the following command:
sudo kubeadm join <master-node-ip>:<master-node-port> --token <token> --discovery-token-ca-cert-hash <hash>
- Now, LogIn into the Master Node
cd .kube/
cat config
Copy the config file to Jenkins master or the local file manager and save it.
Copy it and save it in documents or another folder save it as secret-file.txt
Add K8s Credentials
Go to manage Jenkins โ manage credentials โ Click on Jenkins global โ add credentials
- The final step to deploy on the Kubernetes cluster, add this stage to the pipeline.
stage('Deploy to k8s'){
steps{
dir('K8S') {
withKubeConfig(caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s', namespace: '', restrictKubeConfigAccess: false, serverUrl: '') {
sh 'kubectl apply -f deployment.yaml'
}
}
}
}
Before starting a new build remove old containers.
Output:
kubectl get svc
#copy service port
<worker-ip:svc port>
You can see over there
PermalinkStep 6: Destroy All the Infrastructure
Go to your terminal, locate the directories and run:
cd /k8s_Instances terraform destroy --auto-approve cd /Instance_Terraform terraform destroy --auto-approve
PermalinkConclusion
By following the guide, organizations can streamline workflows using Docker containers and Kubernetes for scalability. Monitoring CPU performance offers insights into operational efficiency, enhancing deployment reliability and paving the way for resilient, scalable architectures in modern IT landscapes.