Aman kant Mahto's Blog

Building multi node cluster of kubernetes automatically with Jenkins

Building multi node cluster of kubernetes automatically with Jenkins

Aman kant Mahto's photo
Aman kant Mahto
·

3 min read

In the realm of modern DevOps practices, the ability to swiftly provision and manage Kubernetes clusters is paramount. Kubernetes, with its robust orchestration capabilities, has become the de facto standard for containerized applications. However, setting up a Kubernetes cluster manually can be tedious and error-prone. In this blog post, we'll explore how to automate the deployment of a multi-node Kubernetes cluster on AWS EC2 instances using Jenkins.

Prerequisites:

Before diving into the automation process, ensure you have the following prerequisites:

  • An AWS account with appropriate permissions to provision EC2 instances.

  • Basic knowledge of Kubernetes architecture and components.

  • Jenkins installed on an EC2 instance.

  • Docker installed on all EC2 instances.

Step 1: Setting up Kubernetes Cluster

Begin by selecting one EC2 instance as the master node and additional instances as worker nodes. Install Kubernetes on all nodes using tools like kubeadm. Initialize the master node with kubeadm init and join worker nodes with kubeadm join.

Here we will use kubeadm tool for installing kubernetes

# Set SELinux in permissive mode (effectively disabling it)
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

# This overwrites any existing configuration in /etc/yum.repos.d/kubernetes.repo
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF

sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

After we need to install kubectl command

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

after give the file execution permission

sudo chmod +x kubectl

and after we can MV this file in default directory

sudo mv kubectl /usr/bin

Step 2: Put Master and slave node codes into github and save with .sh file.

Codes for master slave



$ swapoff -a

install the traffic control utility package
$ dnf install -y iproute-tc

$ modprobe overlay
$ modprobe br_netfilter

$ cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF


$ cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sysctl --system

# Set SELinux in permissive mode (effectively disabling it)
# disable SELinux and set it to ‘permissive’ in order to allow smooth communication between the nodes and the pods.
$ setenforce 0
$ sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config


Container Runtime is an application that supports running containers, we will install CRI-O
$ export VERSION=1.26
$ curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_8/devel:kubic:libcontainers:stable.repo
$ curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/CentOS_8/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo

$ dnf install cri-o
$ systemctl enable crio
$ systemctl start crio




# This overwrites any existing configuration in /etc/yum.repos.d/kubernetes.repo
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF


sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet

initialize a Kubernetes cluster using the kubeadm command as follows. This initializes a control plane in the master node., use network range of kube server below cidr
$ kubeadm init --pod-network-cidr=192.168.0.0/16


$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

By default, apps won’t get scheduled on the master node. If you want to use the master node for scheduling apps, taint the master node.
$ kubectl taint nodes --all node-role.kubernetes.io/control-plane-

For slave node

#for slave node
swapoff -a
dnf install -y iproute-tc

modprobe overlay
modprobe br_netfilter

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sysctl --system

setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

KUBERNETES_VERSION=v1.29
PROJECT_PATH=prerelease:/main

cat <<EOF | tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/rpm/repodata/repomd.xml.key
EOF

cat <<EOF | tee /etc/yum.repos.d/cri-o.repo
[cri-o]
name=CRI-O
baseurl=https://pkgs.k8s.io/addons:/cri-o:/$PROJECT_PATH/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/addons:/cri-o:/$PROJECT_PATH/rpm/repodata/repomd.xml.key
EOF


dnf install -y cri-o kubelet kubeadm kubectl

systemctl enable --now crio

systemctl enable --now kubele

Step 3: install java and jenkins and after that setup jenkins.

And after all the three steps when we will create the job and trigger of our github repository and write command in execute shell it will make a kubernetes cluster.

JenkinsGitHubKubernetescluster