In the realm of modern DevOps practices, the ability to swiftly provision and manage Kubernetes clusters is paramount. Kubernetes, with its robust orchestration capabilities, has become the de facto standard for containerized applications. However, setting up a Kubernetes cluster manually can be tedious and error-prone. In this blog post, we'll explore how to automate the deployment of a multi-node Kubernetes cluster on AWS EC2 instances using Jenkins.
Prerequisites:
Before diving into the automation process, ensure you have the following prerequisites:
An AWS account with appropriate permissions to provision EC2 instances.
Basic knowledge of Kubernetes architecture and components.
Jenkins installed on an EC2 instance.
Docker installed on all EC2 instances.
Step 1: Setting up Kubernetes Cluster
Begin by selecting one EC2 instance as the master node and additional instances as worker nodes. Install Kubernetes on all nodes using tools like kubeadm. Initialize the master node with kubeadm init and join worker nodes with kubeadm join.
Here we will use kubeadm tool for installing kubernetes
# Set SELinux in permissive mode (effectively disabling it)
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
# This overwrites any existing configuration in /etc/yum.repos.d/kubernetes.repo
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF
sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
After we need to install kubectl command
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
after give the file execution permission
sudo chmod +x kubectl
and after we can MV this file in default directory
sudo mv kubectl /usr/bin
Step 2: Put Master and slave node codes into github and save with .sh file.
Codes for master slave
$ swapoff -a
install the traffic control utility package
$ dnf install -y iproute-tc
$ modprobe overlay
$ modprobe br_netfilter
$ cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
$ cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
sysctl --system
# Set SELinux in permissive mode (effectively disabling it)
# disable SELinux and set it to ‘permissive’ in order to allow smooth communication between the nodes and the pods.
$ setenforce 0
$ sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
Container Runtime is an application that supports running containers, we will install CRI-O
$ export VERSION=1.26
$ curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_8/devel:kubic:libcontainers:stable.repo
$ curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/CentOS_8/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo
$ dnf install cri-o
$ systemctl enable crio
$ systemctl start crio
# This overwrites any existing configuration in /etc/yum.repos.d/kubernetes.repo
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF
sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet
initialize a Kubernetes cluster using the kubeadm command as follows. This initializes a control plane in the master node., use network range of kube server below cidr
$ kubeadm init --pod-network-cidr=192.168.0.0/16
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
By default, apps won’t get scheduled on the master node. If you want to use the master node for scheduling apps, taint the master node.
$ kubectl taint nodes --all node-role.kubernetes.io/control-plane-
For slave node
#for slave node
swapoff -a
dnf install -y iproute-tc
modprobe overlay
modprobe br_netfilter
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
sysctl --system
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
KUBERNETES_VERSION=v1.29
PROJECT_PATH=prerelease:/main
cat <<EOF | tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/rpm/repodata/repomd.xml.key
EOF
cat <<EOF | tee /etc/yum.repos.d/cri-o.repo
[cri-o]
name=CRI-O
baseurl=https://pkgs.k8s.io/addons:/cri-o:/$PROJECT_PATH/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/addons:/cri-o:/$PROJECT_PATH/rpm/repodata/repomd.xml.key
EOF
dnf install -y cri-o kubelet kubeadm kubectl
systemctl enable --now crio
systemctl enable --now kubele
Step 3: install java and jenkins and after that setup jenkins.
And after all the three steps when we will create the job and trigger of our github repository and write command in execute shell it will make a kubernetes cluster.