Kbaran: /* Cluster initialisation */

2023-10-04T09:45:17Z

Cluster initialisation

← Older revision		Revision as of 09:45, 4 October 2023
Line 66:		Line 66:
	* start cluster networking and check the status once again		* start cluster networking and check the status once again
	<syntaxhighlight lang="Bash">		<syntaxhighlight lang="Bash">
	kubectl apply -f "https://~~cloud~~.weave.~~works~~/~~k8s/net?k8s~~-~~version=$(kubectl version \| base64 \| tr~~ -~~d '\n')"~~		kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml
	</syntaxhighlight>		</syntaxhighlight>

Kbaran at 23:16, 20 October 2021

2021-10-20T23:16:49Z

New page

[[Category:Docker]]
[[Category:course_code_dockcm]]

{{Copyright Notice}}

== Author ==
* [http://www.kamilbaran.pl Kamil Baran]
 

== Environment preparation - local virtual machines ==
* Change ''server-s1, server-s2 and server-s3'' network configuration from DHCP into static IP (edit '''/etc/network/interfaces''').
* Verify the host names and IP addresses in '''/etc/hostname''' and '''/etc/hosts'''.
* Remove SWAP partition from '''/etc/fstab'''

<syntaxhighlight lang="Bash">
# on your client machine generate keys, then transfer public key to server-s1, server-s2, server-s2
$ ssh-keygen
$ ssh-copy-id student@server-s1
$ ssh-copy-id student@server-s2
$ ssh-copy-id student@server-s2
# check the connection
$ ssh student@server-s1
</syntaxhighlight>
 

== How to create a secure Kubernetes cluster? ==
=== Docker installation ===
<syntaxhighlight lang="Bash">
sudo apt-get update && sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update && apt-cache madison docker-ce
sudo apt-get install -y docker-ce=5:20.10.8~3-0~ubuntu-focal
sudo usermod -aG docker nobleprog
</syntaxhighlight>
* more info: docs.docker.com/engine/installation/linux/docker-ce/ubuntu/ and kubernetes.io/docs/setup/production-environment/container-runtimes/
 

=== Kubernetes installation ===
<syntaxhighlight lang="Bash">
sudo apt-get update && sudo apt-get install -y apt-transport-https bash-completion curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo add-apt-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
sudo apt-get update && sudo apt-get install -y kubeadm=1.21.5-00 kubectl=1.21.5-00 kubelet=1.21.5-00
source <(kubectl completion bash) && echo "source <(kubectl completion bash)" >> ~/.bashrc
source <(kubeadm completion bash) && echo "source <(kubeadm completion bash)" >> ~/.bashrc
</syntaxhighlight>
* more info: kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
 

=== Cluster initialisation ===
* initialise the master on server-s0 and configure kubectl
<syntaxhighlight lang="Bash">
sudo kubeadm init
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
</syntaxhighlight>

* check the status of the master node and pods running in the kube-system namespace
<syntaxhighlight lang="Bash">
kubectl get node,pod --namespace=kube-system
kubectl describe pod --namespace=kube-system kube-dns-86f...
</syntaxhighlight>

* start cluster networking and check the status once again
<syntaxhighlight lang="Bash">
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
</syntaxhighlight>

* optionally allow pods to be scheduled on the master node (useful if you need a single-node Kubernetes cluster)
<syntaxhighlight lang="Bash">
kubectl taint nodes --all node-role.kubernetes.io/master-
</syntaxhighlight>

* join server-s1 and server-s2 to the cluster as worker nodes (aka minions)
<syntaxhighlight lang="Bash">
ssh ubuntu@server-s1
sudo kubeadm join --token bf37a8... 11.0.2.10:6443 --discovery-token-ca-cert-hash sha256:9dfed...
exit
ssh ubuntu@server-s2
sudo kubeadm join --token bf37a8... 11.0.2.10:6443 --discovery-token-ca-cert-hash sha256:9dfed...
exit
</syntaxhighlight>

* check the status of all nodes
<syntaxhighlight lang="Bash">
kubectl get nodes
</syntaxhighlight>

* more info: kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
 

== Deploying sample applications in Kubernetes cluster ==

=== Deploy Training App ===
* deploy an app by creating Deployment and Service resource, then check the status of both objects
<syntaxhighlight lang="YAML">
apiVersion: apps/v1
kind: Deployment
metadata:
name: training
labels:
app: training
spec:
selector:
matchLabels:
app: training
replicas: 3
template:
metadata:
labels:
app: training
spec:
containers:
- name: training
image: kamilbaran/training:app-v1
</syntaxhighlight>

<syntaxhighlight lang="YAML">
apiVersion: v1
kind: Service
metadata:
name: training
labels:
app: training
spec:
type: ClusterIP
selector:
app: training
ports:
- port: 80
protocol: TCP
targetPort: 80
</syntaxhighlight>

<syntaxhighlight lang="Bash">
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-app-base.yaml
kubectl get service,deployment,pod
</syntaxhighlight>

* run the following loop in the second terminal window to see the application responses
<syntaxhighlight lang="Bash">
while :;
do
((i++))
curl --connect-timeout 2 $(kubectl get service -o go-template='{{(.spec.clusterIP)}}' training):80/?count=$i;
sleep 1;
done
</syntaxhighlight>

* scale the deployment up to five pods by changing spec.replicas in training-app-base.yaml and re-applying the file or by running the following command
<syntaxhighlight lang="Bash">
kubectl scale deployment --replicas=5 simple-app
</syntaxhighlight>
 
==== Rolling update ====
* imagine that you have the newer version of the application and you want to replace the current one
** current one is based on training-app-base.yaml
** new version is using image kamilbaran/training:app-v2
** make sure that you have multiple replicas of the current version
* update the deployment by changing .spec.template.spec.containers[0].image in training-app-base.yaml and re-applying the file or by running the following command
<syntaxhighlight lang="Bash">
kubectl set image deployment/training training=kamilbaran/training:app-v2
</syntaxhighlight>

* before and after every step you can display all objects that are affected
<syntaxhighlight lang="Bash">
kubectl get deployment,replicaset,pod -l app=training
</syntaxhighlight>

* run the commands below to check the current status
<syntaxhighlight lang="Bash">
kubectl rollout status deployment training
</syntaxhighlight>

* to go back to the previous working configuration and check the status once again
<syntaxhighlight lang="Bash">
kubectl rollout undo deployment training
</syntaxhighlight>

* '''Exercise''': upgrade the app to image kamilbaran/training:app-v3
 
==== Blue-Green deployment ====
* using B/G deployment in Kubernetes is easy and helpful when you need to upgrade many components of your app at once
* to achieve this, you will need two deployments (training and training-new) and one service (training)
 
* the general idea is as follows:
# create the current version that needs to be updated (use training-app-base.yaml as a starting point)
# before creating the second deployment (training-new) use labels and selectors to make sure that the service will use pods only from the first deployment (training)
# create the second deployment (training-new) and wait for all pods being available
# change the selectors in the service to use pods only from the second deployment (training-new)
 

* blue/green deployment can be implemented in Kubernetes cluster as follows:
** start the current/base version
<syntaxhighlight lang="Bash">
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-app-base.yaml
</syntaxhighlight>

** before and after every step you can display all objects that are affected (keep an eye also at the results returned by the loop)
<syntaxhighlight lang="Bash">
kubectl get service,deployment,pod -l app=training -L app -L release -o wide
</syntaxhighlight>

** before creating the second deployment (training-new) use labels and selectors to make sure that the service will use pods only from the first deployment (training)
** step 1: add new label release=stable to deployment
** step 2: change service selector to include the release label
<syntaxhighlight lang="Bash">
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-app-bg-step1.yaml
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-app-bg-step2.yaml
</syntaxhighlight>

** step 3: create the second deployment training-new (it is using an image with the new version and two labels: app=training, release=new
<syntaxhighlight lang="Bash">
kubectl apply -f http://www.kamilbaran.pl/download/kubernetes/training-app-bg-step3.yaml
</syntaxhighlight>

** now the service is using still only pods from training service
** step 4: change the value of label release from stable to new to switch to the pods with a new version
<syntaxhighlight lang="Bash">
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-app-bg-step4.yaml
</syntaxhighlight>

** at this point service is using the new version, but there are still two deployments and pods that are wasting resources
** you should clean-up the cluster, and this is one of the possible ways:
** step 5: update the training deployment to new version
** step 6: switch the service to use the release=stable selector
** step 7: delete or downscale training-new to zero and leave it for the next image update
<syntaxhighlight lang="Bash">
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-app-bg-step5.yaml
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-app-bg-step6.yaml
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-app-bg-step7.yaml
</syntaxhighlight>
 
==== Canary deployment ====
* using Canary deployment in Kubernetes is useful if you want to try the new version on a selected subset of requests
* for example, let's say that you want to send only 10% of all requests to the new version of your app and if everything is OK to increase the ratio from 9:1 to 7:3
* to achieve this, you will need two deployments (training and training-new) and one service (training)
 
* the general idea is as follows:
# create the current version that needs to be updated (use training-app-base.yaml as a starting point)
# before creating the second deployment (training-new) use labels and selectors to make sure that the service will use pods only from the first deployment (training)
# create the second deployment (training-new) and wait for all pods being available
# change the selectors in the service to '''use pods from both deployments''' in the same time
# modify the ratio by changing the number of pods in the first and second deployment (you will send 20% of requests to the new version if you start eight pods in the first deployment and only two pods in the second deployment)
 
* '''Exercise''': write Canary deployment step-by-step config files for the Training app (like in the B/G deployment presented above)
 

=== Deploy Wordpress ===
<syntaxhighlight lang="Bash">
kubectl create secret generic mysql-pass --from-literal=password=NobleProg
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/kube-wp-volumes.yaml
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/kube-wp-mysql.yaml
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/kube-wp-wordpress.yaml
</syntaxhighlight>

* display all related objects to Wordpress
<syntaxhighlight lang="Bash">
kubectl get secret,service,deployment,pod,pvc,pv
</syntaxhighlight>

* display logs of mysql container running in mysql-bcc89f687-hn2tl pod
<syntaxhighlight lang="Bash">
kubectl logs mysql-bcc89f687-hn2tl mysql
</syntaxhighlight>

* get more details about the current state of wordpress-57955d6d4f-9qxvz pod (including pod related events)
<syntaxhighlight lang="Bash">
kubectl describe pod wordpress-57955d6d4f-9qxvz
</syntaxhighlight>

* edit persistent volume to increase it's size
<syntaxhighlight lang="Bash">
kubectl edit pv pv-1
</syntaxhighlight>

* delete pod with mysql to check if the app is able to persist the data
<syntaxhighlight lang="Bash">
kubectl delete pod mysql-bcc89f687-hn2tl
</syntaxhighlight>

== Kubernetes objects ==
=== StatefulSet ===

* deploy sample StatefulSet and related Service
<syntaxhighlight lang="Bash">
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/kube-statefulset.yaml
kubectl get service,statefulset,pod -o wide -l example=StatefulSet
</syntaxhighlight>

* start a pod based on the alpine image (or attach to existing one) and install curl
<syntaxhighlight lang="Bash">
kubectl run -i -t alpine --image=alpine --namespace=default
apk add --update curl
kubectl attach -i -t alpine-695c86655b-xfnzg
</syntaxhighlight>

* use curl to access simple-app pods
<syntaxhighlight lang="Bash">
curl simple-app-0.simple-app
curl simple-app-0.simple-app.default.svc.cluster.local
curl simple-app.default.svc.cluster.local
</syntaxhighlight>

* remove everything related to StatefulSet example
<syntaxhighlight lang="Bash">
kubectl delete all -l example=StatefulSet
</syntaxhighlight>

* '''Exercise''': start the MongoDB Counter App using StatefulSet with three replicas
 

=== NetworkPolicy ===

* deploy sample Training App and make sure there are no existing network policies
<syntaxhighlight lang="Bash">
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-app-base.yaml
kubectl get svc,deploy,pod,networkpolicy
</syntaxhighlight>

* use wget to access deployed application from another container
<syntaxhighlight lang="Bash">
kubectl run alpine -it --rm --image=alpine /bin/sh
wget --spider --timeout 1 training
</syntaxhighlight>

* apply network policy that will allow connections to Training App only from pods labeled trusted=yes
<syntaxhighlight lang="Bash">
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-network-policy.yaml
</syntaxhighlight>

* try to access the application again, then change the label trusted=yes and try one more time
<syntaxhighlight lang="Bash">
wget --spider --timeout 1 training
kubectl label pod --overwrite alpine-... trusted="yes"
wget --spider --timeout 1 training
</syntaxhighlight>
* more info: kubernetes.io/docs/concepts/services-networking/network-policies/
 

=== Ingress ===

* deploy Nginx as Ingress Controller
<syntaxhighlight lang="Bash">
kubectl apply -f https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/v1.3.0/install/common/ns-and-sa.yaml
kubectl apply -f https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/v1.3.0/install/common/default-server-secret.yaml
kubectl apply -f https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/v1.3.0/install/common/nginx-config.yaml
kubectl apply -f https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/v1.3.0/install/rbac/rbac.yaml
kubectl apply -f https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/v1.3.0/install/daemon-set/nginx-ingress.yaml
kubectl apply -f https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/v1.3.0/install/service/nodeport.yaml (or /loadbalancer.yaml)
kubectl get all --namespace=nginx-ingress
</syntaxhighlight>

* deploy sample Training App and Ingress Resource and try to access the app
<syntaxhighlight lang="Bash">
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-ingress-backend.yaml
kubectl apply -f https://www.kamilbaran.pl/training/kubernetes/training-ingress.yaml
kubectl get ingress,svc,deploy,pod
</syntaxhighlight>
* more info: kubernetes.io/docs/concepts/services-networking/ingress/
 

== Helm ==

<syntaxhighlight lang="Bash">
wget https://get.helm.sh/helm-v2.16.3-linux-386.tar.gz
tar -xvpf helm-v2.16.3-linux-386.tar.gz
sudo mv linux-386/helm /usr/local/bin/helm
kubectl apply -f rbac-config.yaml
helm init --service-account tiller
source <(helm completion bash) && echo "source <(helm completion bash)" >> ~/.bashrc
</syntaxhighlight>

<syntaxhighlight lang="yaml">
# rbac-config.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
</syntaxhighlight>

Kubernetes (KB) - Revision history

Kbaran: /* Cluster initialisation */

Kbaran at 23:16, 20 October 2021