Overview
The master instance is the main instance which controls the applications or the containers on the cluster. Don't forget, in our case, kubernetes consists of at least 1 master and 2 node. In total 2 machines which can run the application.
So let's initialize the cluster from the master instance:
Initiliaze the cluster
To initialize the cluster, we have to take two factors into consideration:
- Which will be the advertise IP ?
- Which will be the network which we will use for the pods.
The first question is pretty easy. Just use the network which is assigned to your master. In our case, we have 1 master and 2 noides. So we will assign the advertise IP of the master:
- master - 192.168.50.10
- node1 - 192.168.50.11
- node2 - 192.168.50.12
The second question however, depends on the network which will be used for the pods. In our example I have used calico, because of the reasons listed below.Thus, our pod network by default is: 192.168.0.0/16.
So let's see how our commands
Initialize the cluster
root@k8s-master:~# kubeadm init --ignore-preflight-errors=NumCPU --apiserver-advertise-address=192.168.50.10 --pod-network-cidr=192.168.0.0/16
W0421 09:20:50.597038 21388 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.18.2
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
**************************************************************************************************************************
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.50.10:6443 --token k7cnjt.c0vkn3i6sc9qp2it \
--discovery-token-ca-cert-hash sha256:8c7874be67b9670c52a729b7a26bdefb4b55f5a49402624c0d262c0253732228
root@k8s-master:~#
After that, we have to perform a couple commands from the user, which will be responsible for the kubernetes and won't be root. (P.S. usage of root for applications is STRONGLY DISCOURAGED because of security stuff :) )
So just transfer it using the instructions above:
Execute as normal User
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
Once we have done that, we can check the cluster:
Check the cluster
ubuntu@k8s-master:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master NotReady master 62s v1.18.2
Now, you can see that the cluster is saying that the cluster isn't Ready. But what that means, let's see which part isn't ready:
Check cluster components
ubuntu@k8s-master:~$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-66bff467f8-rgh8d 0/1 Pending 0 62s kube-system coredns-66bff467f8-tql72 0/1 Pending 0 62s kube-system etcd-k8s-master 1/1 Running 0 72s kube-system kube-apiserver-k8s-master 1/1 Running 0 72s kube-system kube-controller-manager-k8s-master 1/1 Running 0 72s kube-system kube-proxy-jkmql 1/1 Running 0 62s kube-system kube-scheduler-k8s-master 1/1 Running 0 72s </Code From this we can see that the CoreDNS isn't ready, meaning our network isn't applied from the steps above: <Code:none|Missed steps> Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/
Configure Calico Pod Network
So which podnetwork, we will use. As already mentioned if you are using Kubernetes >1.16, then you cannot use the weave network. Because of that I had to use Calico:
Apply Calico Pod network
ubuntu@k8s-master:~$ kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml configmap/calico-config created customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created clusterrole.rbac.authorization.k8s.io/calico-node created clusterrolebinding.rbac.authorization.k8s.io/calico-node created daemonset.apps/calico-node created serviceaccount/calico-node created deployment.apps/calico-kube-controllers created serviceaccount/calico-kube-controllers created ubuntu@k8s-master:~$
After that we can check the components again:
Check cluster components
ubuntu@k8s-master:~$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-77c5fc8d7f-88lsl 0/1 Pending 0 33s kube-system calico-node-bqw8q 0/1 Init:0/3 0 33s kube-system coredns-66bff467f8-rgh8d 0/1 Pending 0 114s kube-system coredns-66bff467f8-tql72 0/1 Pending 0 114s kube-system etcd-k8s-master 1/1 Running 0 2m4s kube-system kube-apiserver-k8s-master 1/1 Running 0 2m4s kube-system kube-controller-manager-k8s-master 1/1 Running 0 2m4s kube-system kube-proxy-jkmql 1/1 Running 0 114s kube-system kube-scheduler-k8s-master 1/1 Running 0 2m4s
We see they are being Initialized as well: “Init:0/3”, so give them little time to start up. During that time the machine can be very slow, so have little patience. In the end you will see something like that:
Check cluster components
ubuntu@k8s-master:~/.kube$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready master 56m v1.18.2 ubuntu@k8s-master:~/.kube$ ubuntu@k8s-master:~/.kube$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-77c5fc8d7f-88lsl 1/1 Running 0 54m kube-system calico-node-bqw8q 1/1 Running 0 54m kube-system coredns-66bff467f8-rgh8d 1/1 Running 0 55m kube-system coredns-66bff467f8-tql72 1/1 Running 0 55m kube-system etcd-k8s-master 1/1 Running 0 55m kube-system kube-apiserver-k8s-master 1/1 Running 0 55m kube-system kube-controller-manager-k8s-master 1/1 Running 2 55m kube-system kube-proxy-jkmql 1/1 Running 0 55m kube-system kube-scheduler-k8s-master 1/1 Running 2 55m
That concludes the initialization of the cluster. In the next section we will discuss how to add new nodes :)
Joint to the cluster
Be sure that you installed the necessary packages from the introduction section. Once this is done we can add the node to the cluster as follow:
Add node
root@node-1:~# kubeadm join 192.168.50.10:6443 --token k7cnjt.c0vkn3i6sc9qp2it --discovery-token-ca-cert-hash sha256:8c7874be67b9670c52a729b7a26bdefb4b55f5a49402624c0d262c0253732228
W0421 10:28:13.551137 21280 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
root@node-1:~#
As with the master node, it might take sometime until you see the node as Ready and all components running from the Control Panel Machine:
Check the newly added Node
ubuntu@k8s-master:~/.kube$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready master 67m v1.18.2 node-1 Ready <none> 82s v1.18.2 ubuntu@k8s-master:~/.kube$ ubuntu@k8s-master:~/.kube$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-77c5fc8d7f-88lsl 1/1 Running 0 65m kube-system calico-node-bqw8q 1/1 Running 0 65m kube-system calico-node-wwfc5 0/1 Running 0 75s kube-system coredns-66bff467f8-rgh8d 1/1 Running 0 67m kube-system coredns-66bff467f8-tql72 1/1 Running 0 67m kube-system etcd-k8s-master 1/1 Running 0 67m kube-system kube-apiserver-k8s-master 1/1 Running 0 67m kube-system kube-controller-manager-k8s-master 1/1 Running 2 67m kube-system kube-proxy-hnmxb 1/1 Running 0 75s kube-system kube-proxy-jkmql 1/1 Running 0 67m kube-system kube-scheduler-k8s-master 1/1 Running 2 67m
Please execute that step on all nodes. In the end you should have something like this:
Check the newly added Node
ubuntu@k8s-master:~/.kube$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready master 77m v1.18.2 node-1 Ready <none> 11m v1.18.2 node-2 Ready <none> 88s v1.18.2 ubuntu@k8s-master:~/.kube$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-77c5fc8d7f-88lsl 1/1 Running 0 75m kube-system calico-node-bqw8q 1/1 Running 0 75m kube-system calico-node-fl6ft 1/1 Running 0 84s kube-system calico-node-wwfc5 1/1 Running 0 11m kube-system coredns-66bff467f8-rgh8d 1/1 Running 0 77m kube-system coredns-66bff467f8-tql72 1/1 Running 0 77m kube-system etcd-k8s-master 1/1 Running 0 77m kube-system kube-apiserver-k8s-master 1/1 Running 0 77m kube-system kube-controller-manager-k8s-master 1/1 Running 2 77m kube-system kube-proxy-hnmxb 1/1 Running 0 11m kube-system kube-proxy-jkmql 1/1 Running 0 77m kube-system kube-proxy-s4nrh 1/1 Running 0 84s kube-system kube-scheduler-k8s-master 1/1 Running 2 77m
Assign role to a Node
You saw that our nodes have no roles. We have 1 master and that is that :)
Nodes' roles
ubuntu@k8s-master:~/.kube$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready master 77m v1.18.2 node-1 Ready <none> 11m v1.18.2 node-2 Ready <none> 88s v1.18.2 </Code So, how to assign roles to the node. Well, in Kubernetes, we assign labels. Labels are assigned as follows: <Code:none|Assign label> kubectl label node <node name> node-role.kubernetes.io/<role name>=<key - (any name)> - To assign the label kubectl label node <node name> node-role.kubernetes.io/<role name> - To remove the label
So let's assign worker to our node-1 and node-2
Assign Labels to Node-1 and Node-2
ubuntu@k8s-master:~/.kube$ kubectl label node node-1 node-role.kubernetes.io/worker=worker node/node-1 labeled ubuntu@k8s-master:~/.kube$ kubectl label node node-2 node-role.kubernetes.io/worker=worker node/node-2 labeled ubuntu@k8s-master:~/.kube$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready master 83m v1.18.2 node-1 Ready worker 17m v1.18.2 node-2 Ready worker 7m39s v1.18.2 ubuntu@k8s-master:~/.kube$
Alternatively we can remove a label from a node. So let's remove and add that label again on Node-2:
Remove and Add Label on Node-2
ubuntu@k8s-master:~/.kube$ kubectl label node node-2 node-role.kubernetes.io/worker- node/node-2 labeled ubuntu@k8s-master:~/.kube$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready master 86m v1.18.2 node-1 Ready worker 20m v1.18.2 node-2 Ready <none> 10m v1.18.2 ubuntu@k8s-master:~/.kube$ kubectl label node node-2 node-role.kubernetes.io/worker=worker node/node-2 labeled ubuntu@k8s-master:~/.kube$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready master 87m v1.18.2 node-1 Ready worker 20m v1.18.2 node-2 Ready worker 11m v1.18.2 ubuntu@k8s-master:~/.kube$