How to Install a Kubernetes Cluster on CentOS 8(쿠버네티스 설치)

출처 : https://hiberstack.com/how-to-install-kubernetes-on-centos-8/

        https://www.tecmint.com/install-a-kubernetes-cluster-on-centos-8/

(혹시 에러나거나 안되면 댓글로 말씀해주세요)

Prerequisites

1. Three servers running CentOS 8 – 1 Master Node and 2 Worker Nodes.
2. It is recommended that your nodes should have at least 2 CPUs with 2GB RAM or more per machine. This is not a strict requirement but is largely driven by the needs of the application you intend to run.
3. Internet connectivity on all your nodes. We will be fetching Kubernetes and docker packages from the repository. Equally, you will need to make sure that the DNF package manager is installed by default and can fetch packages remotely.
4. All your nodes should also be able to connect to one another, either on a private or public network, whichever is available.
5. You will also need access to an account with sudo or root privileges. In this tutorial, I will be using my root account.

Logical Architecture

Our installation is designed to have the Master-Node controlling the Worker Nodes. At the end of this installation, our logical architecture will look something like this.

Master Node – This machine generally acts as the control plane and runs the cluster database and the API server (which the kubectl CLI communicates with).

Our 3-node Kubernetes Cluster will look something like this:

Kubernetes Cluster Diagram

 

Installation of Kubernetes Cluster on Master-Node

Step 1: Prepare Hostname, Firewall, and SELinux

### 네트워크 셋팅

sudo hostnamectl set-hostname master-node

 

sudo cat <<EOF>> /etc/hosts

192.168.0.47 master-node

192.168.0.48 node-1 worker-node-1

192.168.0.49 node-2 worker-node-2

EOF

 

ping 192.168.0.48

ping 192.168.0.49

 

sudo cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf

br_netfilter

EOF

 

sudo cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf

net.ipv4.ip_forward = 1

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

EOF

 

modprobe br_netfilter

 

sudo sysctl --system

 

sed ‘s/^SELINUX=enforcing$/SELINUX=permissive/’ /etc/selinux/config

### Container가 host filesystem에 access할 수 있도록 설정

sudo setenforce 0

sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

sudo sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

reboot

### 주로 사용하는 port은 PORT OPEN&방화벽을 열어준다

Control plane

Protocol	Direction	Port Range	Purpose	Used By
TCP	Inbound	6443	Kubernetes API server	All
TCP	Inbound	2379-2380	etcd server client API	kube-apiserver, etcd
TCP	Inbound	10250	Kubelet API	Self, Control plane
TCP	Inbound	10259	kube-scheduler	Self
TCP	Inbound	10257	kube-controller-manager	Self

Worker node(s)

Protocol	Direction	Port Range	Purpose	Used By
TCP	Inbound	10250	Kubelet API	Self, Control plane
TCP	Inbound	30000-32767	NodePort Services†	All

 

Kubernetes Ports

##확인

iptables -L INPUT -vn

firewall-cmd --list-all

 

sudo firewall-cmd --permanent --add-port=6443/tcp

sudo  firewall-cmd --permanent --add-port=2379-2380/tcp

sudo  firewall-cmd --permanent --add-port=10250/tcp

sudo  firewall-cmd --permanent --add-port=10251/tcp

sudo  firewall-cmd --permanent --add-port=10259/tcp

sudo  firewall-cmd --permanent --add-port=10257/tcp

sudo  firewall-cmd --reload

sudo  modprobe br_netfilter

sudo  echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

Step 2: Install Docker-CE on CentOS 8

### Docker repository 등록

dnf : sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo

yum : sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

 

### containerd.io package 설치

dnf : sudo dnf install https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.6-3.3.el7.x86_64.rpm

yum : sudo yum install containerd.io

 

### docker ce 설치

dnf : sudo dnf install docker-ce docker-ce-cli

yum : sudo yum install docker-ce docker-ce-cli 

 

### systemctl 등록

sudo systemctl enable docker

cat > /etc/docker/daemon.json <<EOF

{ 

  "exec-opts": ["native.cgroupdriver=systemd"], 

  "log-driver": "json-file", 

  "log-opts": { 

    "max-size": "100m" 

  }, 

  "storage-driver": "overlay2", 

  "storage-opts": [ 

    "overlay2.override_kernel_check=true" 

  ] 

} 

EOF

 

systemctl restart docker

docker info | grep -i cgroup

 

sudo systemctl start docker

Step 3: Install Kubernetes (Kubeadm) on CentOS 8

### yum repo 등록

 

sudo cat <<EOF > /etc/yum.repos.d/kubernetes.repo

[kubernetes]

name=Kubernetes

baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64

enabled=1

gpgcheck=1

repo_gpgcheck=1

gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

EOF

 

### 설치

dnf : sudo dnf install kubelet kubectl -y

yum : sudo yum install kubelet kubectl -y

### kubelet, kubectl, kubeadm 설치 및 기동(kubeadm 설치하면서 dependency 체크 및 kubelet과 kubectl도 설치함)

### kubeadm : 나중에 ansible과 terraform 연계시에도 자주 사용됨

 

dnf : sudo dnf install kubeadm -y

yum : sudo yum install kubeadm -y

 

sudo systemctl enable --now kubelet

sudo systemctl start kubelet

Step 4: Create a control-plane Master with kubeadm

### Master node 설정위해 kubeadm 초기화

 

sudo swapoff -a

kubeadm init --apiserver-advertise-address=192.168.0.47 --pod-network-cidr=192.168.0.0/16

 

### kubeadm init이 성공적으로 수행된 이후 나오는 명령어는 따로 반드시 저장해두기

(ex) sudo kubeadm join 192.168.0.47:6443 --token nu06lu.xrsux0ss0ixtnms5  \ --discovery-token-ca-cert-hash ha256:f996ea35r4353d342fdea2997a1cf8caeddafd6d4360d606dbc82314683478hjmf7

####(만일  kubeadm init이 실패하면 원인 확인 후 다시 kubeadm init 하면 되는데, 그 자체로 에러날경우 kubeadm reset 수행)      

 

### user config 생성 및 kubectl 명령어 확인

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/admin.config

cd $HOME/.kube

ln -s admin.config config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

 

kubectl get nodes

### Status는 Pod Network를 셋업안했으니 NOT Ready가 맞음

Step 5: Setup Your Pod Network

 

export KUBECONFIG=$HOME/.kube/config

 

### Flannel은 Pod들간 통신을 원활하게 해주는 network plugin 중 하나

kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml

 

kubectl get pods --all-namespaces

 

kubectl cluster-info

Adding Worker Nodes to Kubernetes Cluster

Step 1: Prepare Hostname, Firewall, and SELinux

 

sudo cat <<EOF>> /etc/hosts

192.168.0.47 master-node

192.168.0.48 node-1 worker-node-1

192.168.0.49 node-2 worker-node-2

EOF

ping 192.168.0.47

ping 192.168.0.48

ping 192.168.0.49

 

sudo cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf

br_netfilter

EOF

 

setenforce 0

sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

firewall-cmd --permanent --add-port=6783/tcp

firewall-cmd --permanent --add-port=10250/tcp

firewall-cmd --permanent --add-port=10255/tcp

firewall-cmd --permanent --add-port=30000-32767/tcp

firewall-cmd --reload

echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

 

Step 2: Setup Docker-CE and Kubernetes Repo

 

### Docker repository 등록

dnf : sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo

yum : sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

 

### containerd.io package 설치

dnf : sudo dnf install https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.6-3.3.el7.x86_64.rpm

yum : sudo yum install containerd.io

 

### docker ce 설치

dnf : sudo dnf install docker-ce docker-ce-cli

yum : sudo yum install docker-ce docker-ce-cli 

 

### systemctl 등록

sudo systemctl enable docker

cat > /etc/docker/daemon.json <<EOF

{ 

  "exec-opts": ["native.cgroupdriver=systemd"], 

  "log-driver": "json-file", 

  "log-opts": { 

    "max-size": "100m" 

  }, 

  "storage-driver": "overlay2", 

  "storage-opts": [ 

    "overlay2.override_kernel_check=true" 

  ] 

} 

EOF

 

systemctl restart docker

docker info | grep -i cgroup

 

sudo systemctl start docker

Step 3: Install Kubernetes (Kubeadm) on CentOS 8

### yum repo 등록

 

sudo cat <<EOF > /etc/yum.repos.d/kubernetes.repo

[kubernetes]

name=Kubernetes

baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64

enabled=1

gpgcheck=1

repo_gpgcheck=1

gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

EOF

 

### 설치

dnf : sudo dnf install kubelet kubectl -y

yum : sudo yum install kubelet kubectl -y

### kubelet, kubectl, kubeadm 설치 및 기동(kubeadm 설치하면서 dependency 체크 및 kubelet과 kubectl도 설치함)

### kubeadm : 나중에 ansible과 terraform 연계시에도 자주 사용됨

 

dnf : sudo dnf install kubeadm -y

yum : sudo yum install kubeadm -y

 

sudo systemctl enable --now kubelet

sudo systemctl start kubelet

 

Step 4: Join the Worker Node to the Kubernetes Cluster

## kubeadm join (만일 잃어버렸을경우, master node에서 명령어 수행 : kubeadm token create --print-join-command)

kubeadm join 192.168.0.47:6443 --token nu06lu.xrsux0ss0ixtnms5  --discovery-token-ca-cert-hash sha256:f996ea35r4353d342fdea2997a1cf8caeddafd6d4360d606dbc82314683478hjmf78

 

## master node에서 수행

kubectl get nodes