ELIOT Deployments in Virtual Environments Overview

This document provides a general approach to set up ELIOT ecosystem in virtual environments. The virtual environment taken are X86_64 , ARM64 and ARM32 servers.

X86 Servers with CentOS - 7 and ARM - 64 with Ubuntu 17.10 versions are chosen for setup.

ELIOT Deployment in X86_64/AMD64 Server with CentOS - 7.5 version.

Pre-Installation steps to be executed on ELIOT Manager and ELIOT Node

Disable SELinux:

$ setenforce 0

$ sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

Disable swap:

$ swapoff -a

Note : To make sure on reboot of server the swap isn't re-enabled, comment the swap line UUID in /etc/fstab file.

$ vi /etc/fstab

# /dev/mapper/centos-swap swap swap defaults 0 0

Enable br_netfilter:

$ modprobe br_netfilter
$ echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

Install Docker:

Set up the repository
Install the required packages.

          $ sudo yum install -y yum-utils \
              device-mapper-persistent-data \
          lvm2

Set up stable repository

       $ sudo yum-config-manager \
       -add-repo \
        https://download.docker.com/linux/centos/docker-ce.repo

Install Docker CE

To install latest execute step a or else if specific version has to be installed execute step b.

a) Install the latest version of Docker CE and containerd

$ sudo yum install docker-ce docker-ce-cli containerd.io

b) List and sort specific available in your repo.

            $ yum install docker-ce --showduplicates | sort -r
            $ sudo yum install docker-ce-<VERSION_STRING> docker-ce-cli-<VERSION_STRING> containerd.io

            Example :
            $ yum list docker-ce --showduplicates | sort -r
            docker-ce.x86_64 3:18.09.1-3.el7                     docker-ce-stable
            docker-ce.x86_64 3:18.09.0-3.el7                     docker-ce-stable
            docker-ce.x86_64 18.06.1.ce-3.el7                    docker-ce-stable

         $ sudo yum install docker-ce-18.09.1 docker-ce-cli-18.09.1 containerd.io

Start Docker

$ sudo systemctl start docker

Install Kubernetes

Add kubernetes repository in Cent OS system

        $ cat <<EOF > /etc/yum.repos.d/kubernetes.repo
        [kubernetes]
        name=Kubernetes
        baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
        enabled=1
        gpgcheck=1
        repo_gpgcheck=1
        gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
               https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
        EOF

Install kubernetes packages kubeadm, kubelet and kubectl.

$ yum install -y kubelet kubeadm kubectl

After installation is complete restart the ELIOT Manager and ELIOT Node

$ sudo reboot

Login to the servers and start the services, docker and kubelet

$ systemctl start docker && systemctl enable docker
$ systemctl start kubelet && systemctl enable kubelet

Change the cgroup-driver (Need to make sure docker-ce and kubernetes are using same cgroup)

Check docker cgroup

$ docker info | grep -i cgroup

It will display docker is using 'cgroupfs' as a cgroup-driver
Run the below command to change the kubernetes cgroup-driver to 'cgroupfs' and Reload the systemd system and restart the kubelet service

    $ sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

    $ systemctl daemon-reload
    $ systemctl restart kubelet

Kubernetes Cluster Initialization

Login to ELIOT Manager Server ; Initialize the Kubernetes Master

$ kubeadm init --apiserver-advertise-address=<ELIOT Manager Server IP Address> --pod-network-cidr=10.244.0.0/16

Note :

--apiserver-advertise-address = determines which IP Address Kubernetes should advertise its API server on.
--pod-network-cidr=10.244.0.0/16 - Pod network range. Pod network must not overlap with any of the host networks as this can cause issues.
If you find a collision between your network plugin?s preferred Pod network and some of your host networks, you should think of a suitable CIDR replacement and use that during kubeadm init with --pod-network-cidr and as a replacement in your network plugin?s YAML.

To start using your cluster, you need to run (as a regular user)

             $ mkdir -p $HOME/.kube
           $ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
           $ sudo chown $(id -u):$(id -g) $HOME/.kube/config
                            or
            (for root user)
             $ export KUBECONFIG=/etc/kubernetes/admin.conf

Adding ELIOT Node to the Cluster

Execute the command in the ELIOT Node

        $ kubeadm join --token <token> <master-ip>:6443 --discovery-token-ca-cert-hash sha256:<hash>

       [The Kuberadm join command string is displayed after successfull installation of Kubernetes Master (ELIOT Manager)]

Deploy Flannel network in ELIOT Manager (Kubernetes Cluster)

$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml

After join command is executed successfully , you could check the cluster node and pods

$ kubectl get nodes
$ kubectl get pods --all-namespaces

Note : You will get the 'k8s-master' node is running as a 'master' cluster with status 'ready', and you will get all pods that are needed for the cluster, including the 'kube-flannel-ds' for network pod configuration
Make sure all kube-system pod status is running.

ELIOT Blueprints Installation Overview