Procedure for setup of ELIOT - Virtual Environment using ThinOS:
Pre-requisites
Hardware Level:
x86_64 / arm64 hardware
Software Level:
Ubuntu OS running as VM / Bare Metal
Ubuntu OS version should be 16.04 and above
From Deployment script,
| Panel |
|---|
$ git clone https://gerrit.akraino.org/r/eliot |
Create a folder in $HOME , then do above step
Configurations in eliot_setup.sh
Modify the upcoming variables in eliot_setup.sh according to you ELIOT edge node specifications
- edgenode_password This should be password of where you want to setup your Edge node
- edgenode_username This should be username of where you want to setup your Edge node
- edgenode_ip This should be IP address of where you want to setup your Edge node
Configurations in thin_os_setup.sh
tap value can be tap0 or tap1 ,etc.
Check your ifconfig first and ensure that your tap name is not existed before.
edge_node_ip_pattern - This has to be 'inet <your_edge_node_ip_address>'
Run the script eliot_setup.sh only.
In this process, ELIOT Manager will setup the ELIOT edge node and also provides the overall setup for ELIOT
| Note |
|---|
No need to explicitly invoke the thin_os_setup.sh. In the eliot_setup.sh, thin_os_setup.sh will be invoked automatically. |
After Thinos vm(edge node vm) is installed,
Give login credentials the thinos vm created in edge node
login name as root
Password as root
Configuring the networking for ThinOS VM:
| Panel |
|---|
$ ifconfig eth0 192.168.100.100 |
| Note |
|---|
192.168.100.100 - Private IP address for ThinOS VM |
| Panel |
|---|
$ route add default gw <tap_interface_address> |
| Note |
|---|
This should be the same address which is used to assign the tap interface in edge node on the thin_os_setup.sh in the following line which is executed already. Example in our case, it is $ ifconfig tap0 192.168.100.1 Currently supported Platforms:The deployment script for ELIOT edge node and ELIOT Master node setup is currently suitable for the x86_64 and arm64 machines Common errors to neglect:The following errors may occur after downloading disk.qcow2 - thinos image file from public server RTNETLINK answers: Operation not permitted |
Troubleshooting:
If eliot_setup.sh failed due to some error, while installing/downloading the lightweight os, you have to do the following in ELIOT edge node.
Delete the newly created tap interface if it is created
If your created tap interface name is tap0,
| Panel |
|---|
$ sudo ip link del tap0 |
Remove the created disk.qcow2 file
| Panel |
|---|
$ cd eliot/src $ sudo rm disk.qcow2 |
Kill your running thin_os_setup.sh process by finding pid of it
| Panel |
|---|
$ ps -eaf | grep thin_os $ kill -9 <pid> |
Counter-measures for some errors which might occur while Installing:
If you encounter "Bad argument MASQUERADE" error while running script, then do the following
| Panel |
|---|
$ cd src $ sudo rm nic.txt ( or ) $ cp /dev/null nic.txt |
ELIOT Deployments in Virtual Environments Overview
This document provides a general approach to set up ELIOT ecosystem in virtual environments. The virtual environment taken are X86_64 , ARM64 and ARM32 servers.
X86 Servers with CentOS - 7 and ARM - 64 with Ubuntu 17.10 versions are chosen for setup.
ELIOT Deployment in X86_64/AMD64 Server with CentOS - 7.5 version.
Pre-Installation steps to be executed on ELIOT Manager and ELIOT Node
Disable SELinux:
| Panel |
|---|
$ setenforce 0 $ sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux |
Disable swap:
| Panel |
|---|
$ swapoff -a Note : To make sure on reboot of server the swap isn't re-enabled, comment the swap line UUID in /etc/fstab file. $ vi /etc/fstab # /dev/mapper/centos-swap swap swap defaults 0 0 |
Enable br_netfilter:
| Panel |
|---|
| $ modprobe br_netfilter $ echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables |
Install Docker:
Set up the repository
Install the required packages.
| Panel |
|---|
| $ sudo yum install -y yum-utils \ device-mapper-persistent-data \ lvm2 |
Set up stable repository
| Panel |
|---|
| $ sudo yum-config-manager \ -add-repo \ https://download.docker.com/linux/centos/docker-ce.repo |
Install Docker CE
To install latest execute step a or else if specific version has to be installed execute step b.
a) Install the latest version of Docker CE and containerd
| Panel |
|---|
| $ sudo yum install docker-ce docker-ce-cli containerd.io |
b) List and sort specific available in your repo.
| Panel |
|---|
| $ yum install docker-ce --showduplicates | sort -r $ sudo yum install docker-ce-<VERSION_STRING> docker-ce-cli-<VERSION_STRING> containerd.io Example : $ yum list docker-ce --showduplicates | sort -r docker-ce.x86_64 3:18.09.1-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.0-3.el7 docker-ce-stable docker-ce.x86_64 18.06.1.ce-3.el7 docker-ce-stable $ sudo yum install docker-ce-18.09.1 docker-ce-cli-18.09.1 containerd.io |
Start Docker
| Panel |
|---|
| $ sudo systemctl start docker |
Install Kubernetes
Add kubernetes repository in Cent OS system
| Panel |
|---|
| $ cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOF |
Install kubernetes packages kubeadm, kubelet and kubectl.
| Panel |
|---|
$ yum install -y kubelet kubeadm kubectl |
After installation is complete restart the ELIOT Manager and ELIOT Node
| Panel |
|---|
| $ sudo reboot |
Login to the servers and start the services, docker and kubelet
| Panel |
|---|
| $ systemctl start docker && systemctl enable docker $ systemctl start kubelet && systemctl enable kubelet |
Change the cgroup-driver (Need to make sure docker-ce and kubernetes are using same cgroup)
Check docker cgroup
| Panel |
|---|
| $ docker info | grep -i cgroup |
It will display docker is using 'cgroupfs' as a cgroup-driver
Run the below command to change the kubernetes cgroup-driver to 'cgroupfs' and Reload the systemd system and restart the kubelet service
| Panel |
|---|
| $ sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf $ systemctl daemon-reload $ systemctl restart kubelet |
Kubernetes Cluster Initialization
Login to ELIOT Manager Server ; Initialize the Kubernetes Master
| Panel |
|---|
| $ kubeadm init --apiserver-advertise-address=<ELIOT Manager Server IP Address> --pod-network-cidr=10.244.0.0/16 |
| Note |
|---|
Note : --apiserver-advertise-address = determines which IP Address Kubernetes should advertise its API server on. |
To start using your cluster, you need to run (as a regular user)
| Panel |
|---|
| $ mkdir -p $HOME/.kube $ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config $ sudo chown $(id -u):$(id -g) $HOME/.kube/config or (for root user) $ export KUBECONFIG=/etc/kubernetes/admin.conf |
Adding ELIOT Node to the Cluster
Execute the command in the ELIOT Node
| Panel |
|---|
| $ kubeadm join --token <token> <master-ip>:6443 --discovery-token-ca-cert-hash sha256:<hash> [The Kuberadm join command string is displayed after successfull installation of Kubernetes Master (ELIOT Manager)] |
Deploy Flannel network in ELIOT Manager (Kubernetes Cluster)
| Panel |
|---|
| $ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml |
After join command is executed successfully , you could check the cluster node and pods
| Panel |
|---|
| $ kubectl get nodes $ kubectl get pods --all-namespaces |
...
ELIOT Deployments in Virtual Environments Overview
This document provides a general approach to set up ELIOT ecosystem in virtual environments. The virtual environment taken are X86_64 , ARM64 and ARM32 servers.
X86 Servers with CentOS - 7 and ARM - 64 with Ubuntu 17.10 versions are chosen for setup.
ELIOT support Kubernetes and kubeedge orchestrator based two deployment model.
ELIOT Kubernetes based setup:
ELIOT Deployment in X86_64/AMD64 Server with CentOS - 7.5 version.
Pre-Installation steps to be executed on ELIOT Manager and ELIOT Node
Disable SELinux:
| Panel |
|---|
$ setenforce 0 $ sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux |
Disable swap:
| Panel |
|---|
$ swapoff -a Note : To make sure on reboot of server the swap isn't re-enabled, comment the swap line UUID in /etc/fstab file. $ vi /etc/fstab # /dev/mapper/centos-swap swap swap defaults 0 0 |
Enable br_netfilter:
| Panel |
|---|
| $ modprobe br_netfilter $ echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables |
Install Docker:
Set up the repository
Install the required packages.
| Panel |
|---|
| $ sudo yum install -y yum-utils \ device-mapper-persistent-data \ lvm2 |
Set up stable repository
| Panel |
|---|
| $ sudo yum-config-manager \ -add-repo \ https://download.docker.com/linux/centos/docker-ce.repo |
Install Docker CE
To install latest execute step a or else if specific version has to be installed execute step b.
a) Install the latest version of Docker CE and containerd
| Panel |
|---|
| $ sudo yum install docker-ce docker-ce-cli containerd.io |
b) List and sort specific available in your repo.
| Panel |
|---|
| $ yum install docker-ce --showduplicates | sort -r $ sudo yum install docker-ce-<VERSION_STRING> docker-ce-cli-<VERSION_STRING> containerd.io Example : $ yum list docker-ce --showduplicates | sort -r docker-ce.x86_64 3:18.09.1-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.0-3.el7 docker-ce-stable docker-ce.x86_64 18.06.1.ce-3.el7 docker-ce-stable $ sudo yum install docker-ce-18.09.1 docker-ce-cli-18.09.1 containerd.io |
Start Docker
| Panel |
|---|
| $ sudo systemctl start docker |
Install Kubernetes
Add kubernetes repository in Cent OS system
| Panel |
|---|
| $ cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOF |
Install kubernetes packages kubeadm, kubelet and kubectl.
| Panel |
|---|
$ yum install -y kubelet kubeadm kubectl |
After installation is complete restart the ELIOT Manager and ELIOT Node
| Panel |
|---|
| $ sudo reboot |
Login to the servers and start the services, docker and kubelet
| Panel |
|---|
| $ systemctl start docker && systemctl enable docker $ systemctl start kubelet && systemctl enable kubelet |
Change the cgroup-driver (Need to make sure docker-ce and kubernetes are using same cgroup)
Check docker cgroup
| Panel |
|---|
| $ docker info | grep -i cgroup |
It will display docker is using 'cgroupfs' as a cgroup-driver
Run the below command to change the kubernetes cgroup-driver to 'cgroupfs' and Reload the systemd system and restart the kubelet service
| Panel |
|---|
| $ sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf $ systemctl daemon-reload $ systemctl restart kubelet |
Kubernetes Cluster Initialization
Login to ELIOT Manager Server ; Initialize the Kubernetes Master
| Panel |
|---|
| $ kubeadm init --apiserver-advertise-address=<ELIOT Manager Server IP Address> --pod-network-cidr=10.244.0.0/16 |
| Note |
|---|
Note : --apiserver-advertise-address = determines which IP Address Kubernetes should advertise its API server on. |
To start using your cluster, you need to run (as a regular user)
| Panel |
|---|
| $ mkdir -p $HOME/.kube $ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config $ sudo chown $(id -u):$(id -g) $HOME/.kube/config or (for root user) $ export KUBECONFIG=/etc/kubernetes/admin.conf |
Adding ELIOT Node to the Cluster
Execute the command in the ELIOT Node
| Panel |
|---|
| $ kubeadm join --token <token> <master-ip>:6443 --discovery-token-ca-cert-hash sha256:<hash> [The Kuberadm join command string is displayed after successfull installation of Kubernetes Master (ELIOT Manager)] |
Deploy Flannel network in ELIOT Manager (Kubernetes Cluster)
| Panel |
|---|
| $ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml |
After join command is executed successfully , you could check the cluster node and pods
| Panel |
|---|
| $ kubectl get nodes $ kubectl get pods --all-namespaces |
| Note |
|---|
| Note : You will get the 'k8s-master' node is running as a 'master' cluster with status 'ready', and you will get all pods that are needed for the cluster, including the 'kube-flannel-ds' for network pod configuration Make sure all kube-system pod status is running. |
ELIOT KubeEdge based setup:
Eliot support Light weight orchestrator KubeEdge.
For more details refer
https://wiki.akraino.org/display/AK/KubeEdge
Kube Edge Building from source
| Panel |
|---|
|
Binary keadm is available in current path
Installing KubeEdge Master Node (on the Cloud) component
| Panel |
|---|
keadm init --docker-version=<expected version> --kubernetes-version=<expected version> --kubeedge-version=<expected version> Flags:
--docker-version string[="18.06.0"] Use this key to download and use the required Docker version (default "18.06.0")
-h, --help help for init
--kubeedge-version string[="0.3.0-beta.0"] Use this key to download and use the required KubeEdge version (default "0.3.0-beta.0")
--kubernetes-version string[="1.14.1"] Use this key to download and use the required Kubernetes version (default "1.14.1")
|
Installing KubeEdge Worker Node (at the Edge) component
| Panel |
|---|
keadm join --edgecontrollerip=<ip address> --edgenodeid=<unique string as edge identifier> Flags:
--docker-version string[="18.06.0"] Use this key to download and use the required Docker version (default "18.06.0")
-e, --edgecontrollerip string IP address of KubeEdge edgecontroller
-i, --edgenodeid string KubeEdge Node unique identification string, If flag not used then the command will generate a unique id on its own
-h, --help help for join
-k, --k8sserverip string IP:Port address of K8S API-Server
--kubeedge-version string[="0.3.0-beta.0"] Use this key to download and use the required KubeEdge version (default "0.3.0-beta.0") |
Reset KubeEdge Master and Worker nodes
| Panel |
|---|
Flags: -h, --help help for reset -k, --k8sserverip string IP:Port address of cloud components host/VM |
For more details, please follow below link:
https://kubeedge.readthedocs.io/en/latest/setup/installer_setup.html
ELIOT VM setup on ARM64
This section provides steps about setting the Virtual Machine with Ubuntu OS on ARM64 servers
Environment Details :
...