Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

...

...

...

...

...

...

...

Table of Contents

Introduction

This document describes how to deploy blueprints from Akraino's KNI Blueprint Family. It is common to all blueprints in that family, unless otherwise noted.

...

Pre-Requisites for Deploying to Bare Metal

The baremetal UPI install can be optionally automated when using knictl (see below).  When attempting a manual baremetal UPI install, however, please be sure to read: https://docs.openshift.com/container-platform/4.14/installing/installing_bare_metal/installing-bare-metal.html

...

This is minimal configuration example where only 3 servers are used. Servers and their role are given in below table.

Server#

Role

Purpose

1

Installer node

This host is used for remotely installing and configuring master and worker node. This server also hosts bootstrap node on KVM-QEMU using libvirt. Several components like- HAProxy, DNS server, DHCP server for provisioning and baremetal network, CoreDNS, Matchbox, Terraform, IPMItool, TFTPboot are configured on this server. Since cluster coreDNS is running from here, this node will be required later as well.

2

Master node

This is control plane or master node of K8s cluster that is based on openshift 4.x.

3

Worker node

This is worker node which hosts the application.

4

Bootstrap node

Bootstrap node runs as VM on installer node and it exists only during the installation and later automatically deleted by installer.

Other installation requirements

...

Each server should have 3 Ethernet ports configured, purpose of these is listed below. These three are in addition to IPMI port, which is required for PXE boot.

Interface

Purpose

Management interface

Remote root login from this interface is used for entire setup. This interface needs to have internet connectivity to download various files. This can be shared with external interface. This only needs to be present on the Installer node

External interfaceInterface on the installer node that has internet network connectivity. All external traffic from masters/workers is redirected to the external interface of the installer node.

Baremetal interface

This interface is for baremetal network, also known as SDN network. This interface doesn’t need internet connectivity.

Provisioning interface

This interface is for PXE boot. This interface doesn’t need internet connectivity.

These can be independent NICs or VLANs.

...

Collect IPs and MAC addresses of all the nodes, one sample is given below. This information will be required to populate config files:

Role

iDRAC IP/IPMI port IP

Provisioning network IP

Baremetal network IP

Management network IP

Provisioning network port & mac

Baremetal network port & mac

Management network port & mac

Installer

xx.xx.xx.xx

xx.xx.xx.xx

xx.xx.xx.xx

xx.xx.xx.xx

em1 / 21:02:0E:DC:BC:27

em2/ 21:02:0E:DC:BC:28

em3/ 21:02:0E:DC:BC:29

master-0








worker-0








Enable IPMI over LAN for all master and worker nodes. This is required for remote PXE boot from installer node. Different servers have different ways to enable it.

...

Bare metal node requirements

Node Role

OS requirement

Installer

CentOS 7.6 and above

Bootstrap

RHCOS (Redhat CoreOS)

Master

RHCOS (Redhat CoreOS)

Worker

RHCOS/RHEL/CentOS/CentOS-rt

Pre-Requisites for Deploying to Google Cloud Platform

...

Only one server is needed, that will be acting as a virthost. Master and worker VMs will be created there

Server#

Role

Purpose

1

Installer node

This host is used for remotely installing and configuring master and worker node. This server also hosts bootstrap node on KVM-QEMU using libvirt. Several components like- HAProxy, DNS server, DHCP server for provisioning and baremetal network, CoreDNS, Matchbox, Terraform, IPMItool, TFTPboot are configured on this server. Since cluster coreDNS is running from here, this node will be required later as well.

Network requirements

Network connectivity will be the same as the baremetal case, but these can be dummy interfaces as all the network connectivity will be just inside the same host:

Interface

Purpose

Management interface

Remote root login from this interface is used for entire setup. This interface needs to have internet connectivity to download various files. This can be shared with external interface. This only needs to be present on the Installer node

External interfaceInterface on the installer node that has internet network connectivity. All external traffic from masters/workers is redirected to the external interface of the installer node.

Baremetal interface

This interface is for baremetal network, also known as SDN network. This interface doesn’t need internet connectivity.

Provisioning interface

This interface is for PXE boot. This interface doesn’t need internet connectivity.

Jump host requirements

Node Role

OS requirement

Installer

CentOS 7.6 and above

Installation high level overview

...

.
├── 00_install-config
│   ├── install-config.name.patch.yaml
│   ├── install-config.patch.yaml
│   ├── kustomization.yaml
│   └── site-config.yaml
├── 01_cluster-mods
│   ├── kustomization.yaml
│   ├── manifests
│   └── openshift
├── 02_cluster-addons
│   └── kustomization.yaml
└── 03_services
└── kustomization.yaml

...

Code Block
languageyml
apiVersion: kni.akraino.org/v1alpha1
kind: SiteConfig
metadata:
 name: notImportantHere
 config:
   releaseImageOverride: registry.svc.ci.openshift.org/origin/release:4.1
4

NOTE: If you are deploying on baremetal, specific configuration needs to be set. This is going to be covered in an specific section for it

...

mkdir -p $GOPATH/src/gerrit.akraino.org/kni
cd $GOPATH/src/gerrit.akraino.org/kni
git clone https://gerrit.akraino.org/r/kni/installer
cd installer
make build
mkdir -p $GOPATH/bin/
cp knictl $GOPATH/bin/

cp knictl /usr/local/go/bin/

Secrets

Most secrets (TLS certificates, Kubernetes API keys, etc.) will be auto-generated for you, but you need to provide at least two secrets yourself:

...

This will deploy a cluster based on the specified manifests. You can learn more about how to manage cluster deployment and how to interact with it on https://docs.openshift.com/container-platform/4.14/welcome/index.html

Specific instructions for baremetal are going to be provided later.

...

This will begin to bring up your worker nodes. 
You will need to destroy the bootstrap VM once the deploy_workers command is initiated with:

Code Block
languagebash
virsh destroy <bootstrap_vm_name>

Monitor your worker nodes are you normally would during this process.  If the deployment doesn't hit any errors, you will then have a working baremetal cluster. You can monitor the state of the cluster with:

...

https://docs.openshift.com/container-platform/4.14/installing/installing_bare_metal/installing-bare-metal.html#installation-registry-storage-config_installing-bare-metal to fix image registry operator.

Prepare to deploy CentOS nodes

The default installation is totally automated for RHCOS. However, there is the possibility to deploy CentOS nodes, but this requires some specific preparation steps:

...

Mount it:

Code Block
languagebash
mount -o loop /tmp/CentOS-7-x86_64-DVD-1908.iso /mnt/
mkdir -p $HOME/.kni/$SITE_NAME/baremetal_automation/matchbox-data/var/lib/matchbox/assets/centos7
cp -ar /mnt/. $HOME/.kni/$SITE_NAME/baremetal_automation/matchbox-data/var/lib/matchbox/assets/centos7/
umount /mnt

Prepare a $HOME/settings_upi.env file with the following parameters:

Code Block
languagebash
export CLUSTER_NAME="$CLUSTER_NAME"
export BASE_DOMAIN="$CLUSTER_DOMAIN"
export PULL_SECRET='your_pull_secret'
export KUBECONFIG_PATH=$HOME/.kni/$SITE_NAME/baremetal_automation/ocp/auth/kubeconfig
export OS_INSTALL_ENDPOINT=http://<Installer node provisioning IP>:8080/assets/centos7
export ROOT_PASSWORD="pick_something" 

Navigate to the kickstart script generation and execute it, copying the generated kickstart file:

Code Block
languagebash
cd $HOME/.kni/$SITE_NAME/baremetal_automation/kickstart/
bash add_kickstart_for_centos.sh
cp centos-worker-kickstart.cfg $HOME/.kni/$SITE_NAME/baremetal_automation/matchbox-data/var/lib/matchbox/assets/

...

After masters and workers are up, you can apply the workloads using the general procedure with:

Code Block
languagebash
./knictl apply_workloads $SITE_NAME --kubeconfig $HOME/.kni/$SITE_NAME/baremetal_automation/ocp/auth/kubeconfig

Accessing the Cluster

...

Accessing the Cluster

After the deployment finishes, a kubeconfig file will be placed inside auth directory:

export KUBECONFIG=$HOME/.kni/$SITE_NAME/final_manifests/auth/kubeconfig

...

You can enter the console with kubeadmin user and the password that is shown at the end of the install.

libvirt deployment guide

...

Create site for virtual baremetal

...

Code Block
languagebash
./knictl apply_workloads $SITE_NAME --kubeconfig $HOME/.kni/$SITE_NAME/baremetal_automation/ocp/auth/kubeconfig

...

Verifying the

...

setup

After the deployment finishes, a kubeconfig file will be placed inside auth directory:

...

You can enter the console with kubeadmin user and the password that is shown at the end of the install.

...

Developer guide and troubleshooting

Developer guide -See Developer Documentation

Troubleshooting guide - Please see the upstream documentation for details.

Uninstall guide

Manual

When needed, the site can be destroyed with the openshift-install command, using the following syntax:

...

Code Block
languagebash
./knictl destroy_cluster $SITE_NAME

Troubleshooting the Cluster

...