...
...
...
...
...
...
...
...
Table of Contents |
---|
Introduction
This document describes how to deploy blueprints from Akraino's KNI Blueprint Family. It is common to all blueprints in that family, unless otherwise noted.
...
Pre-Requisites for Deploying to Bare Metal
The baremetal UPI install can be optionally automated when using knictl (see below). When attempting a manual baremetal UPI install, however, please be sure to read: https://docs.openshift.com/container-platform/4.14/installing/installing_bare_metal/installing-bare-metal.html
...
This is minimal configuration example where only 3 servers are used. Servers and their role are given in below table.
Server# | Role | Purpose |
1 | Installer node | This host is used for remotely installing and configuring master and worker node. This server also hosts bootstrap node on KVM-QEMU using libvirt. Several components like- HAProxy, DNS server, DHCP server for provisioning and baremetal network, CoreDNS, Matchbox, Terraform, IPMItool, TFTPboot are configured on this server. Since cluster coreDNS is running from here, this node will be required later as well. |
2 | Master node | This is control plane or master node of K8s cluster that is based on openshift 4.x. |
3 | Worker node | This is worker node which hosts the application. |
4 | Bootstrap node | Bootstrap node runs as VM on installer node and it exists only during the installation and later automatically deleted by installer. |
Other installation requirements
...
Each server should have 3 Ethernet ports configured, purpose of these is listed below. These three are in addition to IPMI port, which is required for PXE boot.
Interface | Purpose |
Management interface | Remote root login from this interface is used for entire setup. This interface needs to have internet connectivity to download various files. This can be shared with external interface. This only needs to be present on the Installer node |
External interface | Interface on the installer node that has internet network connectivity. All external traffic from masters/workers is redirected to the external interface of the installer node. |
Baremetal interface | This interface is for baremetal network, also known as SDN network. This interface doesn’t need internet connectivity. |
Provisioning interface | This interface is for PXE boot. This interface doesn’t need internet connectivity. |
These can be independent NICs or VLANs.
...
Collect IPs and MAC addresses of all the nodes, one sample is given below. This information will be required to populate config files:
Role | iDRAC IP/IPMI port IP | Provisioning network IP | Baremetal network IP | Management network IP | Provisioning network port & mac | Baremetal network port & mac | Management network port & mac |
Installer | xx.xx.xx.xx | xx.xx.xx.xx | xx.xx.xx.xx | xx.xx.xx.xx | em1 / 21:02:0E:DC:BC:27 | em2/ 21:02:0E:DC:BC:28 | em3/ 21:02:0E:DC:BC:29 |
master-0 | |||||||
worker-0 |
Enable IPMI over LAN for all master and worker nodes. This is required for remote PXE boot from installer node. Different servers have different ways to enable it.
...
Bare metal node requirements
Node Role | OS requirement |
Installer | CentOS 7.6 and above |
Bootstrap | RHCOS (Redhat CoreOS) |
Master | RHCOS (Redhat CoreOS) |
Worker | RHCOS/RHEL/CentOS/CentOS-rt |
Pre-Requisites for Deploying to Google Cloud Platform
...
Only one server is needed, that will be acting as a virthost. Master and worker VMs will be created there
Server# | Role | Purpose |
1 | Installer node | This host is used for remotely installing and configuring master and worker node. This server also hosts bootstrap node on KVM-QEMU using libvirt. Several components like- HAProxy, DNS server, DHCP server for provisioning and baremetal network, CoreDNS, Matchbox, Terraform, IPMItool, TFTPboot are configured on this server. Since cluster coreDNS is running from here, this node will be required later as well. |
Network requirements
Network connectivity will be the same as the baremetal case, but these can be dummy interfaces as all the network connectivity will be just inside the same host:
Interface | Purpose |
Management interface | Remote root login from this interface is used for entire setup. This interface needs to have internet connectivity to download various files. This can be shared with external interface. This only needs to be present on the Installer node |
External interface | Interface on the installer node that has internet network connectivity. All external traffic from masters/workers is redirected to the external interface of the installer node. |
Baremetal interface | This interface is for baremetal network, also known as SDN network. This interface doesn’t need internet connectivity. |
Provisioning interface | This interface is for PXE boot. This interface doesn’t need internet connectivity. |
Jump host requirements
Node Role | OS requirement |
Installer | CentOS 7.6 and above |
Installation high level overview
...
.
├── 00_install-config
│ ├── install-config.name.patch.yaml
│ ├── install-config.patch.yaml
│ ├── kustomization.yaml
│ └── site-config.yaml
├── 01_cluster-mods
│ ├── kustomization.yaml
│ ├── manifests
│ └── openshift
├── 02_cluster-addons
│ └── kustomization.yaml
└── 03_services
└── kustomization.yaml
...
Code Block | ||
---|---|---|
| ||
apiVersion: kni.akraino.org/v1alpha1 kind: SiteConfig metadata: name: notImportantHere config: releaseImageOverride: registry.svc.ci.openshift.org/origin/release:4.1 4 |
NOTE: If you are deploying on baremetal, specific configuration needs to be set. This is going to be covered in an specific section for it
...
mkdir -p $GOPATH/src/gerrit.akraino.org/kni
cd $GOPATH/src/gerrit.akraino.org/kni
git clone https://gerrit.akraino.org/r/kni/installer
cd installer
make build
mkdir -p $GOPATH/bin/
cp knictl $GOPATH/bin/cp knictl /usr/local/go/bin/
Secrets
Most secrets (TLS certificates, Kubernetes API keys, etc.) will be auto-generated for you, but you need to provide at least two secrets yourself:
...
This will deploy a cluster based on the specified manifests. You can learn more about how to manage cluster deployment and how to interact with it on https://docs.openshift.com/container-platform/4.14/welcome/index.html
Specific instructions for baremetal are going to be provided later.
...
This will begin to bring up your worker nodes.
You will need to destroy the bootstrap VM once the deploy_workers command is initiated with:
Code Block | ||
---|---|---|
| ||
virsh destroy <bootstrap_vm_name> |
Monitor your worker nodes are you normally would during this process. If the deployment doesn't hit any errors, you will then have a working baremetal cluster. You can monitor the state of the cluster with:
...
https://docs.openshift.com/container-platform/4.14/installing/installing_bare_metal/installing-bare-metal.html#installation-registry-storage-config_installing-bare-metal to fix image registry operator.
Prepare to deploy CentOS nodes
The default installation is totally automated for RHCOS. However, there is the possibility to deploy CentOS nodes, but this requires some specific preparation steps:
...
Mount it:
Code Block | ||
---|---|---|
| ||
mount -o loop /tmp/CentOS-7-x86_64-DVD-1908.iso /mnt/
mkdir -p $HOME/.kni/$SITE_NAME/baremetal_automation/matchbox-data/var/lib/matchbox/assets/centos7
cp -ar /mnt/. $HOME/.kni/$SITE_NAME/baremetal_automation/matchbox-data/var/lib/matchbox/assets/centos7/
umount /mnt |
Prepare a $HOME/settings_upi.env file with the following parameters:
Code Block | ||
---|---|---|
| ||
export CLUSTER_NAME="$CLUSTER_NAME"
export BASE_DOMAIN="$CLUSTER_DOMAIN"
export PULL_SECRET='your_pull_secret'
export KUBECONFIG_PATH=$HOME/.kni/$SITE_NAME/baremetal_automation/ocp/auth/kubeconfig
export OS_INSTALL_ENDPOINT=http://<Installer node provisioning IP>:8080/assets/centos7
export ROOT_PASSWORD="pick_something" |
Navigate to the kickstart script generation and execute it, copying the generated kickstart file:
Code Block | ||
---|---|---|
| ||
cd $HOME/.kni/$SITE_NAME/baremetal_automation/kickstart/
bash add_kickstart_for_centos.sh
cp centos-worker-kickstart.cfg $HOME/.kni/$SITE_NAME/baremetal_automation/matchbox-data/var/lib/matchbox/assets/ |
...
After masters and workers are up, you can apply the workloads using the general procedure with:
Code Block | ||
---|---|---|
| ||
./knictl apply_workloads $SITE_NAME --kubeconfig $HOME/.kni/$SITE_NAME/baremetal_automation/ocp/auth/kubeconfig |
Accessing the Cluster
...
Accessing the Cluster
After the deployment finishes, a kubeconfig
file will be placed inside auth directory:
export KUBECONFIG=$HOME/.kni/$SITE_NAME/final_manifests/auth/kubeconfig
...
You can enter the console with kubeadmin user and the password that is shown at the end of the install.
libvirt deployment guide
...
Create site for virtual baremetal
...
Code Block | ||
---|---|---|
| ||
./knictl apply_workloads $SITE_NAME --kubeconfig $HOME/.kni/$SITE_NAME/baremetal_automation/ocp/auth/kubeconfig |
...
Verifying the
...
setup
After the deployment finishes, a kubeconfig
file will be placed inside auth directory:
...
You can enter the console with kubeadmin user and the password that is shown at the end of the install.
...
Developer guide and troubleshooting
Developer guide -See Developer Documentation
Troubleshooting guide - Please see the upstream documentation for details.
Uninstall guide
Manual
When needed, the site can be destroyed with the openshift-install command, using the following syntax:
...
Code Block | ||
---|---|---|
| ||
./knictl destroy_cluster $SITE_NAME |
Troubleshooting the Cluster
...