KNI PAE Architecture document

KNI PAE Architecture document

Overall Architecture

  • Fully automated deployment on different platforms: AWS, GCP, baremetal and virtualBaremetal (KVM)

  • Deploy kubernetes cluster properly configured and tuned for NFV/MEC workloads

  • Enablement of real time workloads

  • Possibility of deploying apps on virtual machines and containers in parallel

The blueprint is based on site/profile/base pattern:

  • base - it will contain a set of Kubernetes manifests that define the common settings of the Kubernetes cluster to be deployed, and the common workloads to be applied on top of it

  • profile - specialization of the cluster depending on profiles (libvirt, AWS and baremetal). It will contain specific configs for each platform, and specific workloads to be applied on top of it

  • site - individual definition of a site, based on a chosen profile. It will contain specific configurations for the site (name, domain, network/servers settings, etc...)

Platform Architecture

This blueprint is expected to run on multiple environments (libvirt, AWS and baremetal).

Deployments to AWS

nodes

instance type

nodes

instance type

1x bootstrap (temporary)

EC2: m4.xlarge, EBS: 120GB GP2

3x masters

EC2: m4.xlarge, EBS: 120GB GP2

3x workers

EC2: m4.large, EBS: 120GB GP2

Deployments to Bare Metal

nodes

requirements

nodes

requirements

1x provisioning host (temporary)

12 cores, 16GB RAM, 200GB disk free, 3 NICs (1 internet connectivity, 1 provisioning+storage, 1 cluster)

3x masters

12 cores, 16GB RAM, 200GB disk free, 2 NICs (1 provisioning+storage, 1 cluster)

3x workers

12 cores, min. 16GB RAM, 200GB disk free, 2 SR/IOV-capable NICs (1 provisioning+storage, 1 cluster)

 

The blueprint validation lab uses 7 SuperMicro SuperServer 1028R-WTR (Black) with the following specs:

Units

Type

Description

Units

Type

Description

2

CPU

BDW-EP 12C E5-2650V4 2.2G 30M 9.6GT QPI

8

Mem

16GB DDR4-2400 2RX8 ECC RDIMM

1

SSD

Samsung PM863, 480GB, SATA 6Gb/s, VNAND, 2.5" SSD - MZ7LM480HCHP-00005

4

HDD

Seagate 2.5" 2TB SATA 6Gb/s 7.2K RPM 128M, 512N (Avenger)

2

NIC

Standard LP 40GbE with 2 QSFP ports, Intel XL710

 

Networking for the machines has to be set up as follows:

Deployments to vBaremetal (KVM)

nodes

requirements

nodes

requirements

1x bootstrap (temporary)

2 vCPUs, 2GB RAM, 2GB (sparse)

3x masters

4 vCPUs, 8GB RAM, 2GB (sparse)

3x workers

2 vCPUs, 4GB RAM, 2GB (sparse)

Software Platform Architecture

 

 

deploy on AWS, baremetal, Google Cloud, KVM (libvirt)

Release 2 components:

In Release 2 Ceph support for storage, and  Centos-RT support to enable realtime workloads have been added.

Also the other components have bumped their versions, to use latest code upstream. Following there is a list of components with versions, used in release 2 (colored in orange)

  • CentOS-RT: Linux testing-worker-0 3.10.0-1062.4.3.el7.x86_64 #1 SMP Wed Nov 13 23:58:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

  • CoreOS: Linux testing-worker-1 4.18.0-147.el8.x86_64 #1 SMP Thu Sep 26 15:52:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

  • CRI-O: 1.14.11-0.23.dev.rhaos4.2.gitc41de67.el8

  • Kubernetes (OKD) version: openshift-install v4.2.4, built from commit 425e4ff0037487e32571258640b39f56d5ee5572, release image quay.io/openshift-release-dev/ocp-release@sha256:cebce35c054f1fb066a4dc0a518064945087ac1f3637fe23d2ee2b0c433d6ba8

  • Multus, Cluster/Machine operator, Prometheus: versions provided in 4.2.4 Openshift release

  • Ceph: v14.2.3-20190904

  • Kubevirt: v1alpha3

Release 3 components:

  • CentOS-RT: Linux 3.10.0-1062.4.3.el7.x86_64 #1 SMP Wed Nov 13 23:58:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

  • CoreOS: Red Hat Enterprise Linux CoreOS release 4.3

  • CRI-O: 1.16.3-22.dev.rhaos4.3.git11c04e3.el8

  • Kubernetes (OKD) version: openshift-install v4.3.3, built from commit c7325a3c6045c7f4c8f1ac98d037ffca919be05a, release image quay.io/openshift-release-dev/ocp-release@sha256:9b8708b67dd9b7720cb7ab3ed6d12c394f689cc8927df0e727c76809ab383f44

  • Multus, Cluster/Machine operator, Prometheus: versions provided in 4.3.3 Openshift release

  • Ceph: 14.2.8 (2d095e947a02261ce61424021bb43bd3022d35cb) nautilus (stable)

  • Kubevirt: 0.27.2

Release 4 components:

APIs

No specific APIs involved on this blueprint. It relies on Kubernetes cluster so all the APIs used are Kubernetes ones.

Hardware and Software Management

Licensing

  • Apache license