Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
OpenNESS 19.12 Design

...

Microservices of Openness Infrastructure mode DescriptionDeployment methodDeployment of the componentPropose to integrate
eaaapplication/service registration, authentication etcdeploymentedge nodeyes
edgednsfor client to access microservices in edge nodedaemonset (propose to change to deployment)edge nodeyes
interfaceservicesimilar with providernet implemented in ovn4nfv-k8s-plugindaemonsetedge nodeno, will use ovn4nfv-k8s-plugin's provider network
cnca

Core Network Configuration: Configure the access network (e.g., LTE/CUPS, 5G) control plane

deploymentcontrollerno
sysloglog service for opennessdaemonsetcontroller & edge nodeno
multusenabling attaching multiple network interfaces to podsdaemonsetcontroller & edge nodeAlready covered by ONAP4K8s - KUD
nfdnode feature discoverydaemonsetcontroller & edge nodeAlready covered by ONAP4K8s - KUD
sriovsriov network device plugin & sriov cnidaemonsetcontroller & edge nodeAlready covered by ONAP4K8s - KUD
topology managerkubernetes topology managerKubelet componentcontroller & edge nodeWork in Progress to upgrade the K8s v16.0 integrate into ONAP4K8s - KUD
CMKCPU Managerpart of kubeletcontroller & edge nodeWork in Progress - Integrate into ONAP4K8s - KUD
biosUsed for change BIOS and firmware configuration: CPU configuration, Cache and Memory configuration, PCIe Configuration, Power and Performance configuration, etcprivileged Podcontroller & edge nodeRequired for ICN? Already in ICN Metal3, could be enabled part of it
fpgaOpen Programmable Acceleration Engine (OPAE) package consisting of a kernel driver and user space FPGA utils package that enables programming of the FPGA is used. sriov is used to configure the FPGA resources such as Virtual Functions and queuespodcontroller & edge nodeNeed to integrate into ONAP4K8s - KUD with FPGA device

...

Microservice

ICN

OPENNESS

Difference

Next

MULTUS

  1. Apply the bridge-network.yaml[1].
  2. Create Multus-deployment.yaml[2] with two bridge interfaces.
  3. Exec follow command to check if the “net1” interface was created.
    kubectl exec -it $deployment_pod -- ip a


[1]https://github.com/onap/multicloud-k8s/blob/9c63ce2a7b2b66b3e3fce5d1f553f327148df83f/kud/tests/_common.sh#L856
[2]https://github.com/onap/multicloud-k8s/blob/9c63ce2a7b2b66b3e3fce5d1f553f327148df83f/kud/tests/_common.sh#L873

  1. Apply the macvlan-network.yaml.
  2. Create a pod with macvlan annotation.
  3. Verify the “net1” interface was configured in the deployed pod.

Link:
openness multus usage: https://github.com/open-ness/specs/blob/master/doc/enhanced-platform-awareness/openness-sriov-multiple-interfaces.md#multus-usage 

  • Different network types used for testing. ICN is using the ‘bridge’ type, OPENNESS is ‘macvlan’.
  • Update ICN test case with verifying macvlan network type. 

SRIOV CNI

  1. Apply the sriov-network.yaml[1].
  2. Check if the Ethernet adapter version is equal to "XL710".
  3. Create a pod[2] with the sriov annotation field and the sriov resource requested. 
  4. Verify the the deployed pod status.
    kubectl get pods $pod | awk 'NR==2{print $3}'
  5. Check the current sriov resource allocation status[3].

[1]https://github.com/onap/multicloud-k8s/blob/9c63ce2a7b2b66b3e3fce5d1f553f327148df83f/kud/deployment_infra/playbooks/sriov-nad.yml#L1
[2]https://github.com/onap/multicloud-k8s/blob/9c63ce2a7b2b66b3e3fce5d1f553f327148df83f/kud/tests/sriov.sh#L32
[3]https://github.com/onap/multicloud-k8s/blob/9c63ce2a7b2b66b3e3fce5d1f553f327148df83f/kud/tests/sriov.sh#L68

  1. Apply the sriov-network.yaml
  2. Create a pod with the sriov annotation field and the sriov resource requested. 
  3. Verify the “net1” interface was configured  in the deployed pod.


Link:

Openness sriov usage:

https://github.com/open-ness/specs/blob/master/doc/enhanced-platform-awareness/openness-sriov-multiple-interfaces.md#usage

  • Beside deploying the pod with sriov interface, ICN checks the current allocated sriov resource status. 
  • ICN has a more comprehensive testing and It covers openness test scope. So the ICN test case remains unchanged.

SRIOV NETWORK DEVICE PLUGIN

NFD

Verify NFD by setting pod.yaml with ’affinity’ field.
...
apiVersion: v1
kind: Pod
metadata:
  name: $pod_name
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: "feature.node.kubernetes.io/kernel-version.major"
            operator: Gt
            values:
            - '3'
  containers:
  - name: with-node-affinity
    image: gcr.io/google_containers/pause:2.0
...

Link:

KUD test script:
https://github.com/onap/multicloud-k8s/blob/master/kud/tests/nfd.sh 

Verify NFD by setting pod.yaml with ‘nodeSelector’ field.

apiVersion: v1
kind: Pod
metadata:
  labels:
    env: test
  name: golang-test
spec:
  containers:
    - image: golang
      name: go1
  nodeSelector:
    feature.node.kubernetes.io/cpu-pstate.turbo: 'true'

Link:

Openness nfd usage:
https://github.com/open-ness/specs/blob/master/doc/enhanced-platform-awareness/openness-node-feature-discovery.md#usage

  • Node affinity is conceptually similar to nodeSelector. it allows you to constrain which nodes your pod is eligible to be scheduled on, based on labels on the node.

  • Both tests are roughly the same like each other, ICN specifies ‘affinity’ to check if the NFD is effective, and OPENNESS uses the ‘nodeSelector’ field.
  • Add a check condition for  label ‘feature.node.kubernetes.io/cpu-pstate.turbo: 'true'’.

CMK

NIL



Link:

CMK official validate solution:
https://github.com/intel/CPU-Manager-for-Kubernetes/blob/master/docs/operator.md#validating-the-environment

Liang’s patch: 
https://gerrit.onap.org/r/c/multicloud/k8s/+/102311

  1. Create a pod that can be used to deploy applications pinned to a core.

Link:

Openness CMK usage:
https://github.com/open-ness/specs/blob/master/doc/enhanced-platform-awareness/openness-dedicated-core.md#usage

  • CMK’s integration is under way in ICN. So ICN doesn’t provide a test case now.
  • NIL

Topology Manager

NIL





Link:
Topology Manager limitation:
https://kubernetes.io/docs/tasks/administer-cluster/topology-manager/#known-limitations

  1. Create a pod with guaranteed(requests equal to limits) QoS class. 
  2. Check in kubelet's logs on you node (journalctl -xeu kubelet).

Link:
Openness TM usage:
https://github.com/open-ness/specs/blob/master/doc/enhanced-platform-awareness/openness-topology-manager.md#usage

  • Not implement Topology Manager at ICN. So ICN doesn’t provide test case now.
  • Dependence on k8s version.

Openness dns config agent design

Openness extends kubectl command line to set edgedns (Described in 20318887 part). To integrate Openness with ICN, we will not use this but create a config agent to set edge dns. This config agent will monitor below CRD:

Code Block
languageyml
titleOpenness edge dns CRD
apiVersion: openness.akraino.org/v1alpha1
kind: Opennessdns
metadata:
  name: example-dns
spec:
  node: node1
  dns:
    - record_type: A
      fqdn: openvino.openness
      addresses: 10.16.0.10
    - record_type: A
      fqdn: www.google.com
      addresses: 10.16.0.11

The config agent behavior

...

  • Call edgedns on the specific edge node to set the dns

...

Add more realistic test cases for platform related micro-services

MicroserivceTest cases in KUDTest cases to be add
Multus

SR-IOV CNI


SR-IOV  Network Device Plugin
NFD

CMK

Task List

  • Create Ansible scripts to create building environment, build microservices' docker images and push them to docker repository
  • Create helm charts to run microservice in ONAP4K8s

...