Table of Contents

Goal:

This wiki describes the specifications for integration of Baremetal operator required for the Integrated Cloud Native Akraino project. 

Overview of Baremetal Provisioning:

ICN architecture has a bootstrap cluster in all the edge location, this k8s cluster is used to do provisioning of compute nodes in the edge location.

...

Bootstrap cluster will be in the jump machine, it has 3 interfaces, eno1 interface NIC support IPMI, eno2 for provisioning and eht0 for the bare-metal networking. 

Baremetal operator:

ICN stack uses metal3 baremetal operator to do node provisioning in the bootstrap cluster. Baremetal operator runs as deployments in bootstrap cluster, gets the OS image details and baremetal server ipmi details in the edge location to do the provisioning. Baremetal operator uses Ironic as a provisioning agent. 

...

• Baremeta host Custom resource definition(CRD)
• Baremetal host CRD controller
• Ironic
• Ironic Inspector
• Ironic internal DHCP server

Baremetal Host Custom Resource Definition(CRD)

The baremetal operator abstract the baremetal server hardware features and store the hardware profile details in the baremetal host. It hold key information such as CPU information, NIC, FPGA, QAT card and disk details, Baremetalhost CR act as template by a user to send the ipmi username and password encode as k8s secret to the Baremetal operator. And a refer to that K8s secret is refered as CredentialName in Baremetal operator API. Baremetal API defines various baremetal server details that are required to manage and provision the server.

...

---
apiVersion: v1
kind: Secret
metadata:
  name: demo-bmc-secret
type: Opaque
data:
  username: cnllbGVzd2E=
  password: Y2hhbmdlbWUx

---
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
  name: demo
spec:
  online: true
  bmc:
    address: ipmi://172.31.1.17
    credentialsName: demo-bmc-secret
  image:
    url: "http://172.22.0.1/images/bionic-server-cloudimg-amd64.img"
    checksum: "http://172.22.0.1/images/bionic-server-cloudimg-amd64.md5sum"
  userData:
    name: demo-user-data
    namespace: metal3

Baremetal host CRD controller

Baremetal host controller is CRD implementation that list and watch for the creation of the BMH CR in the booststrap cluster. Once the CR is created or applied with patches, this event triggers the CRD controller and invoke the ironic with ipmi address, image and userdata. In order to run the baremetal CRD controller to communicate with Ironic, user has pass down following information.

...

Baremetal CRD controller basically act as a client with Ironic endpoint and Ironic inspector endpoint to send the crd information and to retrieve the hardware details from the Ironic inspector to store the details in the etcd controller

Ironic

Ironic as standalone open source has a lot of capability to control a remote BMC in a server. In ICN architecture, Ironic boot the baremetal server through PXE, it order to assist it. We have a lightweight DHCP server running in the provisioning network. Currently, in metal3 project, the provisioning network is required to boot the ramdisk and receive the hardware details from ramdisk to the ironic inspector. Ram disk gives the information regarding the PXE boot information to the ironic inspector and Ironic uses this information to initiates the deployment of OS images

ICN has L2 network for the BMC connectivity using net-c, DHCP server communication using provisioning network net-a.

Provisioning above Baremetalhost CRD controller can be directly debugged with openstack as follows

# export OS_TOKEN=fake-token
# export OS_URL=http://localhost:6385/
# openstack baremetal node list
+--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+
| UUID                                 | Name | Instance UUID                        | Power State | Provisioning State | Maintenance |
+--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+
| 36b462c1-02a0-499e-b4f1-cc6087a1c574 | demo | 36b462c1-02a0-499e-b4f1-cc6087a1c574 | power on    | active             | False       |
+--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+