Goals
This wiki describes the specifications for designing the Binary Provisioning Agent required for the Integrated Cloud Native Akraino project.
Overview of BPA
The BPA is part of the infra local controller which runs as a bootstrap k8s cluster in the ICN project. As described in Integrated Cloud Native Akraino project, the purpose of the BPA is to install packages that cannot be installed using kubectl. It will be called once the operating system (Linux) has been installed in the compute nodes by the baremetal operator. The Binary Provisioning Agent will carry out the following functions;
...
| Code Block |
|---|
// ProvisioningSpec defines the desired state of Provisioning
// +k8s:openapi-gen=true
type ProvisioningSpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "operator-sdk generate k8s" to regenerate code after modifying this file
// Add custom validation using kubebuilder tags: https://book-v1.book.kubebuilder.io/beyond_basics/generating_crd.html
Masters []map[string]Master `json:"masters,omitempty"`
Workers []map[string]Worker `json:"workers,omitempty"`
KUDPlugins []string `json:"KUDPlugins,omitempty"`
}
// ProvisioningStatus defines the observed state of Provisioning
// +k8s:openapi-gen=true
type ProvisioningStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
// Important: Run "operator-sdk generate k8s" to regenerate code after modifying this file
// Add custom validation using kubebuilder tags: https://book-v1.book.kubebuilder.io/beyond_basics/generating_crd.html
}
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// Provisioning is the Schema for the provisionings API
// +k8s:openapi-gen=true
// +kubebuilder:subresource:status
type Provisioning struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec ProvisioningSpec `json:"spec,omitempty"`
Status ProvisioningStatus `json:"status,omitempty"`
}
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// ProvisioningList contains a list of Provisioning
type ProvisioningList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []Provisioning `json:"items"`
}
// master struct contains resource requirements for a master node
type Master struct {
MACaddress string `json:"mac-address,omitempty"`
CPU int32 `json:"cpu,omitempty"`
Memory string `json:"memory,omitempty"`
}
// worker struct contains resource requirements for a worker node
type Worker struct {
MACaddress string `json:"mac-address,omitempty"`
CPU int32 `json:"cpu,omitempty"`
Memory string `json:"memory,omitempty"`
SRIOV bool `json:"sriov,omitempty"`
QAT bool `json:"qat,omitempty"`
}
func init() {
SchemeBuilder.Register(&Provisioning{}, &ProvisioningList{})
}
|
The variables in the ProvisioningSpec struct are used to create the data structures in the yaml spec for the custom resource. Three variables are added to the ProvisioningSpecstruct;
- Masters: This variable will contain an array of Master objects. The master struct as defined in the *-types.go file above contains CPU and memory information, this information would be used by the BPA operator to determine which compute nodes to assign the role of Master to when it gets the baremetal list from the API server.
- Workers: This variable will contain an array of Worker objects. Similar to the case of the Masters variables, the Worker struct will contain resource requirements for the Worker nodes and the BPA operator will use this information to determine which hosts to assign the role of worker.
- KUDPlugins: This variable will contain the list of KUD plugins to be installed with KUD in the cluster
...
| Code Block | ||
|---|---|---|
| ||
apiVersion: bpa.akraino.org/v1alpha1 kind: softwareSoftware metadata: name: software-sample labels: cluster: cluster-abcxyz owner: c1 spec: mastername: example-software spec: softwareFrommasterSoftware: - configMapRef:curl - htop name: master-configmap workerjq: softwareFrom: version: 1.5+dfsg-1ubuntu0.1 - configMapRefmaven: version: 3.3.9-3 name: worker-configmapworkerSoftware: - curl - htop - tmux - jq |
Cluster CRD
The cluster CRD will have the Cluster name and contain the provisioning CR and/or the software CR for the specified cluster
...
This proposal would make it possible to assign roles to nodes based on the features discovered. The baremetal operator list returns much more information about the nodes, we would be able to extend the feature to allow the operator determine the right nodes to use complex requirements such as CPU model, memory, CPU..etc This would feed into Hardware Platform Awareness (HPA)
References
- https://wikilf-akraino.akrainoatlassian.orgnet/wiki/pages/viewpage.action?pageId=1199587713665333&show-miniview
- https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#advanced-topics
Presentation:
View file name Akraino-Intergrated-Cloud-Native-NestedK8s HA.pptx height 250