...
For some algorithms, cross-edge synchronization is required. More details will be provided later.
Design Details
There are generally two phases for machine learning model development, i.e. training and inference. Model behaviors are quite different depending on whether it is used for training or for inference. So we might as well define two different types of model CRDs:
- InferenceModel
- TrainingModel
InferenceModel CRD
| Code Block | ||||
|---|---|---|---|---|
| ||||
// InferenceModelSpec defines the desired state of an InferenceModel.
// Two ways of deployment are provided. The first is through a docker image.
// The second is through a data store, with a manifest of model files.
// If image is provided, manifest and targetVersion will be ignored.
type InferenceModelSpec struct {
ModelName string `json:"modelName"`
DeployToLayer string `json:"deployToLayer"`
FrameworkType string `json:"fraemworkType,omitempty"`
NodeSelector map[string]string `json:"nodeSelector,omitempty"`
Image string `json:"image,omitempty"`
Manifest []InferenceModelFile `json:"manifest,omitempty"`
TargetVersion string `json:"targetVersion,omitempty"`
Replicas *int32 `json:"replicas,omitempty"`
}
// InferenceModelFile defines an archive file for a single version of the model
type InferenceModelFile struct {
Version string `json:"version,omitempty"`
DownloadURL string `json:"downloadURL,omitempty"`
Sha256sum string `json:"sha256sum,omitempty"`
}
// InferenceModelStatus defines the observed state of InferenceModel
type InferenceModelStatus struct {
URL string `json:"url,omitempty"`
} |