在写 k8s的CRD controller的时候遇到需要用到subresources的情况
首先编写CRD文件的时候生命subresource
这里用kube-batch举例子
apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: podgroups.scheduling.sigs.dev spec: group: scheduling.sigs.dev names: kind: PodGroup plural: podgroups scope: Namespaced validation: openAPIV3Schema: properties: apiVersion: type: string kind: type: string metadata: type: object spec: properties: minMember: format: int32 type: integer queue: type: string priorityClassName: type: string type: object status: properties: succeeded: format: int32 type: integer failed: format: int32 type: integer running: format: int32 type: integer type: object type: object version: v1alpha2 subresources: status: {}
所使用的types.go
// +genclient // +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object // PodGroup is a collection of Pod; used for batch workload. type PodGroup struct { metav1.TypeMeta `json:",inline"` // Standard object's metadata. // More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata // +optional metav1.ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"` // Specification of the desired behavior of the pod group. // More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status // +optional Spec PodGroupSpec `json:"spec,omitempty" protobuf:"bytes,2,opt,name=spec"` // Status represents the current information about a pod group. // This data may not be up to date. // +optional Status PodGroupStatus `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"` } // PodGroupSpec represents the template of a pod group. type PodGroupSpec struct { // MinMember defines the minimal number of members/tasks to run the pod group; // if there's not enough resources to start all tasks, the scheduler // will not start anyone. MinMember int32 `json:"minMember,omitempty" protobuf:"bytes,1,opt,name=minMember"` // Queue defines the queue to allocate resource for PodGroup; if queue does not exist, // the PodGroup will not be scheduled. Queue string `json:"queue,omitempty" protobuf:"bytes,2,opt,name=queue"` // If specified, indicates the PodGroup's priority. "system-node-critical" and // "system-cluster-critical" are two special keywords which indicate the // highest priorities with the former being the highest priority. Any other // name must be defined by creating a PriorityClass object with that name. // If not specified, the PodGroup priority will be default or zero if there is no // default. // +optional PriorityClassName string `json:"priorityClassName,omitempty" protobuf:"bytes,3,opt,name=priorityClassName"` } // PodGroupStatus represents the current state of a pod group. type PodGroupStatus struct { // Current phase of PodGroup. Phase PodGroupPhase `json:"phase,omitempty" protobuf:"bytes,1,opt,name=phase"` // The conditions of PodGroup. // +optional Conditions []PodGroupCondition `json:"conditions,omitempty" protobuf:"bytes,2,opt,name=conditions"` // The number of actively running pods. // +optional Running int32 `json:"running,omitempty" protobuf:"bytes,3,opt,name=running"` // The number of pods which reached phase Succeeded. // +optional Succeeded int32 `json:"succeeded,omitempty" protobuf:"bytes,4,opt,name=succeeded"` // The number of pods which reached phase Failed. // +optional Failed int32 `json:"failed,omitempty" protobuf:"bytes,5,opt,name=failed"` }
注意的几个地方。
1. 当用yaml文件提交的时候,设置了status属性是没用的,因为默认提交的地址是资源的路径 也就是 /apis/{groupname}/{version}/namespaces/{ns}/{resourcesname}/{name}
所以这个subresources的状态设置不上去
例如我提交一个
apiVersion: scheduling.incubator.k8s.io/v1alpha1 kind: PodGroup metadata: name: qj-1 spec: minMember: 6
# 这个属性不会被kubectl命令设置上去 status succeeded: 1
来看下 CRD subresouces在k8s中的定义
// CustomResourceSubresources defines the status and scale subresources for CustomResources. type CustomResourceSubresources struct { // Status denotes the status subresource for CustomResources Status *CustomResourceSubresourceStatus // Scale denotes the scale subresource for CustomResources Scale *CustomResourceSubresourceScale }
// CustomResourceSubresourceStatus defines how to serve the status subresource for CustomResources.
// Status is represented by the `.status` JSON path inside of a CustomResource. When set,
// * exposes a /status subresource for the custom resource
// * PUT requests to the /status subresource take a custom resource object, and ignore changes to anything except the status stanza
// * PUT/POST/PATCH requests to the custom resource ignore changes to the status stanza
type CustomResourceSubresourceStatus struct{}
// CustomResourceSubresourceScale defines how to serve the scale subresource for CustomResources.
type CustomResourceSubresourceScale struct {
// SpecReplicasPath defines the JSON path inside of a CustomResource that corresponds to Scale.Spec.Replicas.
// Only JSON paths without the array notation are allowed.
// Must be a JSON Path under .spec.
// If there is no value under the given path in the CustomResource, the /scale subresource will return an error on GET.
SpecReplicasPath string
// StatusReplicasPath defines the JSON path inside of a CustomResource that corresponds to Scale.Status.Replicas.
// Only JSON paths without the array notation are allowed.
// Must be a JSON Path under .status.
// If there is no value under the given path in the CustomResource, the status replica value in the /scale subresource
// will default to 0.
StatusReplicasPath string
// LabelSelectorPath defines the JSON path inside of a CustomResource that corresponds to Scale.Status.Selector.
// Only JSON paths without the array notation are allowed.
// Must be a JSON Path under .status or .spec.
// Must be set to work with HPA.
// The field pointed by this JSON path must be a string field (not a complex selector struct)
// which contains a serialized label selector in string form.
// More info: https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions#scale-subresource
// If there is no value under the given path in the CustomResource, the status label selector value in the /scale
// subresource will default to the empty string.
// +optional
LabelSelectorPath *string
}
支持 Status, Scale 两种,
// * PUT requests to the /status subresource take a custom resource object, and ignore changes to anything except the status stanza
// * PUT/POST/PATCH requests to the custom resource ignore changes to the status stanza
当请求为 /status 的时会忽略对象的其他字段只修改 status
也就是说:如果想用命令行方式修改status字段,需要提交到 /apis/{groupname}/{version}/namespaces/{ns}/{resourcesname}/{name}/status
2. 在测试的时候如果想手动修改状态怎么办
要修改状态必须先获取当前对象,因为有reversion的限制,提交的对象会进行严重,所以先get
随后在get出来的对象中加入status字段
curl --cacert {ca.pen} -H "Authorization: Bearer {token}" APISERVER/apis/{groupname}/{version}/namespaces/{ns}/{resourcesname}/{name}
假如返回结果为
{
...
"spec":{...}
}
修改添加status
{
...
"spec":{...},
"status":{"success":1}
}
随后用这个body去请求
curl --cacert {ca.pen} -H "Authorization: Bearer {token}" -H "Content-Type: application/json" --request PUT APISERVER/apis/{groupname}/{version}/namespaces/{ns}/{resourcesname}/{name}/status --data '上边body'
记得一定要先去请求最新的对象,因为会验证resourceVersion