In this post, I’ll try to show how to create a CRD(Custom Resource Definitions) and controlling them.

First, we need a kubebuilder binary to initialize a skeleton project. https://github.com/kubernetes-sigs/kubebuilder/releases

$:  kubebuilder init --domain blog.webischia.com                              
Writing scaffold for you to edit...
Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.5.0
Update go.mod:
$ go mod tidy
Running make:
$ make
/home/f/go/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go build -o bin/manager main.go
kubrNext: define a resource with:
$ kubebuilder create api

This command will generate necessary classes and include libraries to our go.mod file. 

Now that we have set up the project, let’s add API. In Kubernetes, resources have their own defined controllers. Such as Deployments. https://kubernetes.io/docs/concepts/architecture/controller/

“A controller tracks at least one Kubernetes resource type. These objects have a spec field that represents the desired state. The controller(s) for that resource are responsible for making the current state come closer to that desired state.” 

In this post, I’m going to create The Expanse’s spaceships in my Kubernetes Cluster.

Creating API

$ kubebuilder create api --group expanse --version v1beta1 --kind SpaceShips
Create Resource [y/n]
y
Create Controller [y/n]
y
Writing scaffold for you to edit...
api/v1beta1/spaceships_types.go
controllers/spaceships_controller.go
Running make:
$ make
/home/f/go/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go build -o bin/manager main.go

After this command, we need to add our fields in the types class. Then make command will generate the necessary classes and YAML files for us. 

apiVersion: expanse.blog.webischia.com/v1beta1
kind: SpaceShips
metadata:
  name: spaceships-sample
spec:
  name: MyShip
  class: Corvette
  owner: Earth

The code above shows SpaceShips resource will have three fields; Name, Class, and Owner. 

Installing in Kubernetes

On my local Minikube is available and I can test my resources and logic in Minikube with Kubernetes v1.17.11. 

Let’s install the CRD’s. 

$ make install
/home/f/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
kustomize build config/crd | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/spaceships.expanse.blog.webischia.com created

If we want to list all CRD’s we’ll be able to see our CRD as well.

$ kubectl get crd |grep webischia
spaceships.expanse.blog.webischia.com         2020-10-31T11:04:34Z

Describing the CRD will show us the fields we set in types.go file.

$ kubectl describe crd spaceships.expanse.blog.webischia.com
Name:         spaceships.expanse.blog.webischia.com
Namespace:    
Labels:       <none>
Annotations:  controller-gen.kubebuilder.io/version: v0.2.5
API Version:  apiextensions.k8s.io/v1
Kind:         CustomResourceDefinition
Metadata:
  Creation Timestamp:  2020-10-31T11:04:34Z
..
Spec:
Description:  SpaceShipsSpec defines the desired state of SpaceShips
Properties:
  Class:
    Type:  string
  Name:
    Type:  string
  Owner:
    Type:  string

So far we created our resources and installed the CustomResourceDefinition in a cluster. If we start our controller we’ll be able to see every event we defined for our resource. 

By default kubebuilder will set these:

//+kubebuilder:rbac:groups=expanse.blog.webischia.com,resources=spaceships,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=expanse.blog.webischia.com,resources=spaceships/status,verbs=get;update;patch

Let’s start our simple manager binary. 

2020-10-31T14:10:11.180+0300    INFO    controller-runtime.metrics      metrics server is starting to listen    {"addr": ":8080"}
2020-10-31T14:10:11.180+0300    INFO    setup   starting manager
2020-10-31T14:10:11.181+0300    INFO    controller-runtime.manager      starting metrics server {"path": "/metrics"}
2020-10-31T14:10:11.181+0300    INFO    controller-runtime.controller   Starting EventSource    {"controller": "spaceships", "source": "kind source: /, Kind="}
2020-10-31T14:10:11.281+0300    INFO    controller-runtime.controller   Starting Controller     {"controller": "spaceships"}
2020-10-31T14:10:11.281+0300    INFO    controller-runtime.controller   Starting workers        {"controller": "spaceships", "worker count": 1}

Awesome everything ok and it connected to our cluster and started to watch. 

We are going to apply a sample YAML for our spaceships.

$ kubectl apply -f sample.yml 
spaceships.expanse.blog.webischia.com/spaceships-sample created

Since we added the log to our controller we should be able to see this object. 

2020-10-31T14:10:52.004+0300    INFO    controllers.SpaceShips  Controller received : default/spaceships-sample
2020-10-31T14:10:52.004+0300    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "spaceships", "request": "default/spaceships-sample"}

Awesome and we also see a new keyword Reconcile. Since we returned a no error that means we successfully concealed the object and no longer watching unless any status or API operations perform.

And we can easily get our object using kubectl get command.

$ kubectl get spaceships
NAME                AGE
spaceships-sample   7m20s

Statuses

In controller pattern we need to be able understand our application state, using this state (or status) we’ll do some operations according our logic. In order to do that I added subresource by following the kubebuilder book.

https://book.kubebuilder.io/reference/generating-crd.html?highlight=stat#status

Here is the implemented status field for my SpaceShip resource.

const Created SpaceShipPhase = "Created"
const Active SpaceShipPhase = "Active"
const Passive SpaceShipPhase = "Passive"
// SpaceShipsStatus defines the observed state of SpaceShips
type SpaceShipsStatus struct {
    Phase SpaceShipPhase `json:"phase,omitempty"`
}
// +kubebuilder:subresource:status
type SpaceShips struct {
...
.
Status SpaceShipsStatus `json:"status,omitempty"`
}

Now, Controller will know which state is the current object. To do necessary operations added the switch block.

switch spaceship.Status.Phase {
    case "":
        // with new created ones are doesn't have any status so its empty
        return r.createSpaceship(spaceship)
    case expansev1beta1.Created:
        return r.startTheEngines(spaceship)
    }

When creating an object to Kubernetes, status field are not sent and this field is will populate in Kubernetes system.

The status describes the current state of the object, supplied and updated by the Kubernetes system and its components. The Kubernetes control plane continually and actively manages every object's actual state to match the desired state you supplied. Link to Kubernetes Docs

DynamoDB

In this example I’ll be writing the yaml content in DynamoDB, therefore I created a necessary package and implemented the simple DynamoDB putItem operation.

func (d *DBConfig) Write(ships v1beta1.SpaceShips) error {
    putItem := &dynamodb.PutItemInput{
        Item: map[string]*dynamodb.AttributeValue{
            “name”: {
                S: aws.String(ships.Spec.Name),
            },
            “class”: {
                S: aws.String(ships.Spec.Class),
            },
            “owner”: {
                S: aws.String(ships.Spec.Owner),
            },
            “status”: {
                S: aws.String(string(ships.Status.Phase)),
            },
        },
        TableName: aws.String(TABLE_NAME),
    }
    _, err := d.DynamoDB.PutItem(putItem)
    if err != nil {
        return err
    }
    return nil
}

Goal

Our goal is the keeping Expanse ship logs with Kubernetes and DynamoDB. So let’s create our ship.

apiVersion: expanse.blog.webischia.com/v1beta1
kind: SpaceShips
metadata:
  name: spaceships-sample
spec:
  name: MyShip
  class: Corvette
  owner: Earth

After applying this, controller will be triggered and in the switch block it can’t find any status and it’ll call the r.createSpaceship(spaceship) function. This function will create the initial DynamoDB entry and update the status of object to CREATED.

After this operation succeed, object status will be updated and that means our controller will be trigger again. This time it can find the status and will call r.startTheEngines(spaceship) function. This function will set status field to DynamoDB and update status of CRD to ACTIVE.

Kubernetes:

$ kubectl describe spaceships
Name:         spaceships-sample
Namespace:    default
Labels:       
Annotations:  API Version:  expanse.blog.webischia.com/v1beta1
Kind:         SpaceShips
Metadata:
  Creation Timestamp:  2020-10-31T13:09:57Z
  Generation:          1
  Resource Version:    35155
  Self Link:           /apis/expanse.blog.webischia.com/v1beta1/namespaces/default/spaceships/spaceships-sample
  UID:                 591b4e89-31f0-4a63-82f4-6b284cfd148a
Spec:
  Class:  Corvette
  Name:   MyShip
  Owner:  Earth
Status:
  Phase:  Active
Events:   

DynamoDB:

You can access the whole code here

Future Work

• Implement Delete/Stop Engines