At the time of writing this tutorial, Terraform is the king of infrastructure as it’s relatively easy to read/write, cloud agnostic, and has a large community. Terraforms syntax is uniformed as opposed to interacting and interacting with each CSP (cloud service provider) individually. You’re able to write similar solutions in a familiar language (HashiCorp Language or HCL for short) regardless of the provider (AWS, Azure, IBM). It’s an integral ingredient in any platform recipe but not the only solution for IaC. This rings true especially for all the Kubernetes shops out there as Crossplane brings infrastructure to your organizations clusters.

Crossplane is a tool brought to you by Upbound, that provides platform and Devop teams a control plane.

A definition from the team at Upbound:

"Control planes create and manage the lifecycle of resources. Control planes constantly check that the intended resources exist, report when the intended state doesn’t match reality and act to make things right.

Crossplane extends the Kubernetes control plane to be a universal control plane to check, report and act on any resource, anywhere."

The strength of Crossplane is inherited from the overall concept of container orchestration and a control plane that comes with Kubernetes. Our control plane defines what we want, and will continuously work to ensure that’s happening. If you say you want 4 pods, then Kubernetes will work to deploy 4 pods. If one of the pods fail, Kubernetes will work to bring that amount of pods back to 4, that’s what we desired remember? We extend this functionality to cloud resources. With Crossplane CRDS and a few other Kubernetes objects, we can submit cloud resources the same as we do pods, and deployments to the cluster.

Expanding Kubernetes to deploy cloud infrastructure may sound unusual but its a powerful approach to provisioning and managing cloud assets.

Quick and Short on CRD's

This YAML here is an example of what I’d submit to my Kubernetes cluster if I wanted to deploy a RDS in Azure with Crossplane. Notice at the top here the type is AzureBlob. If you’re familiar with Kubernetes you’ll say, “that’s an interesting type, I’m familiar with Pods, Secrets, and even Ingresses but I’ve never seen an AzureBlob. This is an example of a custom resource, which gets defined or added with a CRD (custom resource definition). With this feature, Kubernetes allows us to extend its API with a resource that doesn’t come out of the box with Kubernetes. We’ll install CRD's later in the quickstart tutorial.

A definition from Kubernetes documentation:

The CustomResourceDefinition API resource allows you to define custom resources. Defining a CRD object creates a new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource. The name of the CRD object itself must be a valid DNS subdomain name derived from the defined resource name and its API group; see how to create a CRD for more details.

When we submit the YAML, Kubernetes will now be responsible for ensuring that the resource that we defined and its desired state is fulfilled. We define the availability zone, engine and version and with a little configuring, we'll be able to provision it into the defined Azure account. Here’s where the Kubernetes comes in. Let’s say I want to make an edit, I want the database to be another version instead of what I previously described. I’ll change the file, and resubmit to the cluster.

The control plane or controller that works with the CRD will notice the change and make the update happens. Kubernetes wants resources to be in a Synced state. Synced typically means that what Kubernetes sees matches what is actually there. When I changed the version, the controller went to work as the custom resources state wasn’t synced and it did what it had to do to put it back in its desired state. Another example is in the classic accidental deletion. Let’s say a click happy developer on the team went into the Azure console and accidentally deleted a production workload database. This will trigger Kubernetes to say “Hold up, wait a minute. Something is wrong.” Its state has changed and it will again, do whatever needs to be done to ensure that the object returns to a Synced state. In this particular example, Kubernetes will re deploy the database with all the specifications we’ve defined.

With that out the way, let's get into the cluster and do some Crossplane'ing!

* We strongly encourage completing the official quickstart from the Crossplane team found here.

This post will serve as a further explanation of what they're doing in the Crossplane documentation. A deeper dive if you will, written forplatform engineers to understand its value. We want to leave the basics to Crossplane and explain the pieces of the project that are vital for platform teams. Meaning, how to create custom API's and template out infrastructure for our developers. Platforming teams will be able to provide a golden path for infrastucture using Kubernetes.

Core Concepts

This guides is intended for proof-of-concept or lab demonstrations only and not recommended as a guide for production.

Kubernetes is an API and we're able to define a custom API within it using the following core components of Crossplane:

GRAPHIC HERE

‍

Compositions
Composite Resource Definition
Compositie Resource
Claims

Steps:

When defining my own API with Crossplane, I did so in the following steps:

Define API: This is where the platform team decides what details you want users to have access to for defining. How granular do you want your developers to go when it comes to configuring their objects? Use many fields if your users want to have full control of what they deploy, use less and set default values only providing developers with a few customizable options. A mix of both may be ideal but its your configuration, totally up to you. When we actually apply this layout, it will deploy what Crossplane calls a Composite Resource.
Install API: We know what we want from our users, now create the API endpoint in the cluster. Crossplane uses what they call a Composite Resource Definition (XRD) to do so. XRD's are where your API group, the resource/kind that you are defining for users, and installs your schema. This would include things such as defining what input should be a string or number, what entries a required, and input for patches. More on these later.
Define the cloud resources: So far we have an API and we know need to apply this data to what we came here for, cloud objects! In Crossplane, we use a Composition to define the cloud objects we want our API to do. This is an AWS S3 bucket, GCE compute, or the purpose of this post an EKS cluster. We're going to pass user input defined in our API to fill in templates for our EKS cluster and the "child" object that work with EKS.

Define API

The following is the custom object that we want our user to interact with:

apiVersion: cluster.example.com/v1alpha1
kind: Small
metadata:
  name: example-cluster
spec: 
  region: east
  version: "1.28"

apiVersion: Our value cluster.example.com is the endpoint for the group we're defining. Notice the leading "cluster", think of this as the group that holds a collection of the next line kind. Platforms team will offer different types of clusters when we make a request to cluster.example.com. Think of the different types of clusters as different flavors of a cluster, they're held in the cluster group.

kind: Continuing with this example, kind is a flavor. We have Small set for our example but further down the line we want to offer a Medium and Large cluster. These kinds are unique and will have different configurations but we're just stating what object in again the cluster group are we trying to use.

metadata.name: Simply giving our Small cluster a name.

spec.region: This is where the fun starts, when I install my XRD and Composition I'm going to use the values set here to deploy the EKS cluster. We'll let users define what region to use and the version of Kubernetes they'd like for the cluster. You can see that the given value is not a valid AWS region naem such as us-east-1 or us-west2, what I'm doing here is called a patch simplifying input for the user and limiting what value is even offered to the developer.

With that outlined let's install this API using an XRD.

Install API

Now we're ready to install our API and group. Use the following to create an XRD file:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: smalls.cluster.example.com
spec:
  group: cluster.example.com
  names:
    kind: Small
    plural: smalls
  versions:
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              region:
                type: string
                oneOf:
                  - pattern: '^east$'
                  - pattern: '^west$'
              version:
                type: string        
            required:
              - region
          status: 
            type: object
            properties:
              secondResource:
                type: string
    served: true
    referenceable: true
  claimNames:
    kind: SmallClusterClaim
    plural: smallclusterclaim

The file is pretty self explanatory when you look at it, we're naming the XRD and defining the "cluster" group like we explained earlier, defining the version and defining the type of inputs the API excepts. It's spelled out using OpenAPI schema and notice the oneOf key. I'm just saying here that when someone defines spec.region using our custom API, it can only be the string "east" or "west". Anything else will be invalid and fail during validation.

Another point to notice is at the bottom of the file where we define claims. Claims are out of scope for this guide but its worth noting that they allow users to access this API either at the cluster level with theendpoint or in a namespace with the endpoint.

Submit your file to the cluster with a kubectl apply:

kubectl apply -f example-crd.yaml

Define Cloud Resources

The last step is defining the cloud resources to deploy when our custom API kind is submitted to the cluster. Composite can seem daunting at first but remember they're just a definition of multiple cloud resources referenced as a single object. Our Small cluster will be an EKS cluster and a backing NodeGroup.

Both are kinds apart of the AWS EKS Crossplane provider. We define both in our Composition and set our XRD as its backing custom type.

234 56 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70

class="comment linenumber react-syntax-highlighter-line-number" style="display:inline-block;min-width:3.25em;padding-right:1em;text-align:right;user-select:none">1apiVersion: apiextensions.crossplane.io/v1 class="hljs-attr">kind: Composition class="hljs-attr">metadata: name: small-cluster-composition class="hljs-attr">spec: mode: Pipeline pipeline: - step: patch-and-transform functionRef: name: function-patch-and-transform input: apiVersion: pt.fn.crossplane.io/v1beta1 kind: Resources resources: - name: eksCluster base: apiVersion: eks.aws.upbound.io/v1beta1 kind: Cluster metadata: name: default-cluster spec: forProvider: region: us-east-2 roleArn: arn:aws:iam::010459457750976:role/eks-service-role vpcConfig: - subnetIds: - subnet-13559650680564657 - subnet-23076042870358390 providerConfigRef: name: default patches: - type: FromCompositeFieldPath fromFieldPath: "spec.region" toFieldPath: "spec.forProvider.region" transforms: - type: map map: east: "us-east-2" west: "us-west-1" - type: ToCompositeFieldPath fromFieldPath: metadata.name toFieldPath: status.secondResource - type: FromCompositeFieldPath fromFieldPath: "spec.version" toFieldPath: "spec.forProvider.version" - name: nodeGroup base: apiVersion: eks.aws.upbound.io/v1beta1 kind: NodeGroup metadata: name: default-ng spec: forProvider: clusterName: default-cluster-ref nodeRoleArn: arn:aws:iam::486738506059705:role/eks-ec2-service-role region: us-east-2 scalingConfig: - desiredSize: 1 maxSize: 2 minSize: 1 subnetIds: - subnet-13559650680564657 - subnet-23076042870358390 patches: - type: FromCompositeFieldPath fromFieldPath: status.secondResource toFieldPath: spec.forProvider.clusterName compositeTypeRef: apiVersion: cluster.example.com/v1alpha1 kind: Small

It's a lot going on here so let's walk through it a bit:

spec.mode: Our composition is configured to use a pipeline of composition functions to deploy the list of resources. Lines 6-11 define first step would be a function called function-patch-and-transform. This is actually a function from Crossplane themselves but you could create your composite function and reference them the same way. More on composite functions here. You need to have the Function deployed, use this one from Crossplane:

apiVersion: pkg.crossplane.io/v1
kind: Function
metadata:
  name: function-patch-and-transform
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-patch-and-transform:v0.1.4

Then apply:

kubectl apply -f example-function.yaml

patches: We mentioned earlier that patches are ways to take user input from our custom API and fill the values into our composition. Most Crossplane types from the AWS Provider have a spec.forProvider configuration. The values on lines 31-45 are templates of data to be passed to the AWS API. With patches we're able to define what we want to fill in those keys using what we defined in our custom API. The first one on line 32 is saying that what a user submits for spec.region, copy that value and use it in this Composition for spec.forProvider.region. Not only this, but based on the actual value given, "east" means a certain region and "west means another. Remember when we installed our API earlier, we defined that our custom type will only accept those two values.

The next patch is import. On lines 40-42, we're copying the clusters name to a status block field named secondResource. EKS requires a backing node group to run the cluster. When you define the define group, the clusters external name is how the group knows which EKS cluster its apart of. Since we're defining the cluster and node group in the same Composition, once the EKS cluster is created it will pass its name to the nodeGroup so it can be deployed properly. The NodeGroup has its own patches and the resource is able to get the name on lines 64-67.

Deploy the Custom Type

Last thing left to do is deploy an instance of our type from the skeleton created earlier:

apiVersion: cluster.example.com/v1alpha1
kind: Small
metadata:
  name: example-cluster
spec: 
  region: east
  version: "1.28"

Then apply:

kubectl apply -f small-example.yaml

EKS clusters and Node Groups take some time to deploy but after some waiting, you should have a new EKS cluster. When you're done with it just delete it and both the Node Group and cluster are removed:

kubectl delete small example-cluster

Business Value

Infrastructure management gets supercharged when paired with the reactive nature of a control plane. Infrastructure being declarative removes unneccesary complexity for users. With easily understoof and defined YAML files, Crossplane combines the innate power of Kubernetes with orchestration making a suitable competitor to Crossplane. As Crossplane grows, Kubernetes shops and platform teams would be served well if they explored the projects features.

In later posts we will combine Crossplane with other tools to combine a production ready platform.

‍

Infrastructure As Code with Crossplane