Karpenter is a Kuberneters autoprovisioner that provides just-in-time Nodes for any cluster. This repository contains an implementation of Karpenter that uses Cluster API as the underlying infrastructure provider. Meaning that this implementation of Karpenter is intended to be capable of managing Nodes on any Cluster API owned cluster.
What is a "provider"?
The term "provider" is used in different ways depending on the context. Within the Cluster API frame of reference, a "provider" is the controllers and CRDs which form the integration between the Cluster API Kubernetes resources (for example Machine, MachineDeployment, etc.) and the underlying infrastructure (for example OpenStack). Within the Karpenter frame of reference, a "provider" is the implementation of Karpenter for a specific infrastructure.
This repository contains the specific implementation of Karpenter that can interact with Kubernetes clusters which contain the Cluster API CRDs and controllers. In this manner, Cluster API is the infrastructure to which this Karpenter implementation can interact with, in much the same way as other Karpenter implementations may interact directly with AWS or Azure.
This project is currently, as of summer 2024, an experimental proof of concept for how Karpenter might integrate with Cluster API. The intention is for the Kubernetes community to use this experiment for learning more about the design pattern, feature gaps, and areas for improvement. With the ultimate goal being a standard release cycle and production ready Karpenter Cluster API provider.
For information about how to build and run the Karpenter Cluster API provider, please see the Getting Started documentation.
These are a few topics that are under active discussion by the Cluster API Karpenter Feature Group which are impacting the architecture and implementation of this provider.
- Multi-cluster client configuration. For Cluster Api, when using clusters in a hub/spoke topology multiple configurations are required to access all the Kubernetes objects. This poses a design issue when using Karpenter as it assumes a single cluster, and single configuration. A common deployment pattern will be to have Cluster API Machine resources in the same cluster as the Karpenter NodePool resources, with the Node and Pod objects residing in a different cluster.
- Tracking Machines through scalable resource As an initial design approach, this provider will interface with MachineDeployment resource from Cluster API. The intention is to preserve the experience and behavioral configurations encoded in MachineDeployments to help guide the creation of Machines from NodeClaims. This does create some unique issues when translating between replica changes in the MachineDeployment to specific Machines that are created.
- Identification of Cluster API resources participating with Karpenter. There are several types of Cluster API resources which can participate in Karpenter provisioning, from the infrastructure templates to the machines. Users will need to indicate which infrastructure templates can be utilized by Karpenter, and likewise which are the machines created by Karpenter.
- Exposure of price data. Cluster API does not have a mechanism for providers to expose data about the cost of specific instance types. Having this information will ultimately be essential to unlocking the cost saving features of Karpenter.