Skip to content

Latest commit

 

History

History
179 lines (123 loc) · 10.9 KB

README.md

File metadata and controls

179 lines (123 loc) · 10.9 KB
Raw

kcp provides a Kubernetes-like control plane with true multitenancy, with flexible compute powered by real Kubernetes clusters.

build status badge

kcp is a generic CustomResourceDefinition (CRD) apiserver that is divided into multiple "logical clusters" that enable multitenancy of cluster-scoped resources such as CRDs and Namespaces. Each logical cluster is independent: the available APIs and data are separate from one logical cluster to another.

By default, kcp only knows about:

kubectl api-resources showing kcp's API resources

Any other resources, including standard Kubernetes resources like Deployments and the rest, can be added as CRDs and optionally reconciled using the standard controllers launched against the kcp API, or via two-way replication into a Kubernetes cluster.

Why would I want that?

Kubernetes is mainly known as a container orchestration platform today, but we believe it can be even more.

With the power of CRDs, Kubernetes provides a flexible platform for declarative APIs of all types, and the reconciliation pattern common to Kubernetes controllers is a powerful tool in building robust, expressive systems.

At the same time, a diverse and creative community of tools and services has sprung up around Kubernetes APIs.

Imagine a declarative Kubernetes-style API for anything, supported by an ecosystem of Kubernetes-aware tooling, separate from Kubernetes-the-container-orchestrator.

That's kcp.

Can I start using kcp today?

Yes! This work is still in early development, which means it's not ready for production and APIs, commands, flags, etc. are subject to change, but also that your feedback can have a big impact. Please try it out and let us know what you like, dislike, what works, what doesn't, etc.

kcp is currently a prototype, not a project. We're exploring these ideas here to try them out, experiment, and bounce them off each other.

Can kcp do anything else?

Yes! Here are a few of our top-level goals:

Host thousands (*) of small-ish Kubernetes-like logical clusters in a single instance

  • Orders of magnitude fewer resources (~50) in a logical cluster compared to a typical multi-tenant Kubernetes cluster
  • Inexpensive - nearly for free in terms of resource utilization & cost for an empty cluster
  • Independent - install different versions of a CRD in each logical cluster (!!!)
  • Per-cluster administrative rights - each "owner" (person/team/group) of a cluster is a full admin

Treat compute as a utility

  • Transparent multi-cluster - run kcp as a control plane in front of your physical compute clusters for workloads, and let kcp determine how to schedule your workloads to physical compute clusters
  • Dynamically add more compute capacity as demand increases - not just nodes, but entire Kubernetes clusters

Massive scale

  • Model "organizations" as a way to group and manage "workspaces" (logical clusters). Support upwards of 10,000 organizations.
  • 1,000,000 workspaces in a single installation
  • Span across 1,000 shards (a shard is a kcp instance that holds a subset of organization and workspace content)
  • This area is under active investigation. Stay tuned for more details!

Local Kubernetes Development?

kcp could be useful for local development scenarios, where you don't necessarily care about all of Kubernetes' many built-in resources and their reconciling controllers.

Embedded/low-resource scenarios?

kcp could be useful for environments where resources are scarce, by limiting the number of controllers that need to run. Kubernetes' asynchronous reconciliation pattern can also be very powerful in disconnected or intermittently connected environments, regardless of how workloads actually run.

What about a long-term vision?

For more detailed information, check out our GOALS.md doc and our docs directory.

Is kcp a "fork" of Kubernetes? 🍴

No.

kcp as a prototype currently depends on some unmerged changes to Kubernetes, but we intend to pursue these changes through the usual KEP process, until (hopefully!) Kubernetes can be configured to run as kcp runs today.

Our intention is that our experiments improve Kubernetes for everyone, by improving CRDs and scaling resource watching, and enabling more, better controllers for everyone, whether you're using Kubernetes as a container orchestrator or not.

Our kcp specific patches in the kcp-dev/kubernetes repo.

What's in this repo?

  • cmd
    • cluster-controller
      • Keeps track of physical Clusters where workloads and other resources are synchronized
      • Enabled by default in cmd/kcp
    • compat
      • Checks compatibility between two CRDs and can generate the least common denominator CRD YAML if requested
    • crd-puller
      • Downloads CRDs from a cluster and writes them to YAML files
      • This functionality is included in parts of cmd/kcp
    • deployment-splitter
      • Splits a Deployment into multiple "virtual Deployments" across multiple physical clusters
    • kcp
      • The primary executable, which serves a Kubernetes-style API with a minimum of built-in types
    • kubectl-kcp
      • A kubectl plugin that offers kcp specific functionality
    • shard-proxy
      • An early experimental server that provides a workspace index and sharding details
    • syncer
      • Runs on Kubernetes clusters registered with the cluster-controller
      • Synchronizes resources in kcp assigned to the clusters
  • cmd/virtual-workspaces
    • Demonstrates how to implement apiservers for custom access-patterns, e.g. like a workspace index.
  • config:
    • Contains generated CRD YAML and helpers to bootstrap installing CRDs in kcp
  • contrib:
    • CRDs for Kubernetes apps and core (empty group) types
    • Demo scripts
    • Examples
    • Local development utilities
  • docs:
    • Our documentation
  • hack:
    • Scripts and tools to support the development process
  • pkg:
    • The majority of the code base
  • test:
    • End to end tests
  • third_party:
    • Code from third party projects for use in this repository

What does kcp stand for?

kcp as a project stands for equality and justice for all people.

However, kcp is not an acronym.

How do I get started?

  1. Clone the repository.
  2. Install Go (1.17+).
  3. Download the latest kubectl binary for your OS.
  4. Build and start kcp in the background: go run ./cmd/kcp start.
  5. Tell kubectl where to find the kubeconfig: export KUBECONFIG=.kcp/admin.kubeconfig (this assumes your working directory is the root directory of the repository).
  6. Confirm you can connect to kcp: kubectl api-resources.

For more scenarios, see DEVELOPMENT.md.

This sounds cool and I want to help!

Thanks! And great! Please check out:

You can also reach us here, in this repository via issues and discussions, or:

References