From c99564d7d7a71e42cf780cf5fce8e3909f6ee48e Mon Sep 17 00:00:00 2001 From: Jack Jackson Date: Sat, 11 May 2024 16:07:44 -0700 Subject: [PATCH] Base App Infra Entry --- blog/content/posts/base-app-infrastructure.md | 195 ++++++++++++++++++ blog/content/posts/vault-secrets-into-k8s.md | 2 + 2 files changed, 197 insertions(+) create mode 100644 blog/content/posts/base-app-infrastructure.md diff --git a/blog/content/posts/base-app-infrastructure.md b/blog/content/posts/base-app-infrastructure.md new file mode 100644 index 0000000..e30700d --- /dev/null +++ b/blog/content/posts/base-app-infrastructure.md @@ -0,0 +1,195 @@ +--- +title: "Base App Infrastructure" +date: 2024-05-10T03:00:23-07:00 +tags: + - crossplane + - homelab + - k8s + - SDLC + - vault + +--- +In my [previous post]({{< ref "/posts/vault-secrets-into-k8s" >}}), I had figured out how to inject Vault secrets into Kubernetes Secrets using the [Vault Secrets Operator](https://developer.hashicorp.com/vault/tutorials/kubernetes/vault-secrets-operator). My runthrough of the walkthrough worked, but I [swiftly ran into namespacing issues]({{< ref "/posts/vault-secrets-into-k8s#added-2024-04-29-namespacing-secrets" >}}) when trying to use it "_in production_". + + + +# The Problem + +The setup can be divided into two parts[^platform-vs-app-team]: +* Creation of a Vault Role (with `boundServiceAccountNamespaces` corresponding with the k8s namespaces that should be permitted to access it) and Policy, and a k8s `VaultAuth` object telling the Vault Secrets Operator how to access the Vault Role. +* Creation of a `VaultStaticSecret` (referencing the VaultAuth object) in the app's `-deployment` repo, which results in a k8s secret. + +As I started trying to extend my initial installation to other apps, I realized that simply adding more k8s namespaces to the `boundServiceAccountNamespaces` of a single Vault Role would not be a secure solution - it would allow _any_ pods in any of the bound namespaces to access any secret of any of the (other) applications. Ideally, each application-stage (or, equivalently, each k8s namespace[^namespaces-per-application]) would have its own resources created, with the Vault Role only accessible from that namespace[^sub-namespace-permissions]. + +## Why do I care? + +You may be wondering why I care about Least Privilege - after all, it's only my own homelab, surely I know and trust every application that's running on it? Well, to an extent. I trust them enough to install them, but it still doesn't hurt to limit their privileges so that any unforeseen misbehaviour - whether deliberate or accidental - has limited impact. More importantly, my primary motivation in running this homelab is to learn and practice technical skills - the tasks don't have to be entirely practical, so long as they are educational! In fact, as you'll see shortly, this problem is almost-exactly equivalent to one I'm going to be solving at work soon, so doing this "right" is a good head-start. + +# The solution + +Ideally, I'd be able to automate (via extracted-and-parameterized logic) the creation of these resources as part of the application definition, since many apps will have similar requirements and I want to minimize any manual or imperative setup. + +Thankfully, this is pretty close to a problem that I've been looking into at work, so I have a solution ready to go - [Crossplane](https://www.crossplane.io/), a tool that allows: +* management of "_External Resources_" (i.e. resources in systems outside Kubernetes, like Vault, Argo, etc.) via Kubernetes objects - i.e. you can declaratively create and update a Kubernetes object (a "_Managed Resource_") which represents the External Resource, and the Kubernetes reconciliation loop will keep the External Resource up-to-date. +* "bundling" of resources into Compositions - parameterized and inter-related collections of resources, analagous to Constructs in CDK. + +![Diagram of the interrelation of the various Crossplane concepts](https://docs.crossplane.io/media/composition-how-it-works.svg "Diagram of the interrelation of the various Crossplane concepts") + +With Crossplane in hand, the solution becomes simple: +* (while wearing my "Platform Team" hat) install a Provider (the interface between Crossplane and an external service) for Vault, and create a Composition which bundles the Vault resources that are necessary for Vault Secrets Operator setup. +* (wearing my "App team" hat) whenever I install an app which requires secret injection, do so alongside a Composite Resource (an instance of a Composition). All from the convenience of a single deployment repo, and with only a few extra lines of configuration! + +## Walkthrough + +You can see the solution [here](https://gitea.scubbo.org/scubbo/helm-charts/src/commit/e798564692f71187e3ff3f9d77f3aa1c46ca9ee4/charts/vault-crossplane-integration/base-app-infra.yaml). + +### XRD + +(Lines 1-26) A [Composite Resource Definition](https://docs.crossplane.io/latest/concepts/composite-resource-definitions/) (or "XRD" - yeah, I know, but Kubernetes had already taken the term "CRD") is like (in Programming Language terms) the Interface to a [Composition](https://docs.crossplane.io/latest/concepts/compositions/)'s Implementation, or (in Web Service) the API Spec or schema. It defines how a consumer can invoke a Composition - the name they should use, and the parameters they should pass. Consumers can either invoke this by its name if creating a cluster-scoped [Composite Resource](https://docs.crossplane.io/latest/concepts/composite-resources/), or in a namespaced context via a [Claim](https://docs.crossplane.io/latest/concepts/claims/). + +This definition is saying: +* (Lines 6-8) "_There's a Composition that can be addressed as `xbaseapplicationinfrastructures.scubbo.org`_..." +* (Lines 10-12) "_...(which can also be addressed by the Claim Name `BaseAppInfra`)..._" +* (Lines 13-25) "_...which has only a single version defined, which takes a single string parameter named `appName`_" + +It is apparently possible to provide [multiple schema versions](https://docs.crossplane.io/v1.15/concepts/composite-resource-definitions/#multiple-schema-versions) - but since "_new required fields are a 'breaking change.'_" and "_Only one version can be `referenceable` \[...which...\] indicates which version of the schema Compositions use_", I'm not really sure how that is actually useful - and this is borne out by the fact that "_Crossplane recommends implementing breaking schema changes as brand new XRDs._". + +### Top-level Composition + +The only point to note in lines 29-36 is that `spec.compositeTypeRef.apiVersion` and `spec.compositeTypeRef.kind` must match the values set on 6, 8, and 14. + +### Vault Resources + +Lines 37-136 define Vault Resources, provided by the [Vault Provider](https://github.com/upbound/provider-vault). These create a Vault Role, Policy, and KV Secrets Mount roughly as described in the [walkthrough](https://developer.hashicorp.com/vault/tutorials/kubernetes/vault-secrets-operator). Note the use of [patches and transforms](https://docs.crossplane.io/latest/concepts/patch-and-transform/) to set values in the Managed Resources based on properties of the Claim (the Kubernetes namespace and the parameter `appName`) + +### Kubernetes Resource + +The [Vault Secrets Operator walkthrough](https://developer.hashicorp.com/vault/tutorials/kubernetes/vault-secrets-operator) also [requires](https://github.com/hashicorp-education/learn-vault-secrets-operator/blob/main/vault/vault-auth-static.yaml) the creation of a `VaultAuth` object (specifying how the Secrets Operator should authenticate to Vault - i.e. which Role to use), and that is [not an object provided by the Vault Provider](https://doc.crds.dev/github.com/upbound/provider-vault)[^limited-vault-provider], so I also needed to use the [Kubernetes Provider](https://github.com/crossplane-contrib/provider-kubernetes) to create an arbitrary Kubernetes object as part of the Composition. + +### Actual usage + +After deploying this Composition to my cluster, actual usage was a doddle: + +```bash +$ cat < +deletion_time n/a +destroyed false +version 1 + +$ kubectl get secrets +No resources found in example-namespace-for-crossplane-vault-secrets-demo namespace. + +$ cat <}}) +(remove the slashes - this is so that the commented-out content will not prevent a built while editing) +--> \ No newline at end of file diff --git a/blog/content/posts/vault-secrets-into-k8s.md b/blog/content/posts/vault-secrets-into-k8s.md index 2926183..2585df6 100644 --- a/blog/content/posts/vault-secrets-into-k8s.md +++ b/blog/content/posts/vault-secrets-into-k8s.md @@ -38,6 +38,8 @@ I haven't seen this explicitly stated, but it seems like the intended way to con Gee, if [only](https://www.crossplane.io/) there was a way to manage Vault entities via Kubernetes...😉 +(Update 2024-05-11 - see [here]({{< ref "/posts/base-app-infrastructure" >}}) for a solution!) + # Further thoughts ## Type-safety and tooling