Kyverno

Kyverno is the cluster admission policy engine. It evaluates Pods and related controllers against Git-managed policies and writes PolicyReports. The supported install is three Argo CD Applications — engine, policies, and Policy Reporter — reconciled from homelabs and homelabs-private. Do not helm install Kyverno from a workstation.

This layout pins engine chart kyverno 3.9.0 (app v1.19.0) and policies chart kyverno-policies 3.9.0 on their Application targetRevision values, and Policy Reporter chart policy-reporter 3.9.1. Patch bumps within those series follow the same path; a new major needs a deliberate re-pin.

v1 policies run in Audit. They record violations and do not block admission. A Pod that would fail baseline still creates. Flipping any policy to Enforce is a later change, after reports are clean and exceptions exist for workloads that still need to violate.

Role in the stack

#

Kyverno sits on CREATE and UPDATE for matched resources. The engine webhooks use failurePolicy: Fail, so if admission pods are down, matching requests fail closed — except in namespaces the webhook selector excludes. Public values set three admission replicas so that Fail path does not depend on a single pod. Background, reports, and cleanup controllers stay at one replica. Controllers are stateless; there is no Longhorn claim, and reports-server stays off so the chart does not emit a PVC.

Household application namespaces stay in scope. Platform namespaces that already run privileged or host-path workloads are excluded from the webhook so a Kyverno outage cannot stall node heartbeats or those controllers: kube-system, Longhorn, KubeVirt, MetalLB, Traefik, cert-manager, kube-bench, GPU Operator, and Rancher/Fleet system namespaces. Kubernetes reserved namespaces kube-node-lease and kube-public are in that same NotIn list, and Kyverno’s own namespace is excluded as well (excludeKyvernoNamespace: true). Helm replaces the webhook values: list rather than merging it; public values own the full exclude list. A private overlay that sets only extra names would drop kube-system. The shipped overlay is comments only.

Prometheus already scrapes cluster-wide ServiceMonitors. Kyverno enables four of them (admission, background, reports, cleanup). Policy Reporter enables one. Grafana’s sidecar loads the Kyverno dashboard ConfigMap because it is labeled grafana_dashboard: "1". There is no Kyverno PrometheusRule in v1.

GitOps layout

#

Independent Applications let you prune policies without tearing down webhooks.

Wave -1 is after External Secrets Operator (-2) and before household apps (1). Policies and the reporter wait at wave 0 until the engine and CRDs exist. Each Application sets CreateNamespace=true and ServerSideApply=true (large CRDs, same reason as Prometheus). Automated prune and selfHeal are on.

The engine Application is multi-source: Helm repo https://kyverno.github.io/kyverno/, public values, and the private overlay. CreateNamespace creates namespace kyverno. Policies add the extras path kyverno-policies/manifests from public git. Policy Reporter uses Helm repo https://kyverno.github.io/policy-reporter plus public values. Policies and Policy Reporter have no private overlay in v1.

Do not put live hostnames or secrets in homelabs/kyverno/. Site values live in homelabs-private/clusters/<cluster>/.

AppProject

#

Allow Helm repos https://kyverno.github.io/kyverno/ and https://kyverno.github.io/policy-reporter, destination namespaces kyverno and policy-reporter, and the cluster-scoped kinds the charts create (ValidatingPolicy, ClusterCleanupPolicy, ClusterPolicyReport, webhooks, CRDs, ClusterRoles). Keep the whitelist additive. The homelab AppProject is applied outside app-of-apps. Committing the YAML is not enough. Apply the live project before you expect Argo to accept the Applications:

kubectl apply -f homelabs-private/argocd/projects/homelab.yaml

Without those destinations and sourceRepos, sync fails because project homelab does not allow the charts or namespaces.

First sync

#

Do this after the public homelabs trees for Kyverno, kyverno-policies, and policy-reporter are on main. Order matters: AppProject before a successful Application sync, engine Healthy before policies.

1. Apply the AppProject (command above) so namespaces kyverno and policy-reporter and both Helm repos are allowed.
2. Commit the private Applications and overlay. App-of-apps discovers them; wait until Application kyverno is Synced/Healthy, then sync kyverno-policies and policy-reporter.
3. Confirm CRDs, admission pods, and ValidatingPolicies. Policy Reporter stays ClusterIP — do not add a Cloudflare Public Hostname for Kyverno or the reporter.

kubectl -n argocd get application kyverno kyverno-policies policy-reporter
kubectl -n kyverno get pods
kubectl get validatingpolicy
kubectl -n kyverno get pvc

Expect at least two admission-controller pods Ready (the pin is three). The PVC list in kyverno stays empty. ValidatingPolicies should show Audit, not Enforce.

Baseline Pod Security and extras

#

Official chart kyverno-policies ships Kubernetes Pod Security Standards as CEL ValidatingPolicy objects. v1 sets podSecurityStandard: baseline and validationFailureAction: Audit. Restricted stays off. Baseline is the Kubernetes minimally acceptable set: no privileged containers, no host namespaces, hostPath, or host ports, and restricted capabilities, seccomp, and sysctls.

Audit does not block. Kyverno writes a PolicyReport, and background scans fill reports for objects already on the cluster. Argo syncs and rolling Deployments are not Fail-closed by these policies. Enforce is a later change; do not flip validationFailureAction in this install.

The same platform namespaces excluded from the webhook are listed again in vpolExclude.excludeNamespaces so background scans do not treat Longhorn, KubeVirt, MetalLB, Rancher, and the rest as in-scope. Household app namespaces stay in scope; that is the point of installing Kyverno.

A closed companion set lives in homelabs/kyverno-policies/manifests/extras/ — pinned copies, not a remote kustomize URL at sync time. Adding a new policy is a new change.

The four ValidatingPolicy extras are Audit, same as baseline. require-requests-limits is noisy by design; keep it Audit until you are ready to chase missing requests in application charts.

Policy Reporter

#

Application policy-reporter installs chart 3.9.1 into namespace policy-reporter with the Kyverno plugin and UI enabled. The UI Service is ClusterIP on port 8080. There is no Ingress, HTTPRoute, LoadBalancer, or tunnel hostname.

Open the UI from a workstation with a port-forward:

kubectl -n policy-reporter port-forward svc/policy-reporter-ui 8082:8080

Then browse http://127.0.0.1:8082. Metrics are on for the existing Prometheus; there is no public /metrics hostname.

Rollback

#

Revert git rather than helm uninstall. Disable or delete Applications kyverno-policies and policy-reporter first, then delete Application kyverno. The engine chart enables webhooksCleanup so pre-delete Jobs remove validating and mutating webhooks when the release goes away. Do not delete ValidatingWebhookConfiguration objects by hand while Kyverno pods are still expected to serve.

PolicyReports remain until you remove the CRDs. Leave CRDs until you no longer need the reports. Prefer prune via Argo over a workstation Helm uninstall.

See also

#

• Argo CD — AppProject, app-of-apps, sync
• Prometheus — ServiceMonitors and Grafana sidecar
• External Secrets Operator — cluster-scoped operator with ServerSideApply