Kubernetes Admission Controllers: Validating vs. Mutating Webhooks

Admission controllers intercept Kubernetes API requests after authentication and authorization but before an object is persisted. They can change an incoming object, reject it, or both.

Admission webhooks extend this mechanism with HTTPS callbacks. The built-in MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission plugins call endpoints registered through the admissionregistration.k8s.io/v1 API group.

As of Kubernetes 1.37, both plugins are enabled by default. Their configuration API has been stable since Kubernetes 1.16; the older admissionregistration.k8s.io/v1beta1 version stopped being served in Kubernetes 1.22.

Where webhooks run in the request path

The simplified write path is:

Authenticate
    ↓
Authorize
    ↓
Run mutating admission controllers and webhooks
    ↓
Validate the resulting API object
    ↓
Run validating admission controllers and webhooks
    ↓
Persist the object

Admission applies to operations represented as CREATE, UPDATE, DELETE, or CONNECT. It does not apply to GET, LIST, or WATCH, because read requests bypass admission control.

This ordering explains the core distinction:

• A mutating webhook can modify an object before validation.
• A validating webhook evaluates the object after all admission mutations have finished.
• Either type can reject a request by returning allowed: false.

A policy that must inspect the final admitted form of an object belongs in the validating phase. A mutating webhook cannot assume that its output is final because later mutators can still change it.

Validating and mutating webhooks compared

Aspect	Mutating webhook	Validating webhook
Registration object	MutatingWebhookConfiguration	ValidatingWebhookConfiguration
Admission plugin	MutatingAdmissionWebhook	ValidatingAdmissionWebhook
Primary purpose	Default, normalize, or inject fields	Enforce policy and reject invalid requests
Invocation phase	Before validating admission	After all admission mutations
Execution	Matching webhooks are called serially	Matching webhooks are called in parallel
Response	allowed, optionally patchType and patch	allowed, optionally status, warnings, and audit annotations
Supported mutation format	Base64-encoded RFC 6902 JSON Patch	None
Reinvocation	reinvocationPolicy: Never or IfNeeded	Not applicable
Typical uses	Sidecar injection, default labels, resource defaults	Image restrictions, naming rules, ownership checks

Serial invocation makes mutating webhooks especially sensitive to latency. Several two-second mutators can add their delays to the same request, while validating calls can overlap.

The AdmissionReview protocol

The API server sends an HTTP POST with an AdmissionReview object from the admission.k8s.io API group. Do not confuse this payload API with the admissionregistration.k8s.io group used to register webhooks.

Important request fields include:

• request.uid: unique identifier that the response must copy exactly.
• request.operation: CREATE, UPDATE, DELETE, or CONNECT.
• request.kind and request.resource: the object kind and REST resource being processed.
• request.subResource: a subresource such as status or scale.
• request.userInfo: authenticated username, UID, groups, and additional attributes.
• request.object: the proposed object; null for DELETE.
• request.oldObject: the existing object; null for CREATE.
• request.options: the relevant create, update, or delete options.
• request.dryRun: whether the request must avoid persistence and side effects.

The webhook should return HTTP 200 with an AdmissionReview in the same version it received. At minimum, the response needs the copied uid and an allowed boolean. A non-2xx response is treated as a webhook call failure, not as a deliberate admission denial.

Mutating response

A mutating webhook can attach a base64-encoded JSON Patch. JSONPatch is currently the only supported patchType.

This response sets spec.replicas to 3:

{
  "apiVersion": "admission.k8s.io/v1",
  "kind": "AdmissionReview",
  "response": {
    "uid": "request-uid-copied-verbatim",
    "allowed": true,
    "patchType": "JSONPatch",
    "patch": "W3sib3AiOiAiYWRkIiwgInBhdGgiOiAiL3NwZWMvcmVwbGljYXMiLCAidmFsdWUiOiAzfV0="
  }
}

The decoded patch is:

[
  {
    "op": "add",
    "path": "/spec/replicas",
    "value": 3
  }
]

Patch only the fields the webhook owns. When patching a label whose key contains /, escape that character as ~1 in the JSON Pointer. For example, the label platform.example.com/owner uses the path /metadata/labels/platform.example.com~1owner.

Validating response

A validating webhook accepts or rejects without changing the object:

{
  "apiVersion": "admission.k8s.io/v1",
  "kind": "AdmissionReview",
  "response": {
    "uid": "request-uid-copied-verbatim",
    "allowed": false,
    "status": {
      "code": 403,
      "message": "Deployment replicas must be between 2 and 50"
    }
  }
}

Both webhook types can return warnings. Kubernetes presents them to clients through HTTP warning headers with code 299.

Registering both webhook types

The following configurations send apps/v1 Deployment creates and updates to two paths on the same in-cluster Service. The mutator fails open; the validator fails closed and verifies the resulting object.

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  name: deployment-defaults.platform.example.com
webhooks:
  - name: mutate-deployments.platform.example.com
    admissionReviewVersions:
      - v1
    sideEffects: None
    failurePolicy: Ignore
    matchPolicy: Equivalent
    reinvocationPolicy: IfNeeded
    timeoutSeconds: 2
    clientConfig:
      service:
        namespace: webhook-system
        name: deployment-policy
        path: /mutate
        port: 443
      caBundle: BASE64_ENCODED_PEM_CA_BUNDLE
    namespaceSelector:
      matchExpressions:
        - key: kubernetes.io/metadata.name
          operator: NotIn
          values:
            - kube-system
            - webhook-system
    rules:
      - operations:
          - CREATE
          - UPDATE
        apiGroups:
          - apps
        apiVersions:
          - v1
        resources:
          - deployments
        scope: Namespaced
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: deployment-policy.platform.example.com
webhooks:
  - name: validate-deployments.platform.example.com
    admissionReviewVersions:
      - v1
    sideEffects: None
    failurePolicy: Fail
    matchPolicy: Equivalent
    timeoutSeconds: 2
    clientConfig:
      service:
        namespace: webhook-system
        name: deployment-policy
        path: /validate
        port: 443
      caBundle: BASE64_ENCODED_PEM_CA_BUNDLE
    namespaceSelector:
      matchExpressions:
        - key: kubernetes.io/metadata.name
          operator: NotIn
          values:
            - kube-system
            - webhook-system
    rules:
      - operations:
          - CREATE
          - UPDATE
        apiGroups:
          - apps
        apiVersions:
          - v1
        resources:
          - deployments
        scope: Namespaced

The caBundle placeholder must be replaced with the base64-encoded PEM certificate authority that signed the webhook server certificate. For this Service reference, the certificate normally covers deployment-policy.webhook-system.svc.

The example deliberately lists deployments, not deployments/scale. Subresources must be matched explicitly. Other useful resource patterns include:

• * for all resources but no subresources.
• */* for all resources and subresources.
• pods/* for all Pod subresources.
• */status for every status subresource.

Matching requests precisely

Every webhook rule combines operations, apiGroups, apiVersions, resources, and scope. A request is sent when it matches every dimension in any one rule.

Use the empty string for the core API group:

rules:
  - operations: ["CREATE"]
    apiGroups: [""]
    apiVersions: ["v1"]
    resources: ["pods"]
    scope: Namespaced

Avoid cluster-wide wildcard rules unless the webhook genuinely understands every matching kind and subresource.

API-version matching

matchPolicy accepts two values:

• Exact calls the webhook only for the group and version explicitly listed.
• Equivalent also matches equivalent served versions and converts the object to a version declared by the webhook.

Equivalent is the default and is normally the safer upgrade behavior. Register stable resource versions such as apps/v1; once an obsolete API version is no longer served, Kubernetes no longer considers it equivalent.

Selectors and CEL conditions

namespaceSelector filters namespaced resources using labels on their namespace. For a request involving a Namespace object, it evaluates that object's labels. It has no filtering effect on other cluster-scoped resources.

objectSelector checks both the new and old object's labels and matches if either satisfies the selector. It is appropriate for opt-in behavior, but usually unsafe as an enforcement boundary: a user who can change labels might bypass the webhook.

For finer filtering, matchConditions contains CEL expressions. This field has been stable since Kubernetes 1.30 and permits up to 64 conditions per webhook. Conditions can inspect object, oldObject, request, authorizer, and authorizer.requestResource.

All conditions must evaluate to true for the endpoint to be called. If evaluation produces an error and no condition is false, failurePolicy determines whether the request is rejected or the webhook is skipped.

Failure policy is an availability decision

failurePolicy controls call failures such as timeouts, connection errors, non-2xx responses, malformed responses, and unsupported mutation patches:

• Fail rejects the API request.
• Ignore skips the failed webhook and continues admission.

The default is Fail. It is important to set the field explicitly so the intended availability and security behavior is visible during review.

failurePolicy does not override an intentional rejection. If a reachable webhook returns a valid response with allowed: false, Kubernetes rejects the request even when the configuration says failurePolicy: Ignore.

A robust pattern is:

1. Let a non-security-critical mutator fail open with Ignore.
2. Validate the final required state with a validating webhook or ValidatingAdmissionPolicy.
3. Use Fail for enforcement only when blocking API writes during an outage is an acceptable tradeoff.

timeoutSeconds can be between 1 and 30 seconds and defaults to 10. Admission logic should normally complete in milliseconds, so an explicit short timeout such as two seconds is preferable.

Reinvocation and idempotence

Mutating webhook order is not a stable contract. Later mutators may overwrite or build on earlier changes.

reinvocationPolicy: IfNeeded allows Kubernetes to call a mutating webhook again if another admission plugin subsequently changes the object. The default, Never, prevents more than one invocation during an admission evaluation.

Even with IfNeeded, Kubernetes does not guarantee exactly one additional invocation or another pass after every later change. Therefore:

• Make every mutating webhook idempotent.
• Make the collection of mutators idempotent when applied together.
• Do not generate a different sidecar name, timestamp, or random value on each call.
• Check whether an injected container, volume, label, or finalizer already exists.
• Validate critical mutations during the later validating phase.

A good mutator turns an object into a desired form. Running it again on that form should produce no additional patch.

Production hardening

Limit the blast radius

Scope rules to the required resources, operations, and namespaces. Exclude the namespace hosting the webhook so that an unavailable webhook cannot prevent replacement of its own Pods.

Avoid intercepting control-plane dependencies such as coordination.k8s.io Lease updates. Also restrict RBAC access to the mutatingwebhookconfigurations and validatingwebhookconfigurations resources in admissionregistration.k8s.io.

Webhooks are never invoked for changes to MutatingWebhookConfiguration or ValidatingWebhookConfiguration objects. This prevents a broken webhook from making its own configuration impossible to repair.

Kubernetes 1.37 also enables the beta ExcludeAdmissionWebhookVirtualResources behavior by default. Webhooks no longer intercept non-persisted authentication and authorization review resources such as TokenReview, SubjectAccessReview, SelfSubjectAccessReview, and SelfSubjectRulesReview, even if a rule names them.

Design for availability

Run multiple webhook replicas behind a ClusterIP Service and distribute them across nodes or failure domains. Avoid synchronous calls to unrelated external services in the admission path; use bounded caches where possible.

Deploy the server before creating its webhook configuration. Roll out initially to a test namespace, observe errors and latency, and then expand the selector.

Avoid side effects

For admissionregistration.k8s.io/v1, sideEffects is required and must be None or NoneOnDryRun.

Prefer None. If a webhook performs out-of-band work, it must suppress that work during dry-run requests and use NoneOnDryRun. It also needs reconciliation because a later admission stage or storage failure can still prevent the reviewed object from being persisted.

Test and observe

Server-side dry run exercises admission without persistence:

kubectl apply --dry-run=server -f deployment.yaml

At audit level Metadata or higher, mutating invocations can appear under annotations named like:

mutation.webhook.admission.k8s.io/round_0_index_0

At audit level Request or higher, applied patches can appear under:

patch.webhook.admission.k8s.io/round_0_index_0

The API server also exposes apiserver_admission_webhook_rejection_count, labeled with the webhook name, operation, admission type, error type, and rejection code. Monitor it alongside webhook latency, timeout rates, endpoint availability, and API server request latency.

When an in-process policy is better

An external webhook is not always necessary.

ValidatingAdmissionPolicy, stable since Kubernetes 1.30, evaluates CEL rules inside the API server. MutatingAdmissionPolicy, first available in 1.30 and stable since Kubernetes 1.36, performs in-process CEL-driven mutations using apply configurations or JSON patches. Both use policy and binding resources in admissionregistration.k8s.io/v1.

Prefer these policy APIs when CEL can express the rule. They avoid network callouts, certificates, webhook deployment dependencies, and much of the associated latency. A webhook remains appropriate when admission requires arbitrary program logic, integration with an external system, or behavior that the policy APIs cannot express.

References

• Kubernetes: Admission Control in Kubernetes - kubernetes[.]io/docs/reference/access-authn-authz/admission-controllers/
• Kubernetes: Dynamic Admission Control - kubernetes[.]io/docs/reference/access-authn-authz/extensible-admission-controllers/
• Kubernetes: Admission Webhook Good Practices - kubernetes[.]io/docs/concepts/cluster-administration/admission-webhooks-good-practices/
• Kubernetes: ValidatingWebhookConfiguration API - kubernetes[.]io/docs/reference/kubernetes-api/admissionregistration/validating-webhook-configuration-v1/
• Kubernetes: MutatingWebhookConfiguration API - kubernetes[.]io/docs/reference/kubernetes-api/admissionregistration/mutating-webhook-configuration-v1/
• Kubernetes: Validating Admission Policy - kubernetes[.]io/docs/reference/access-authn-authz/validating-admission-policy/
• Kubernetes: Mutating Admission Policy - kubernetes[.]io/docs/reference/access-authn-authz/mutating-admission-policy/
• Kubernetes: Deprecated API Migration Guide - kubernetes[.]io/docs/reference/using-api/deprecation-guide/
• Kubernetes: Releases - kubernetes[.]io/releases/