目录
Zachary Nixon

perf: read the informer cache without deep copies in the policy reference resolver (#273)

What type of PR is this? cleanup (performance)

Which issue does this PR fix: Follow-up to #268. Reduces the per-event cost of the pod, namespace and service event handlers.

What does this PR do / Why do we need it: The pod, namespace and service event handlers run on the informer goroutine. For every event they resolve which policies are affected, and the resolver (pkg/resolvers/policies_for_{pod,namespace,service}.go, policies.go) read every policy and namespace through the cache with the default deep copy. #268 removed this work from startup, but every live pod event still pays it.

I took a CPU profile of the pre-#268 startup stall: 10k pods and 10k policies over 50 namespaces, which is the same resolver code a live pod event runs. Of the handler’s CPU:

  • 43% went to NetworkPolicy/Namespace DeepCopy inside cache reads, with most of the GC work on top
  • 32% went to LabelSelectorAsSelector, re-parsing selectors

The resolver only reads what it fetches. Its callers only read the policies it returns: the policy tracker keeps their names, and the queue keeps their keys. So this PR passes client.UnsafeDisableDeepCopy on the resolver’s 15 cache reads. Reconcile-path reads (the endpoints resolvers, finalizers) are unchanged and still deep-copy.

Measured on the pre-#268 build, where the handler cost isn’t hidden by #268’s skip. This is the warm-takeover startup barrier on envtest at 10k pods and 10k policies, with CacheSyncTimeout lifted so the barrier runs to completion:

Shape Default deep copy Without the deep copy Speedup
50 namespaces 50s 28s 1.8×
1 namespace 6m35s 4m05s 1.6×

Screenshot 2026-10-22 at 4 37 41 PM

Each event still lists and evaluates every policy in its namespace and re-parses each selector, so the cost still grows with policies per namespace. Caching parsed selectors is the next largest saving and would be a separate PR.

Safety. The objects are now shared with the cache, so writing to any of them would corrupt the cache for every reader. Three things guard against that:

  • Documented contract: the PolicyReferenceResolver interface comment says its results are read-only.

  • New test, TestPolicyReferenceResolverDoesNotMutateCache (envtest, runs in make test). It loads a workload that reaches every resolver path into a real controller-runtime cache, using the controller’s own cache options. The workload covers pod, namespace and ipBlock peers, namespace selectors, services, ANPs and CNPs. The test snapshots every cached object, then:

  • drives the real pod, namespace, service and policy handlers with create, update and delete events built from the cached objects themselves, as controller-runtime delivers them

    • calls all 9 resolver methods

    It fails with a field-level diff if any cached object changed.

  • Resolver unit tests: the gomock expectations now require the option on each call.

If an issue # is not available please add steps to reproduce and the controller logs: N/A

Testing done on this change:

  • go test ./... with envtest, go vet, gofmt.
  • Checked that the new test catches mutations: I injected three separately and each failed the test with a diff of the changed field. They were a write to a listed policy’s selector map in the resolver, a write to the namespace returned by the resolver’s lookup, and a write to an emitted policy in the policy handler. The test passes on the real code.
  • The barrier measurements above.

Automation added to e2e: TestPolicyReferenceResolverDoesNotMutateCache (runs in make test).

Will this PR introduce any new dependencies?: No

Will this break upgrades or downgrades. Has updating a running cluster been tested?: No. Reads only; no API or behavior change.

Does this PR introduce any user-facing change?: No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

1天前347次提交

Amazon Network Policy Controller for Kubernetes

Controller for Kubernetes NetworkPolicy resources.

Network Policy Controller resolves the configured network policies and publishes the resolved endpoints via Custom CRD (PolicyEndpoints) resource.

Getting Started

When you create a new Amazon EKS cluster, the network policy controller is automatically installed on the EKS control plane. It actively monitors the creation of network policies within your cluster and reconciles policy endpoints. Subsequently, the controller instructs the node agent to create or update eBPF programs on the node by publishing pod information through the policy endpoints. Network policy controller configures policies for pods in parallel to pod provisioning, until then new pods will come up with default allow policy. All ingress and egress traffic is allowed to and from the new pods until they are reconciled against the existing policies. To effectively manage network policies on self-managed Kubernetes clusters, you need to deploy a network policy controller on a node.

Stay tuned for additional instructions for installing Network Policy Controller on nodes. The controller image is published to AWS ECR.

The controller does not require any IAM policies. It does not make AWS API calls.

Prerequisites

  • Kubernetes Version - 1.25+
  • Amazon VPC CNI version - 1.14.0+

Deploy Controller on Dataplane for Development Testing

To deploy the network policy controller on dataplane nodes for development and testing:

  1. Deploy the controller: This will deploy the image specified in helm chart
    make deploy-controller-on-dataplane
    If want to deploy a custom image, you can use the cmd
    make deploy-controller-on-dataplane NP_CONTROLLER_IMAGE=<your-image-repository> NP_CONTROLLER_TAG=<your-image-tag>
    Verify the image deployed
    kubectl get deployment amazon-network-policy-controller-k8s -n kube-system | grep -i image

Optional steps (only needed if using custom images with additional CRDs/permissions):

  1. Apply updated RBAC permissions:

    kubectl apply -f config/rbac/role.yaml
  2. Apply latest CRDs:

    kubectl apply -f config/crd/bases/
  3. Restart controller to pick up new permissions:

    kubectl rollout restart deployment/amazon-network-policy-controller-k8s -n kube-system

Verify deployment:

kubectl get deployment amazon-network-policy-controller-k8s -n kube-system
kubectl logs deployment/amazon-network-policy-controller-k8s -n kube-system

Considerations

NetworkPolicy podSelector must match the target Service’s selector

When a pod reaches another pod through its Service ClusterIP, the network policy controller resolves the traffic through the Service IP (pre-DNAT). To allow this traffic with a NetworkPolicy, the podSelector in the policy must select pods using the same labels the Service uses in its spec.selector.

Pods often carry several labels, and a Service may select them by only one of those labels. If a NetworkPolicy targets the pod using a different label than the one the Service selects on, traffic sent to the ClusterIP is not matched and is denied, even though the label technically matches the destination pod. Traffic sent directly to the pod IP is unaffected. See issue #224 for background.

Security Disclosures

If you think you’ve found a potential security issue, please do not post it in the Issues. Instead, please follow the instructions here or email AWS security directly.

Contributing

See CONTRIBUTING for further details.

License

This project is licensed under the Apache-2.0 License.

邀请码
    Gitlink(确实开源)
  • 加入我们
  • 官网邮箱:gitlink@ccf.org.cn
  • QQ群
  • QQ群
  • 公众号
  • 公众号

版权所有:中国计算机学会技术支持:开源发展技术委员会
京ICP备13000930号-9 京公网安备 11010802047560号