Zone Management

Reference for configuring Zones and the cluster-wide rules that apply to every Zone. For how to create a Zone, see Create a Zone. For the concept, see Zone.

Assigning Permissions

User groups are granted roles through roleMapping, which can be set in two places:

• Global — set in the platform-api Helm chart values. Applies across all Zones and their Namespaces in the cluster.
• Per-Zone — set in the Zone object spec. Applies only to that Zone's Namespaces.

A role's permissions via the Kubernetes API (kubectl) are nearly identical to those granted in the Zone's ArgoCDProject, though technical limitations keep them from being exactly equal.

Detailed Role Descriptions

Observer — read-only access to applications; can read objects but not modify them.

• Reads these cluster-scoped objects: namespaces, nodes, persistentvolumes, storageclasses, ingressclasses, ingressclassparams.elbv2.k8s.aws, zones, nodegroups.eks.aws.upbound.io.
• Reads all namespace-scoped objects in the Zone's Namespaces.

Contributor — deploy and manage applications within the Zone boundary. Includes everything Observer can do, plus:

• Create new Namespaces in Zones through ArgoCDApplications (not kubectl); the Namespace is assigned to the Zone whose ArgoCDProject it was created in.
• Read, create, edit, and delete all namespace-scoped objects in the Zone's Namespaces — including ArgoCDApplication objects, but only within the Zone's ArgoCDProject.
• Configure ArgoCDApplications and repositories.

Maintainer — configure Zone objects and let Contributors deploy within the Zone boundary. Includes all Contributor permissions, plus:

• Read, create, edit, and delete Zones and Namespaces.
• Assign user groups to Contributor permissions in a Zone, via the Zone object's spec.roleMapping field.

A Maintainer is less powerful than a cluster administrator and cannot manage the infralib Zone or its Namespaces. Note, however, that because of spec.clusterPermissions in the Zone object, a Maintainer can escalate to cluster administrator.

Future state: these roles' capabilities and existence are currently fixed by the platform-api. In the future it should be possible to tune the permissions and which roles exist — which is why roleMapping uses roleRef as a free text field.

Example for Global/Cluster-wide roleMappings

Set roles that apply across every Zone in the cluster, in the platform-apis environmentConfig.

steps:
  ...
  - name: apps
    type: argocd-apps
    modules:
    ...
    - name: platform-apis
      source: platform-apis
      inputs:
        platform-apis:
          zone:
            environmentConfig:
              roleMapping:
                - roleRef: maintainer
                  groups: ["maintainer-group"]
                - roleRef: observer
                  groups: ["observer-group"]

Example for Per-Zone roleMappings

Set roles that apply only to a single Zone, in that Zone object's spec.roleMapping.

apiVersion: tenancy.entigo.com/v1alpha1
kind: Zone
metadata:
  name: example-zone
spec:
  roleMapping:
    - groups:
        - contributor-group
      roleRef: contributor
    - groups:
        - observer-group
      roleRef: observer

Example for Tuning ArgoCDProject Permissions

Use platform-apis Helm values to control which resources ArgoCD may deploy — appProject sets the whitelist and blacklist on each Zone's own ArgoCDProject, and clusterResourceWhitelist sets it on the cluster-wide zone project.

        platform-apis:
          zone:
            environmentConfig:
              appProject:
                namespaceResourceBlacklist: #Affects the namespace resource blacklist of the Zone ArgoCDProject. Example contains default values.
                - group: '*.m.upbound.io'
                  kind: '*'
                namespaceResourceWhitelist: []
          clusterResourceWhitelist: #Affects the cluster resource whitelist of the "zone" project. Example contains default values.
          - group: tenancy.entigo.com
            kind: Zone
          - group: eks.aws.upbound.io
            kind: NodeGroup

Namespace Labels and Annotations from an Application

An ArgoCD Application can describe the labels and annotations of its destination Namespace under spec.syncPolicy.managedNamespaceMetadata. ArgoCD itself cannot manage Namespaces through a Zone's ArgoCDProject, so the Zone applies this metadata on ArgoCD's behalf. The field then behaves exactly as it would in a cluster without Zones.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: a3
  namespace: a-apps
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: a3
  project: example-zone
  syncPolicy:
    managedNamespaceMetadata:
      labels:
        team: payments
        tenancy.entigo.com/pool: myspot
        pod-security.kubernetes.io/enforce: restricted
      annotations:
        owner: payments@example.com
  source:
    ...

The a3 Namespace is created, assigned to the example-zone Zone, and labelled and annotated as the Application declares — nothing needs to be applied with kubectl afterwards.

A new Application's metadata is applied as soon as the Application is created. Edits to an existing Application are picked up by a periodic refresh (every 10 minutes by default), so they don't appear on the Namespace right away.

Which Namespace an Application Describes

An Application only describes the Namespace named in its own spec.destination.namespace, and only while that Namespace belongs to the same Zone as the Application's project. Moving a Namespace to another Zone with kubectl therefore stops the Applications of the old Zone from writing to it.

The <zone-name>-apps Namespace is managed by the Zone itself and is never described this way. If several Applications of a Zone deploy to the same Namespace, the first one found wins, so keep the metadata in a single Application per Namespace.

Keys That Are Ignored

The keys that decide where a Namespace belongs cannot be set from an Application, as either a label or an annotation:

Key	Where it is set instead
tenancy.entigo.com/*	With kubectl on the Namespace, see Assign a Namespace to a Zone. This is what decides which Zone a Namespace belongs to. The Pool label is the one exception, see below.
kubernetes.io/metadata.name	By Kubernetes itself.
app.kubernetes.io/managed-by, generate.kyverno.io/*	By the platform, on Namespaces it creates.

These keys are ignored rather than rejected: an Application that carries one is still accepted and the rest of its labels and annotations are applied as usual.

The Pool Label

Unlike the ignored keys above, tenancy.entigo.com/pool is applied. An Application can therefore choose which of its Zone's Pools the Namespace runs on — the same choice setting the label with kubectl gives.

The value must name a Pool of the Namespace's own Zone. The Namespace accepts any value, but the Zone validates it on every Pod: if the name matches no Pool, the Pod is rejected as it is created, with an error listing the valid Pools. ArgoCD surfaces that error on the ReplicaSet, StatefulSet, or Job that tried to create the Pod — not on the Application that set the label.

The Pool only affects Pods created after the label reaches the Namespace; Pods already running are not rescheduled.

Pod Security Labels

The pod-security.kubernetes.io/enforce and pod-security.kubernetes.io/warn labels are applied too. A team can use them to run one of its Namespaces stricter than the cluster's podSecurity floor, without involving the platform team.

Only stricter is allowed. A level looser than the floor is rejected on the Application — when it is created or edited — with an error naming the level the cluster requires. That way the person writing the Application sees the problem, instead of the Namespace silently keeping its old labels.

Removing a Label or Annotation

Deleting a label or annotation from managedNamespaceMetadata does not remove it from the Namespace — the Namespace keeps whatever value it was last given. To actually remove it, delete it from the Namespace with kubectl:

$ kubectl label ns a3 team-

For the same reason, manage a given label or annotation from either the Application or kubectl, not both: as long as it is still listed in the Application, the next refresh restores its value from there.

Disabling Namespace Metadata

Applying Namespace metadata from an Application is enabled for the whole cluster by default. Turn it off, or change the refresh interval, in the platform-api Helm values file. Note that these values sit next to environmentConfig, not inside it.

zone:
  namespaceMetadata:
    enabled: true
    syncInterval: 10m

EnvironmentConfig

EnvironmentConfig is the block of cluster-wide Zone defaults in the platform-apis Helm values, under zone.environmentConfig. A Zone object defines a single team's boundary; EnvironmentConfig holds the settings that apply to every Zone in the cluster.

Configuring these once, centrally, means the platform team sets how strict the cluster is, and developers don't repeat it in each Zone. The settings:

Setting	Default	What it does
podSecurity	baseline	The lowest security level a Namespace is allowed to run at.
granularEgress	false	Whether outgoing traffic rules are shared by the whole cluster or kept per Zone.
granularEgressExclude	[]	Zones that do not have to list their external services.
granularNamespaceNetworkPolicy	false	Whether Namespaces inside the same Zone can talk to each other freely.
createAppsNamespace	true	Creates the <zone-name>-apps Namespace for ArgoCD Applications.
roleMapping	maintainer / observer / contributor	Which user groups get which role in every Zone. See Assigning Permissions.
appProject	see below	What ArgoCD is allowed to deploy into a Zone. See Tuning ArgoCDProject Permissions.
tags	{}	Cloud tags added to Zone resources such as NodeGroups. See Resource Tagging.

podSecurity

Kubernetes Pod Security Admission has three levels: privileged (no restrictions), baseline (blocks the well-known dangerous options) and restricted (also requires containers to run as a non-root user with no extra privileges).

This setting is the floor for the whole cluster. Every Namespace in a Zone is labelled with it, and a Namespace can only be changed to a stricter level, never a looser one:

• baseline (default) — Namespaces run at baseline, and a team may opt into restricted for its own Namespace.
• restricted — every Namespace must run at restricted. Containers that need root or extra Linux capabilities will not start. This is the recommended setting for production clusters, but existing workloads have to be checked first because some off-the-shelf images require root.
• privileged — no floor at all. Not recommended.

granularEgress

This setting decides who an outgoing connection is opened for when a team needs to reach an external address.

• false (default) — outgoing rules are shared by the whole cluster. When one Zone opens an external address, that address is reachable from every other Zone as well.
• true — each Zone gets its own set of outgoing rules. An address opened for Zone A stays closed for Zone B. Teams no longer inherit each other's external dependencies, and it becomes visible which Zone actually needs which external service.

Not your internet firewall. This setting is not what stops your cluster from reaching the internet in the first place. Outgoing traffic leaving the cluster is normally restricted by a firewall in front of the cluster, whether granularEgress is on or off. What this setting changes is only whether the rules inside Kubernetes are cluster-wide or per Zone.

Cost of enabling it. A proxy container is added to every pod in the Zone's Namespaces, so applications need to be restarted. The Namespaces are labelled for that automatically — nothing has to be added by hand. After that, every external service a Zone uses (payment providers, mail services, package registries) has to be listed for that Zone, apart from the defaults every Zone can already reach that the platform opens for everyone.

Opening an external host. Use an Istio ServiceEntry object, created in one of the Zone's own Namespaces — that is what keeps it scoped to this Zone:

apiVersion: networking.istio.io/v1
kind: ServiceEntry
metadata:
  name: stripe-api
  namespace: example-zone-team
spec:
  hosts:
    - api.stripe.com
  ports:
    - number: 443
      name: https
      protocol: TLS
  resolution: DNS
  location: MESH_EXTERNAL

With granularEgress disabled there is no per-Zone scoping, so a ServiceEntry created anywhere in the cluster applies to every Zone.

What every Zone can already reach

Some destinations are opened by the platform for the whole cluster, so no team has to list them:

• The managed data services this platform provisions — PostgreSQL, MariaDB, Valkey, Kafka and RabbitMQ — on the subnets and ports they run on. A Zone reaches the database it created without writing a ServiceEntry for it.
• The AWS service APIs an application normally calls: STS, S3, ECR, KMS, Secrets Manager, Parameter Store, DynamoDB, SQS, SNS, CloudWatch, EC2, and the control plane of each managed data service.

These are shared by the whole cluster rather than scoped per Zone. That is a smaller exception than it looks: all of them are authenticated with IAM, so being able to open a connection is not the same as being allowed to do anything over it — what a Zone may actually do is decided by its own IAM role. Everything else stays per Zone.

granularEgressExclude

A list of Zone names that do not have to list their external services, even while granularEgress is enabled for the rest of the cluster. Use it as an escape hatch for a Zone that hasn't been migrated yet, or one whose workloads legitimately talk to many external services.

Excluding a Zone lifts only the listing requirement — its pods may reach any external address without a ServiceEntry. It does not turn granularEgress off for that Zone:

• The proxy container is still added to every pod in the Zone's Namespaces, so its applications still have to be restarted.
• The Zone still gets everything else the proxy does for its traffic.
• Other Zones are unaffected and stay restricted to the external services they have listed.

granularEgress: true
granularEgressExclude:
  - legacy-zone

Shared destinations

"Shared destinations" are external services opened for every Zone in the cluster — the cluster-wide egress allow-list behind What every Zone can already reach. They are set by the platform-api Helm chart, and a cluster that needs more can extend the list. Note that these values sit next to environmentConfig, not inside it.

zone:
  istioServiceEntries:
    # Added to the destinations that are open by default, rather than replacing them.
    extraServices:
      - name: bedrock
        hosts: ["bedrock-runtime.{region}.amazonaws.com"]
      - name: partner
        hosts: ["api.partner.example.com"]
    extraSubnets:
      - name: partner-database
        cidrs: ["10.90.0.0/24"]
        ports:
          - {number: 5432, name: tcp-postgres, protocol: TCP}

Which of the two to use depends on how the destination is reached, not on who owns it:

• By hostname (extraServices) — for anything reached over TLS. The proxy matches it by the server name in the TLS handshake, so the hostname is the identifier, and any hostname not listed is refused. List each one in full; wildcards are rejected, so a destination reached under several names (an S3 bucket at <bucket>.s3.<region>.amazonaws.com, say) needs every name in use listed.
• By address (extraSubnets) — for plain connections such as a database, where the traffic carries no hostname the proxy can read. The addresses scope the entry, and ports must be listed one by one.

{region} in a hostname is replaced with the cluster's region. Removing an entry from the defaults closes that service for every Zone in the cluster, and a Zone's own ServiceEntry objects are unaffected by any of this — those stay scoped to the Zone they are created in.

granularNamespaceNetworkPolicy

Controls how open the network is inside a Zone.

• false (default) — all Namespaces belonging to the same Zone can reach each other. One team's services can call each other without any extra configuration.
• true — every Namespace is isolated on its own. Even two Namespaces in the same Zone must allow incoming traffic with a standard Kubernetes NetworkPolicy before they can communicate. Traffic within a single Namespace is always allowed.

In both cases, traffic between different Zones always has to be allowed explicitly, and outgoing traffic is unaffected by this setting.

A single Zone can override the cluster-wide value with spec.granularNamespaceNetworkPolicy:

apiVersion: tenancy.entigo.com/v1alpha1
kind: Zone
metadata:
  name: example-zone
spec:
  granularNamespaceNetworkPolicy: true

createAppsNamespace

Creates the <zone-name>-apps Namespace described in The ArgoCD App-of-Apps Pattern. Set it to false if your teams do not use the app-of-apps pattern and you do not want the extra Namespace.

Example

A full environmentConfig block with the common settings and their default values:

steps:
  ...
  - name: apps
    type: argocd-apps
    modules:
    ...
    - name: platform-apis
      source: platform-apis
      inputs:
        platform-apis:
          zone:
            environmentConfig:
              podSecurity: baseline
              granularEgress: false
              granularEgressExclude: []
              granularNamespaceNetworkPolicy: false
              createAppsNamespace: true