Skip to main content

What is a Zone

A Zone is used to separate different teams or products that use the same Kubernetes Cluster. Improved secuity through access permissions, policies, network rules and dedicated compute resources will result in a far better isolation than just using Namesapces. The main reson for using Zones is security and cost saving.

A Zone contains Namespaces. A Zone can create Kubernetes Node Pools(In AWS NodeGroups). The Pods that are assigned to a Namespace that is part of a Zone will automatically get appropriate NodeSelectors and are assigned to the correct Node Pool. The PSA of the Namesapce is controlled by the Zone, preventing any privileged containers to be started. The PSA enforcment level is configurable on the cluster level. Since a Zone can contain multiple Namespaces and Node Pools then it is also possible to specify the Nodes Pools to be used by each Namesapce in a Zone. Per Pod Affinity is also allowed. For the user it is convinient since the required affinity rules are applied automatically, unless the user wants to control it themselves. In the latter case Validations are in place to enforce rules.

A Zone will also allow network traffic between the Namesapces that belong to the same Zone. When Different Zone Namespaces need to communicate they have to allow incoming connections using normal Kubernetes NetwokPolicy objects. Only NetworkPolocy ingress needs to be added, egress is open by default.

When Ingress objects are created then the needed NetworkPolicy for the LoadBalancer to access the Service Pods is automatically created.

Each Zone will also create an ArgoCD Project that has permissions to deploy resources to the Namesapces of the Zone. Allowing the use of "app-of-apps" for each team while containing them in the designated Zone.

Zones can also assign Contributor groups to allow access only to Namespaces in the Zone. This access is granted in ArgoCD and Kubernetes API.

Example​

This is an example of how to create a zone. This will create two Node Groups in AWS "default" and "myspot".

It is a good practice to manage zones using GitOps methodology, similar to how applications are deployed. For better organization, consider creating a dedicated git repository and ArgoCD project specifically for zones.

Git repository with the zone manifest or create zone using kubectl.

# Example zone
apiVersion: tenancy.entigo.com/v1alpha1
kind: Zone
metadata:
  name: example-zone
  labels:
    tenancy.entigo.com/default-zone: "true"
spec:
  appProject:
    contributorGroups:
      - 123456789-1234-1234-1234-123456789 # AWS IAM Group ID
  namespaces:
    # Deprecated, please create namesapces separately and assign them to corect zone using labels. Kubernetes namespaces in this zone.
    - name: example-namespace
  pools:
    - name: default
      requirements:
        - key: instance-type
          values:
            - t3.large
        - key: capacity-type
          value: ON_DEMAND
        - key: min-size
          value: 1
        - key: max-size
          value: 2
    - name: myspot
      requirements:
        - key: instance-type
          values:
            - t3.large
        - key: capacity-type
          value: SPOT
        - key: min-size
          value: 2
        - key: max-size
          value: 2

Create a Zone using ArgoCD​

1. Create read-only token in GitLab​

Settings -> Access Tokens -> Add new token

Add token name: argocd-read-token

Select Expiration date

Select read_repository scope

Click Create project access token

Refer to GitLab documentation for more information.

2. Connect repository in ArgoCD​

Settings -> Repositories -> Connect repo

Choose your connection method: VIA HTTPS

Type: git

Name: example-zone

Project: zone

Repository URL: <repository-url-that-contains-the-zone-manifest>

Username: argocd-read-token

Password: <argocd-read-token-password>

Click Connect

Refer to ArgoCD documentation for more information.

3. Create new ArgoCD application in ArgoCD​

Applications -> + NEW APP

Application name: example-zone

Project Name: zone

Source -> Repository URL: <repository-url-that-contains-the-zone-manifest>

Source -> Revision: HEAD

Source -> Path: .

Destination -> Cluster URL: https://kubernetes.default.svc

Click Create

Refer to ArgoCD documentation for more information.

4. Sync ArgoCD application​

Applications -> example-zone

Click Refresh and Sync

5. Result​

Assign a Namespace to a Zone​

$ kubectl create ns example-ns

Since the "example-zone" zone has the label tenancy.entigo.com/default-zone with value true. Then th Namesapce is automatically assigned to that Zone. If no Zone has this label, then the first available Zone is chosen.

To change the Zone of the Namesapace example-ns to the Zone another-zone change the tenancy.entigo.com/zone label of the namesapce.

$ kubectl label ns example-ns tenancy.entigo.com/zone=another-zone --overwrite

It is possible to assign a Zone to a Namesapce immediately by defining the labels.

kind: Namespace
metadata:
  name: a2
  labels:
    tenancy.entigo.com/zone: example-zone
    tenancy.entigo.com/pool: myspot

To specify the Pool to use for this namesapce specify the label tenancy.entigo.com/pool. This will assign Pods of this Namesapce to that pool of nodes.

After the Namesapce is created, the permissions are also applied to the Zones ArgoCD Project. Allowing the use of these Namesapces from ArgoCD.

If the user does not want to create Namespaces using kubectl, they can also create a new Application in ArgoCD. When this Application belongs to a Zones project then the destination Namesapce is automatically created and assigned to the same Zone.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: a3
  namespace: a-apps
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: a3
  project: example-zone
  source:
    ...

If the "a3" Namespace did not exist, then it will be created and assigned to the example-zone Zone.