Azure Self Hosting

This section explains how to setup and create resources for self-hosting Zerve on a pre-existing AKS cluster in Azure.

---

General Infrastructure

At this time, Azure Self-Hosting is possible only with an existing AKS cluster.

Storage Account and blob container

Zerve requires blob storage in order to store block state and user files. We recommend creating a separate storage account and container for this purpose using the installation steps below.

Artifact Registry

Zerve needs a container registry to store Docker images. We recommend creating a separate one for this purpose using the installation steps below.

IAM Identities

Application Service Principal

An identity that represents Zerve application in your Azure tenant.

Created by registering Zerve's multi-tenant app in your tenant. The service principal will be authorized to perform operations within your Azure account, such as scheduling compute jobs or managing canvas storage.

Execution Managed Identity

A user-assigned managed identity for compute jobs that Zerve schedules to execute code blocks.

This identity can be used to grant users' code blocks access to other Azure resources in your tenant.

Build Managed Identity

A user-assigned managed identity identity for build jobs.

This identity will be used to grant build jobs access to the blob container and container registry.

Azure Kubernetes Service

Zerve can use your existing AKS cluster to schedule build and compute jobs.

Cluster requirements:

• Version 1.28 or higher.

• Enabled Workload Identity

• Enabled Microsoft Entra integration. Zerve will authenticate to the cluster using aforementioned service principal, whereas authorization will be carried out by Kubernetes.

---

Setup Instructions

Prerequisites

• azcli

• kubectl pointing to the existing AKS cluster used for Zerve

• helm

Cloud infrastructure

You can use az CLI and Azure Resource Manager to provision the necessary infrastructure.

Point your CLI to the subscription where Zerve resources should be created

az subscription --set <subscription id>

Create a service principal in your tenant and get the application ID

export ZERVE_SP_ID=$(az ad sp create --id 05b06594-79df-44b0-8554-3e0a8a2c71be --query id --output tsv)

Create a resource group for Zerve's infrastructure. For best performance, colocate it with your existing AKS cluster.

az group create --name zerve --location <location>

Set env vars referencing the existing AKS cluster by resource group name and cluster name

export CLUSTER_RG=
export CLUSTER_NAME=

Deploy Zerve's ARM (Azure Resource Manager) template as a deployment stack:

az stack group create \
  --name zerve-infra \
  --resource-group zerve \
  --template-uri https://zerve-deployment-templates.s3.eu-west-1.amazonaws.com/azure/zerve.json \
  --action-on-unmanage 'deleteAll' \
  --deny-settings-mode 'none' \
  --parameters aksClusterRg=$CLUSTER_RG aksClusterName=$CLUSTER_NAME zerveSpId=$ZERVE_SP_ID

Setup RBAC in your cluster by installing our helm chart:

export BUILD_CLIENT_ID=$(az stack group show --name zerve-infra --resource-group zerve --query 'outputs.buildIdentityClientId.value' --output tsv)
export EXECUTION_CLIENT_ID=$(az stack group show --name zerve-infra --resource-group zerve --query 'outputs.executionIdentityClientId.value' --output tsv)

cat <<- EOF > /tmp/values.yaml
  builder:
    # Associate "builder" k8s SA with the corresponding IAM SA
    serviceAccount:
      annotations:
        azure.workload.identity/client-id: $BUILD_CLIENT_ID
  executor:
    # Associate "executor" k8s SA with the corresponding IAM SA
    serviceAccount:
      annotations:
        azure.workload.identity/client-id: $EXECUTION_CLIENT_ID
  scheduler:
    # Grant Zerve application full access to the namespace
    user:
      enabled: true
      name: $ZERVE_SP_ID
      
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: admin
EOF

helm install zerve oci://public.ecr.aws/x8w6c8k3/helm/zerve \
  -n zerve \
  -f /tmp/values.yaml \
  --create-namespace

---

Zerve Organization Self-Hosting Settings

Navigate to your organization's self-hosting settings in Zerve app.

Fill out the form with the following values:

• Tenant ID: tenant ID of Zerve service principal

• Storage Account Name

• Bucket Name

• Image Registry: login server of the container registry

• Namespace: zerve

• Endpoint: Control plane endpoint of AKS cluster

• Service Account Token: leave empty

Exact values for all of the above should be available as deployment stack outputs. To find them, run the following command:

az stack group show --name zerve-infra --resource-group zerve --query 'outputs' 

• Certificate Authority Data: base64-encoded certificate used to verify the self-signed certificate presented by AKS cluster's control plane

  • Run the following command to find it:

    kubectl config view \ 
      -o jsonpath="{.clusters[?(@.name == \"$CLUSTER_NAME\")].cluster.certificate-authority-data}" \ 
      --raw