# Proxy on-prem Terraform deployment (Azure)

This guide explains how to deploy Espresso AI's Proxy Service on your Azure infrastructure with Terraform.

You can deploy either:

1. In a dedicated VNet that Terraform creates.
2. In an existing VNet that you provide.

## Prerequisites

* Access to an Azure subscription with permissions to create resource groups, VNets, AKS clusters, public IPs, role assignments, user-assigned managed identities, and (optionally) Key Vault, Azure DNS records, and Azure Front Door.
* In the [Espresso AI dashboard](https://dashboard.espressocomputing.com/), go to `Proxy Onboarding` and:
  * Enter your Azure Subscription ID so we can grant ACR access for the Proxy image. We will generate a username and password for you to be able to pull the image from our ACR.
  * Copy your customer name.
  * If running on Azure, copy Espresso AI's Azure Account ID. This is needed for the ACR url.
  * Generate an API key for Espresso API authentication.

## What this module creates

* Resource group (optional) or uses your existing resource group.
* VNet with a node subnet (optional) or uses your existing VNet/subnet.
* AKS cluster with a system-assigned identity, Workload Identity + OIDC issuer enabled, and an autoscaling node pool.
* Static public IP for ingress (placed in the AKS-managed node resource group).
* `ingress-nginx` controller wired to the static public IP, with Azure LB annotations tuned for AKS Standard LB.
* Proxy deployment, service, and HPA in Kubernetes.
* TLS for the Ingress, via either:
  * cert-manager + Let's Encrypt (HTTP-01 by default, DNS-01 via Azure DNS for wildcard hosts), or
  * a bring-your-own `kubernetes.io/tls` secret you pre-create in the proxy namespace.
* Optional Azure DNS A record pointing at the ingress public IP.
* Optional managed API key flow via Azure Key Vault + External Secrets Operator (federated workload identity).
* Optional Azure Front Door fronting the AKS LB (recommended when clients enforce strict OCSP behavior, e.g. Snowflake's connector).

## Example usage

### Dedicated VNet + cert-manager (Let's Encrypt) + Azure DNS record

```hcl
module "proxy_on_prem" {
  source = "github.com/espressocomputing/espresso-ai-proxy-tf//azure?ref=v0.4.0"

  location = "eastus"
  customer = "<Value from Espresso AI dashboard>"

  resource_group_config = {
    create = true
    name   = "espresso-ai-proxy-rg"
  }

  create_dedicated_vnet = true
  vnet_config = {
    address_space    = ["10.240.0.0/16"]
    node_subnet_cidr = "10.240.0.0/22"
  }

  aks_config = {
    api_server_authorized_ranges = ["203.0.113.10/32"]
  }

  proxy_config = {
    repository = "<Espresso AI's Azure Account ID>.azurecr.io/proxy"
    image      = "0.1-dev-91e316fa12478ad0ae77aa320ff60e6ab63627131a914bb2f5c26ef2579b99b4"
    proxy_host = "proxy.customer.example.com"
  }

  ingress_config = {
    enable_ingress    = true
    ingress_host      = "proxy.customer.example.com"
    letsencrypt_email = "ops@customer.example.com"
  }

  dns_config = {
    create_record            = true
    zone_name                = "customer.example.com"
    zone_resource_group_name = "dns-rg"
    record_name              = "proxy.customer.example.com"
  }
}
```

### Existing VNet + bring-your-own TLS secret + managed API key in Key Vault

```hcl
variable "proxy_api_key_value" {
  description = "Managed proxy API key value for Key Vault sync."
  type        = string
  sensitive   = true
}

module "proxy_on_prem" {
  source = "github.com/espressocomputing/espresso-ai-proxy-tf//azure?ref=v0.4.0"

  location = "eastus"
  customer = "<Value from Espresso AI dashboard>"

  resource_group_config = {
    create = false
    name   = "existing-platform-rg"
  }

  create_dedicated_vnet = false
  existing_vnet_config = {
    vnet_id        = "/subscriptions/<sub>/resourceGroups/network-rg/providers/Microsoft.Network/virtualNetworks/platform-vnet"
    node_subnet_id = "/subscriptions/<sub>/resourceGroups/network-rg/providers/Microsoft.Network/virtualNetworks/platform-vnet/subnets/aks-nodes"
  }

  aks_config = {
    api_server_authorized_ranges = ["203.0.113.10/32"]
  }

  proxy_config = {
    repository                     = "<Espresso AI's Azure Account ID>.azurecr.io/proxy"
    image                          = "0.1-dev-91e316fa12478ad0ae77aa320ff60e6ab63627131a914bb2f5c26ef2579b99b4"
    proxy_host                     = "proxy.customer.example.com"
    api_key_secret_mode            = "MANAGED_AZURE_KEY_VAULT"
    api_key_azure_key_vault_secret = "espresso-ai-proxy-api-key"
  }

  proxy_api_key_value = var.proxy_api_key_value

  ingress_config = {
    enable_ingress  = true
    ingress_host    = "proxy.customer.example.com"
    tls_secret_name = "proxy-tls"
  }
}
```

### Wildcard ingress + Azure Front Door + DNS-01 wildcard cert

Use this shape when fronting Snowflake's connector (or any client with strict OCSP behavior). AFD terminates TLS for clients with a DigiCert-issued managed cert, and the LE cert on the AKS LB only secures the AFD-to-origin hop.

```hcl
module "proxy_on_prem" {
  source = "github.com/espressocomputing/espresso-ai-proxy-tf//azure?ref=v0.4.0"

  location = "eastus"
  customer = "<Value from Espresso AI dashboard>"

  aks_config = {
    api_server_authorized_ranges = ["203.0.113.10/32"]
  }

  proxy_config = {
    repository = "<Espresso AI's Azure Account ID>.azurecr.io/proxy"
    image      = "0.1-dev-91e316fa12478ad0ae77aa320ff60e6ab63627131a914bb2f5c26ef2579b99b4"
    proxy_host = "proxy.customer.example.com"
  }

  ingress_config = {
    enable_ingress    = true
    ingress_host      = "*.customer.example.com"
    letsencrypt_email = "ops@customer.example.com"

    front_door = {
      enabled  = true
      sku_name = "Standard_AzureFrontDoor"
    }
  }

  # Wildcard hosts can only be issued by Let's Encrypt via DNS-01.
  letsencrypt_dns01_azure_dns = {
    zone_name                = "customer.example.com"
    zone_resource_group_name = "dns-rg"
  }
}
```

## Argument reference

### Top-level arguments

* `location`: Required. Azure region for deployment.
* `customer`: Required. Customer identifier used in naming and `API_URL` suffixing.
* `resource_group_config`: Optional. Set `create = false` and supply `name` to deploy into an existing resource group. Default: creates `espresso-ai-proxy-rg`.
* `create_dedicated_vnet`: Optional. Creates a dedicated VNet (`true`) or uses an existing VNet (`false`). Default: `true`.
* `vnet_config`: Optional/conditional. Used when `create_dedicated_vnet = true`.
* `existing_vnet_config`: Optional/conditional. Required when `create_dedicated_vnet = false`.
* `aks_config`: Optional. AKS cluster and node pool settings.
* `proxy_config`: Required. Proxy runtime configuration.
* `proxy_api_key_value`: Optional/conditional, sensitive. Required when `proxy_config.api_key_secret_mode = MANAGED_AZURE_KEY_VAULT`.
* `ingress_config`: Optional. NGINX ingress configuration, TLS provisioning, and optional Azure Front Door fronting.
* `dns_config`: Optional. Azure DNS A-record configuration.
* `letsencrypt_dns01_azure_dns`: Optional. Switches cert-manager to the DNS-01 solver via Azure DNS. Required when `ingress_config.ingress_host` is a wildcard.
* `autoscaling_config`: Optional. Proxy HPA configuration.
* `tags`: Optional. Additional Azure tags. Default: `{}`.

### `resource_group_config`

* `create`: Optional. Default: `true`.
* `name`: Optional. Default: `espresso-ai-proxy-rg`. Must be non-empty. When `create = false`, an existing resource group with this name must exist.

### `vnet_config`

* `vnet_name`: Optional. Default: `espresso-ai-proxy-vnet`.
* `address_space`: Optional. Default: `["10.240.0.0/16"]`. Must contain at least one valid CIDR when `create_dedicated_vnet = true`.
* `node_subnet_cidr`: Optional. Default: `10.240.0.0/22`. Must be a valid CIDR within `address_space`.

### `existing_vnet_config`

* `vnet_id`: Required in existing-VNet mode.
* `node_subnet_id`: Required in existing-VNet mode.

### `aks_config`

* `cluster_name`: Optional. Default: `espresso-ai-proxy`. Used as the resource name prefix throughout the module.
* `kubernetes_version`: Optional. Default: `1.35`.
* `api_server_authorized_ranges`: Optional. Required when `enable_private_cluster = false`. CIDRs allowed to reach the AKS API server.
* `enable_private_cluster`: Optional. Default: `false`. When `true`, the API server has only a private endpoint.
* `pod_cidr`: Optional. Default: `10.244.0.0/16`. CNI Overlay pod range; not part of the VNet.
* `service_cidr`: Optional. Default: `10.245.0.0/16`. ClusterIP service range.
* `dns_service_ip`: Optional. Default: `10.245.0.10`. Must lie within `service_cidr`.
* `vm_size`: Optional. Default: `Standard_D8s_v5`.
* `node_pool_min_count`: Optional. Default: `2`.
* `node_pool_max_count`: Optional. Default: `10`. Must be ≥ `node_pool_min_count`.
* `enable_log_analytics`: Optional. Default: `false`. When `true`, attaches a Log Analytics workspace and enables Container Insights.
* `log_analytics_retention_days`: Optional. Default: `90`.

### `proxy_config`

* `image`: Required. Proxy container image URI in Espresso AI's ACR.
* `replicas`: Optional. Default: `2`.
* `proxy_host`: Required. Non-empty value injected as `PROXY_HOST`.
* `otel_collector`: Optional. OTEL Collector sidecar configuration. See [`otel_collector`](#otel_collector) below.
* `api_key_secret_name`: Optional. Kubernetes secret name in the proxy namespace from which `ESPRESSO_AI_API_KEY` is mounted. Default: `espresso-ai`.
* API key secret key name is fixed to `ESPRESSO_AI_API_KEY` and is not configurable.
* `api_key_secret_mode`: Optional. `BYO_K8S_SECRET` or `MANAGED_AZURE_KEY_VAULT`. Default: `BYO_K8S_SECRET`.
* `api_key_azure_key_vault_secret`: Optional. Key Vault secret name used in managed mode. Default: `espresso-ai-proxy-api-key`. Required (non-empty) when `api_key_secret_mode = MANAGED_AZURE_KEY_VAULT`.
* `api_url`: Optional. Base URL. Default: `https://api.espressocomputing.com:25831`.
* `env_vars`: Optional. Map of environment variable key/value pairs. Currently supported keys:

  | key                  | type   | definition                                                                                                                                  |
  | -------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------- |
  | `EXCLUDE_QUERY_TEXT` | `bool` | Default: `false`. Whether to exclude query text on requests to Espresso AI's API. *Note: Enabling this will limit supported functionality.* |

### `otel_collector`

Nested object under `proxy_config`. When `enabled = true` (the default, starting in `v0.4.0`), the module deploys an OpenTelemetry Collector sidecar in the proxy pod and renders a ConfigMap with its pipeline. The proxy is automatically pointed at `http://localhost:4318`; the sidecar forwards to your-owned OTLP backend.

* `enabled`: Optional. Default: `true`. Set to `false` to disable the sidecar; the proxy will then emit OTLP directly to `otel_exporter_otlp_endpoint`.
* `image`: Optional. Full image reference for the collector container. Default: `otel/opentelemetry-collector-contrib:0.152.0`.
* `customer_endpoint`: Optional. OTLP endpoint for the customer's own observability backend. Leave empty (default) to disable the customer exporter; the Espresso pipeline still runs.
* `customer_protocol`: Optional. `grpc` (renders the `otlp/customer` exporter) or `http` (renders `otlphttp/customer`). Default: `grpc`.
* `customer_signals`: Optional. Signals to mirror to the customer exporter. Any subset of `traces`, `metrics`, `logs`. Default: all three.
* `customer_auth_secret_name`: Optional. Existing Kubernetes Secret in the proxy namespace whose value is mounted as `CUSTOMER_OTLP_AUTH` and sent as the customer exporter's `Authorization` header. Leave empty for unauthenticated endpoints.
* `customer_auth_secret_key`: Optional. Key within `customer_auth_secret_name`. Default: `authorization`.
* `customer_tls_insecure`: Optional. Disable TLS verification on the customer exporter. Default: `false`.

Example — also mirror traces and metrics (not logs) to the customer's own OTLP backend with bearer-token auth:

```hcl
proxy_config = {
  repository = "<Espresso AI's Azure Account ID>.azurecr.io/proxy"
  image      = "0.1-dev-..."
  proxy_host = "proxy.customer.example.com"

  otel_collector = {
    customer_endpoint         = "https://otlp.observability.customer.example.com:4317"
    customer_protocol         = "grpc"
    customer_signals          = ["traces", "metrics"]
    customer_auth_secret_name = "customer-otlp-auth"
  }
}
```

The `customer-otlp-auth` Secret must exist in the `proxy` namespace and contain an `authorization` key whose value is the full header (e.g. `Bearer eyJ...`).

For the full list of metrics, spans, and resource attributes the proxy emits — useful for building dashboards and alerts against the customer exporter — see [Proxy telemetry reference](/snowflake-optimizer/proxy-onboarding/proxy-telemetry-reference.md).

### `ingress_config`

* `enable_ingress`: Optional. Enables the nginx Ingress. Default: `true`.
* `ingress_host`: Required when `enable_ingress = true`. Hostname (or wildcard, e.g. `*.example.com`) the Ingress serves.
* `letsencrypt_email`: Optional. When set, installs cert-manager and a Let's Encrypt `ClusterIssuer`, and cert-manager auto-issues/renews a `kubernetes.io/tls` secret for the Ingress. Mutually exclusive with `tls_secret_name`. Exactly one must be provided when `enable_ingress = true`.
* `use_letsencrypt_staging`: Optional. Default: `false`. Switches the issuer to Let's Encrypt staging (useful while iterating to avoid hitting prod rate limits).
* `tls_secret_name`: Optional. Bring-your-own `kubernetes.io/tls` secret name in the proxy namespace (e.g. synced from a Key Vault cert via the Secrets Store CSI driver). Mutually exclusive with `letsencrypt_email`.
* `front_door`: Optional. Azure Front Door fronting configuration:
  * `enabled`: Optional. Default: `false`.
  * `sku_name`: Optional. `Standard_AzureFrontDoor` or `Premium_AzureFrontDoor`. Default: `Standard_AzureFrontDoor`. Premium adds WAF and Private Link to origin.

### `dns_config`

* `create_record`: Optional. Creates an Azure DNS A record pointing at the ingress public IP. Default: `false`.
* `zone_name`: Required when `create_record = true`. Apex of the existing Azure DNS zone (e.g. `customer.example.com`).
* `zone_resource_group_name`: Required when `create_record = true`. Resource group of the DNS zone.
* `record_name`: Optional. Falls back to `ingress_config.ingress_host` when omitted.
* `ttl`: Optional. Default: `300`.

### `letsencrypt_dns01_azure_dns`

When set, cert-manager uses DNS-01 (which supports wildcard certs) instead of HTTP-01. Required when `ingress_config.ingress_host` is a wildcard, since Let's Encrypt only issues wildcards via DNS-01. The module provisions a user-assigned managed identity, federates it to cert-manager's controller service account, and grants it `DNS Zone Contributor` on the named zone — no static credentials are needed.

* `zone_name`: Required.
* `zone_resource_group_name`: Required.

### `autoscaling_config`

* `min_replicas`: Optional. Default: `2`.
* `max_replicas`: Optional. Default: `10`. Must be ≥ `min_replicas`.
* `target_cpu_utilization`: Optional. Default: `70`. Must be between 1 and 100.

## Secret modes

* `BYO_K8S_SECRET` (default): Proxy reads from an existing Kubernetes secret (`api_key_secret_name`) in the proxy namespace using fixed key `ESPRESSO_AI_API_KEY`.
* `MANAGED_AZURE_KEY_VAULT`: Module provisions an Azure Key Vault, writes the API key as a secret, federates a user-assigned managed identity to the External Secrets Operator service account, installs ESO, and creates an `ExternalSecret` that syncs the Key Vault secret into a `kubernetes.io/tls`-style Kubernetes secret in the proxy namespace.

## TLS modes

* **Let's Encrypt (default path).** Set `ingress_config.letsencrypt_email`. cert-manager runs an HTTP-01 challenge through the nginx Ingress and writes a `proxy-tls` secret in the proxy namespace. Renewals are automatic.
* **Let's Encrypt with DNS-01.** Add `letsencrypt_dns01_azure_dns` to switch the solver to Azure DNS. Required for wildcard hosts. The module wires up a federated managed identity scoped to `DNS Zone Contributor` on the target zone — no service-principal credentials needed.
* **Bring-your-own TLS secret.** Set `ingress_config.tls_secret_name` (and pre-create that secret in the proxy namespace) — useful when an existing process syncs a Key Vault cert via the Secrets Store CSI driver, or when cert-manager is managed outside this module.
* **Front Door fronting.** When `ingress_config.front_door.enabled = true`, AFD terminates TLS for end clients with a DigiCert-issued managed cert (which has working OCSP). The LE/BYO cert on the AKS LB then only secures the AFD-to-origin hop. Use this when the client enforces strict OCSP (e.g. Snowflake's connector).

## Outputs

The module exports:

* `resource_group_name`
* `vnet_id`
* `node_subnet_id`
* `aks_cluster_name`
* `aks_node_resource_group`
* `aks_oidc_issuer_url`
* `aks_kubelet_identity_object_id` — exposed for advanced cases (e.g. granting `AcrPull` on a private registry of your own). Not needed for the standard flow, which uses Espresso AI's ACR.
* `proxy_namespace`
* `proxy_service_name`
* `proxy_ingress_public_ip`
* `proxy_hpa_name`
* `proxy_dns_fqdn`
* `proxy_api_key_key_vault_name` — only set when `MANAGED_AZURE_KEY_VAULT` is enabled.
* `front_door_endpoint_hostname` — point a CNAME from your custom domain at this. Only set when AFD fronting is enabled.
* `front_door_custom_domain_validation_token` — publish at `_dnsauth.<ingress_host>` so AFD will issue the managed cert. Only set when AFD fronting is enabled.
* `front_door_route_id`, `front_door_custom_domain_id`, `front_door_custom_domain_association_id` — AFD resource IDs, only set when AFD fronting is enabled.

### Customer-side DNS work after AFD provisioning

When `ingress_config.front_door.enabled = true`, after `terraform apply` finishes, publish:

1. `CNAME <ingress_host> → <front_door_endpoint_hostname>`
2. `TXT _dnsauth.<ingress_host> → <front_door_custom_domain_validation_token>`

Both values are exposed as outputs above.

## How to deploy

Deployment typically takes around 20-30 minutes (longer when Azure Front Door is enabled and waiting on managed-cert validation).

```bash
terraform init
terraform plan
terraform apply
```

ACR pull is handled on the Espresso AI side — once your tenant ID is registered in the dashboard, our onboarding workflow grants your AKS cluster pull access on the `proxy` repository in Espresso AI's ACR. No `az role assignment` is needed in your subscription.

## Best practices

* Manage sensitive variables (e.g. `proxy_api_key_value`) via environment variables or `.tfvars` files excluded from source control.
* Keep `aks_config.api_server_authorized_ranges` tight when running a public API server, or set `enable_private_cluster = true` and reach the cluster via a peered network/jumpbox.
* For wildcard ingress hosts, always pair `letsencrypt_email` with `letsencrypt_dns01_azure_dns` — HTTP-01 cannot validate a wildcard.
* When fronting clients with strict OCSP behavior (Snowflake's connector, in particular), enable `ingress_config.front_door` so end clients see AFD's DigiCert cert instead of the LE cert on the AKS LB.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.espresso.ai/snowflake-optimizer/proxy-onboarding/proxy-onboarding-terraform-deployment-azure.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.