Terraform provider

Authentication

In the Espresso dashboard, select an existing account, open Tools → API Keys, choose Generate API key → Organization key, and copy the complete ok_ secret. It cannot be displayed again. Set it as ESPRESSO_API_KEY.

One account per Databricks workspace

An espresso_account is the Espresso account boundary. Key the Terraform resources by Databricks workspace ID and give each workspace a permanent Espresso slug:

variable "databricks_workspaces" {
  type = map(object({
    espresso_slug = string
    display_name  = string
    workspace_url = string
  }))
}

resource "espresso_account" "workspace" {
  for_each = var.databricks_workspaces

  slug         = each.value.espresso_slug
  display_name = each.value.display_name
  product      = "databricks"
}

resource "espresso_databricks_warehouse_agent" "workspace" {
  for_each = var.databricks_workspaces

  account     = espresso_account.workspace[each.key].slug
  enabled     = false
  auto_opt_in = false
}

output "espresso_account_by_workspace_id" {
  value = {
    for workspace_id, account in espresso_account.workspace :
    workspace_id => account.slug
  }
}

For example:

databricks_workspaces = {
  "1234567890123456" = {
    espresso_slug = "acme_production"
    display_name  = "Acme Production"
    workspace_url = "https://1234567890123456.cloud.databricks.com"
  }
  "9876543210987654" = {
    espresso_slug = "acme_staging"
    display_name  = "Acme Staging"
    workspace_url = "https://9876543210987654.cloud.databricks.com"
  }
}

Espresso prepends databricks_ when a Databricks slug omits it, so these accounts are stored as databricks_acme_production and databricks_acme_staging. Every global and warehouse setting for workspace 1234567890123456 must use espresso_account.workspace["1234567890123456"].slug as its account.

An account's display_name can be updated in place. Its slug and product are immutable. Removing an account resource from Terraform stops managing it but leaves the account in Espresso.

Databricks onboarding must still be run for each account.

Databricks credentials

The credentials resource authenticates with Databricks, verifies access to the configured SQL warehouse, and saves the connection in Espresso:

resource "espresso_databricks_credentials" "workspace" {
  for_each = var.databricks_workspaces

  account                = espresso_account.workspace[each.key].slug
  workspace_url          = each.value.workspace_url
  workspace_id           = each.key
  workspace_name         = each.value.display_name
  client_id              = databricks_service_principal.espresso.application_id
  client_secret          = databricks_service_principal_secret.espresso.secret
  service_principal_id   = databricks_service_principal.espresso.id
  service_principal_name = databricks_service_principal.espresso.display_name
  warehouse_id           = databricks_sql_endpoint.espresso[each.key].id
  warehouse_name         = databricks_sql_endpoint.espresso[each.key].name
}

client_secret is write-only in the Espresso provider and is not retained in that resource's state. The Databricks provider retains the generated service-principal secret in Terraform state, so use encrypted remote state with tightly restricted access.

See Databricks Terraform Onboarding for a complete configuration that creates the Databricks identity, permissions, SQL warehouse, and Espresso credentials.

Warehouse Agent settings

locals {
  shared_workspace_id = "1234567890123456"
  shared = {
    min_clusters = 1
    max_clusters = 8
  }
}

resource "databricks_sql_endpoint" "shared" {
  name             = "Shared SQL"
  min_num_clusters = local.shared.min_clusters
  max_num_clusters = local.shared.max_clusters
  cluster_size     = "Large"
  warehouse_type   = "PRO"

  lifecycle {
    ignore_changes = [min_num_clusters, max_num_clusters]
  }
}

resource "espresso_databricks_warehouse_agent_warehouse" "shared" {
  account        = espresso_account.workspace[local.shared_workspace_id].slug
  name           = databricks_sql_endpoint.shared.name
  enabled        = true
  min_clusters   = local.shared.min_clusters
  max_clusters   = local.shared.max_clusters
}

The lifecycle list prevents the Databricks and Espresso providers from fighting over Warehouse Agent settings. Terraform lifecycle values cannot be conditional. To return control safely, first set the Espresso warehouse's enabled to false and apply, then remove its ignore_changes entries and apply again. The Databricks provider then reconciles the warehouse to the configured values.

Each Warehouse Agent warehouse configuration is managed as a discrete resource. Its settings fields are optional, so an account and name can adopt the current values without changing them. Removing a Warehouse Agent resource stops Terraform management without changing the current Espresso settings or the underlying warehouse.