> For the complete documentation index, see [llms.txt](https://docs.espresso.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.espresso.ai/databricks-optimizer/databricks-terraform-onboarding.md).

# Databricks Terraform Onboarding

The Espresso Databricks module manages the Databricks service principal, workspace assignments, system-table grants, bootstrap SQL warehouse, Espresso credentials, and Warehouse Agent settings.

## Prerequisites

* Terraform 1.11 or newer
* Databricks account-admin access
* Databricks metastore-admin access
* Identity-federated workspaces attached to Unity Catalog
* Serverless SQL support in each workspace
* An Espresso organization API key

In Espresso, select an existing account, open **Tools → API Keys**, choose **Generate API key → Organization key**, and copy the complete `ok_` secret. It cannot be displayed again. Store it in your secret manager and expose it to Terraform as `ESPRESSO_API_KEY`.

Configure an account-level Databricks provider using a bootstrap identity that can manage every workspace passed to the module:

```hcl
terraform {
  required_version = ">= 1.11.0"

  required_providers {
    databricks = {
      source  = "databricks/databricks"
      version = ">= 1.122.0"
    }
    espresso = {
      source  = "espressocomputing/espresso"
      version = ">= 0.1.2"
    }
  }
}

provider "databricks" {
  host          = "https://accounts.cloud.databricks.com"
  account_id    = var.databricks_account_id
  client_id     = var.databricks_client_id
  client_secret = var.databricks_client_secret
}

provider "espresso" {}
```

## Configure the module

Create one Espresso account per Databricks workspace. Warehouse Agent settings are disabled unless explicitly enabled.

```hcl
module "espresso_databricks" {
  source  = "espressocomputing/databricks/espresso"
  version = "~> 0.1"

  providers = {
    databricks = databricks
    espresso   = espresso
  }

  workspaces = {
    "1234567890123456" = {
      espresso_slug  = "databricks_acme_production"
      display_name   = "ACME Production Databricks"
      workspace_url  = "https://dbc-example.cloud.databricks.com"
      workspace_name = "production"

      warehouse_agent = {
        enabled     = true
        auto_opt_in = true
      }

      managed_warehouses = {
        "ANALYTICS_WH" = {
          enabled = true
        }
      }
    }
  }
}
```

The module grants `USE_CATALOG` on the `system` catalog and grants `USE_SCHEMA` and `SELECT` on every system schema discovered during planning. Run Terraform again after Databricks adds a system schema so the new grant is created.

When several workspaces share a Unity Catalog metastore, set `system_table_access_workspace_ids` to one representative workspace ID per metastore. This prevents Terraform from managing the same grants through multiple workspace APIs.

## Secret storage

The module creates a Databricks OAuth secret for the Espresso service principal. The Databricks provider returns that generated secret to Terraform, so it is stored in Terraform state. Use an encrypted remote backend and restrict state access.

The Espresso provider treats its `client_secret` argument as write-only and does not add another copy to state. When importing an existing Databricks secret, the module also accepts its plaintext through the sensitive, ephemeral `existing_service_principal_client_secret` input. The Databricks resource itself remains represented in state, and any secret Terraform generates during later rotation is stored there.

Use a separate bootstrap identity for the Databricks provider. Terraform cannot safely rotate the same OAuth secret it needs to authenticate the rotation operation.

## Existing onboarding resources

If the service principal, OAuth secret, workspace assignment, grants, or bootstrap warehouse already exist, import them into the module addresses before applying. Review the plan until it contains imports and expected in-place changes only. Do not allow Terraform to replace an active service principal or bootstrap warehouse during adoption.

After Terraform finishes, verify the Databricks connection and enabled warehouses in Espresso.