> For the complete documentation index, see [llms.txt](https://docs.espresso.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.espresso.ai/databricks-optimizer/proxy-onboarding/proxy-onboarding-airflow.md).

# Airflow

### Overview

This guide walks you through updating your Apache Airflow Databricks connection to point to Espresso's infrastructure. The key change is replacing your Databricks host (e.g. `<workspace>.cloud.databricks.com`) with the corresponding Espresso host (`<workspace>.cloud.databricks.espressocomputing.com`). The `http_path` stays the same.

### Prerequisites

* Access to your Airflow environment (UI or config files)
* The package `apache-airflow-providers-databricks` installed
* Your Databricks connection details (Server Hostname, HTTP Path, credentials)

### Steps

#### Option A: Update via the Airflow UI

1. Navigate to **Admin → Connections** in the Airflow web UI.
2. Find and click on your Databricks connection (default ID: `databricks_default`).
3. Set **Host** to your Espresso endpoint, e.g. `<workspace>.cloud.databricks.espressocomputing.com`.
4. Leave the **Extra** field as-is. The `http_path` does not change:

```json
{
    "http_path": "/sql/1.0/warehouses/<warehouse_id>"
}
```

5. Leave **Login** / **Password** (or the token in **Extra**) unchanged. Your existing Databricks credentials continue to work through the proxy.
6. Click **Save**.

#### Option B: Update via Environment Variable (JSON format)

If you manage connections through environment variables, update the variable as follows:

```json
export AIRFLOW_CONN_DATABRICKS_DEFAULT='{
    "conn_type": "databricks",
    "host": "<workspace>.cloud.databricks.espressocomputing.com",
    "password": "<personal_access_token>",
    "extra": {
        "http_path": "/sql/1.0/warehouses/<warehouse_id>"
    }
}'
```

#### Option C: Update via Environment Variable (URI format)

```
export AIRFLOW_CONN_DATABRICKS_DEFAULT='databricks://:<personal_access_token>@<workspace>.cloud.databricks.espressocomputing.com?http_path=%2Fsql%2F1.0%2Fwarehouses%2F<warehouse_id>'
```

**Note:** All URI components should be URL-encoded.

### Verifying the Connection

After updating, test the connection by:

1. Clicking **Test** on the connection page in the Airflow UI (Airflow 2.5+), or
2. Running a simple DAG task that queries through the proxy (e.g. a `DatabricksSqlOperator` with `SELECT 1`).

### Troubleshooting

* **Authentication errors** — Your existing Databricks credentials should work unchanged. If you use service principal OAuth, confirm the relevant Extra fields (`service_principal_oauth`, client ID/secret) are still present.
* **Connection timeouts** — Verify that your Airflow environment has network access to `<workspace>.cloud.databricks.espressocomputing.com`.
* **Warehouse not found** — Confirm the `http_path` in **Extra** is unchanged and still points at an existing SQL warehouse.