Here is a step-by-step procedure for connecting dbt Cloud or dbt Core to Databricks:
- Create a new Databricks workspace, or use an existing one.
- In the Databricks workspace, navigate to the “Workspace” menu and select “API Tokens.”
- Create a new API token and copy it to your clipboard.
- In dbt Cloud, navigate to the “Settings” menu and select “Connections.”
- Select “Databricks” as the connection type.
- Enter the Databricks URL, API token, and your Databricks username.
- Test the connection to ensure it is successful.
- Once the connection is established, you can now use dbt to run data transformation on your Databricks cluster.
Note: If you are using dbt Core, you will also need to install the Databricks package.
You can install by running pip install databricks-cli
You need to configure the Databricks CLI by running databricks configure --token and enter the URL, token and username.
Get more useful articles on dbt
