Skip to content

feat: add Azure ADLS Gen2 staging support to Databricks warehouse skill - #76

Merged
nicklasl merged 3 commits into
spotify:mainfrom
jondatan:feat/databricks-azure-adls-staging
Sep 29, 2026
Merged

nicklasl merged 3 commits into
spotify:mainfrom
jondatan:feat/databricks-azure-adls-staging

Conversation

@jondatan

Copy link
Copy Markdown
Contributor

Summary

  • Add Azure ADLS Gen2 as a second staging provider option alongside AWS S3 in the setup-warehouse-databricks skill
  • Insert a staging provider choice step and full Azure ADLS walkthrough (managed identity, federated credential, storage permissions)
  • Update connector configJson to use the connectionConfig wrapper and support azureAdlsStorageConfig
  • Remove s3BucketConfig from the Metrics Data Warehouse config (it doesn't use staging)
  • Update schema step to Unity Catalog only (fully qualified catalog.schema)
  • Update parent setup-warehouse skill description to mention both staging options

Context

The Confidence webapp and docs now support Azure ADLS Gen2 as a staging provider for Databricks connectors (alongside the existing S3 path). This PR brings the onboarding skill in sync with those changes.

Test plan

  • Run the skill with S3 staging and verify the flow is unchanged
  • Run the skill with Azure ADLS staging and verify managed identity + federated credential instructions are correct
  • Verify the configJson shapes match what the Confidence API expects for both staging providers

🤖 Generated with Claude Code

Jonatan Nyström and others added 3 commits September 24, 2026 10:05
The Databricks connector now supports Azure ADLS Gen2 as a staging
provider alongside AWS S3. Update the setup-warehouse-databricks skill
to let users choose between the two, walk through Azure managed identity
and federated credential setup, and pass the correct configJson shape
(with connectionConfig wrapper and azureAdlsStorageConfig) for
connectors. Also update the schema step to Unity Catalog only and remove
s3BucketConfig from the Metrics Data Warehouse config since it does not
use staging storage.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After the user picks S3 or ADLS in Step 5, the Step 6 label now
reflects their choice for better readability.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…icks

Align the dry-run Databricks warehouse section with the real skill's
new staging provider choice (AWS S3 or Azure ADLS Gen2).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@nicklasl
nicklasl merged commit 99c4913 into spotify:main Sep 29, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants