diff --git a/.changeset/hashicorp-vault-plugin.md b/.changeset/hashicorp-vault-plugin.md new file mode 100644 index 000000000..4f47d64d8 --- /dev/null +++ b/.changeset/hashicorp-vault-plugin.md @@ -0,0 +1,5 @@ +--- +"@varlock/hashicorp-vault-plugin": patch +--- + +New plugin for loading secrets from HashiCorp Vault (KV v2 secrets engine) and OpenBao. diff --git a/bun.lock b/bun.lock index 7c14286d1..8c2c71d5c 100644 --- a/bun.lock +++ b/bun.lock @@ -182,6 +182,21 @@ "varlock": "workspace:^", }, }, + "packages/plugins/hashicorp-vault": { + "name": "@varlock/hashicorp-vault-plugin", + "version": "0.0.1", + "devDependencies": { + "@env-spec/utils": "workspace:^", + "@types/node": "catalog:", + "ky": "catalog:", + "tsup": "catalog:", + "varlock": "workspace:^", + "vitest": "catalog:", + }, + "peerDependencies": { + "varlock": "workspace:^", + }, + }, "packages/plugins/infisical": { "name": "@varlock/infisical-plugin", "version": "0.0.5", @@ -1147,6 +1162,8 @@ "@varlock/google-secret-manager-plugin": ["@varlock/google-secret-manager-plugin@workspace:packages/plugins/google-secret-manager"], + "@varlock/hashicorp-vault-plugin": ["@varlock/hashicorp-vault-plugin@workspace:packages/plugins/hashicorp-vault"], + "@varlock/infisical-plugin": ["@varlock/infisical-plugin@workspace:packages/plugins/infisical"], "@varlock/nextjs-integration": ["@varlock/nextjs-integration@workspace:packages/integrations/nextjs"], diff --git a/packages/plugins/hashicorp-vault/README.md b/packages/plugins/hashicorp-vault/README.md new file mode 100644 index 000000000..920e10f9d --- /dev/null +++ b/packages/plugins/hashicorp-vault/README.md @@ -0,0 +1,341 @@ +# @varlock/hashicorp-vault-plugin + +[![npm version](https://img.shields.io/npm/v/@varlock/hashicorp-vault-plugin.svg)](https://www.npmjs.com/package/@varlock/hashicorp-vault-plugin) [![GitHub stars](https://img.shields.io/github/stars/dmno-dev/varlock.svg?style=social&label=Star)](https://github.com/dmno-dev/varlock) [![license](https://img.shields.io/npm/l/@varlock/hashicorp-vault-plugin.svg)](https://github.com/dmno-dev/varlock/blob/main/LICENSE) + +This package is a [Varlock](https://varlock.dev) [plugin](https://varlock.dev/guides/plugins/) that enables loading data from [HashiCorp Vault](https://www.vaultproject.io/) (KV v2 secrets engine) / [OpenBao](https://openbao.org/) into your configuration. + +## Features + +- **Zero-config authentication** - Automatically uses Vault token from environment or CLI +- **AppRole authentication** - For automated and CI/CD workflows +- **Vault CLI integration** - Works seamlessly with `vault login` for local development +- **Auto-infer secret keys** from environment variable names +- **JSON key extraction** from secrets using `#` syntax or named `key` parameter +- **Path prefixing** with `pathPrefix` option for organized secret management +- **Default path** support for sharing a common secret path across items +- Support for Vault Enterprise namespaces +- Support for multiple Vault instances +- Automatic AppRole token caching and renewal +- Lightweight implementation using REST API (no heavy Vault SDK dependencies) + +## Installation + +If you are in a JavaScript based project and have a package.json file, you can either install the plugin explicitly +```bash +npm install @varlock/hashicorp-vault-plugin +``` +And then register the plugin without any version number +```env-spec title=".env.schema" +# @plugin(@varlock/hashicorp-vault-plugin) +``` + +Otherwise just set the explicit version number when you register it +```env-spec title=".env.schema" +# @plugin(@varlock/hashicorp-vault-plugin@1.2.3) +``` + +See our [Plugin Guide](https://varlock.dev/guides/plugins/#installation) for more details. + +## Setup + Auth + +After registering the plugin, you must initialize it with the `@initHcpVault` root decorator. + +### Automatic auth + +For most use cases, you only need to provide the Vault URL: + +```env-spec +# @plugin(@varlock/hashicorp-vault-plugin) +# @initHcpVault(url="https://vault.example.com:8200") +``` + +**How this works:** + +- **Local development:** Run `vault login` → automatically uses the token from `~/.vault-token` +- **CI/CD pipelines:** Wire up a token explicitly via `token=$VAULT_TOKEN` +- **Works everywhere** with zero configuration beyond the URL! + +### AppRole auth (For automated workflows) + +For CI/CD or server environments, use AppRole authentication: + +```env-spec +# @plugin(@varlock/hashicorp-vault-plugin) +# @initHcpVault( +# url="https://vault.example.com:8200", +# roleId=$VAULT_ROLE_ID, +# secretId=$VAULT_SECRET_ID +# ) +# --- + +VAULT_ROLE_ID= +# @sensitive +VAULT_SECRET_ID= +``` + +You would then need to inject these env vars using your CI/CD system. + +### Explicit token + +You can also provide a token directly: + +```env-spec +# @initHcpVault( +# url="https://vault.example.com:8200", +# token=$VAULT_TOKEN +# ) +# --- + +# @type=vaultToken @sensitive +VAULT_TOKEN= +``` + +### Authentication Priority + +The plugin tries authentication methods in this order: +1. **Explicit token** - If `token` is provided in `@initHcpVault()` +2. **AppRole** - If both `roleId` and `secretId` are provided +3. **CLI token file** - From `~/.vault-token` (created by `vault login`) or `~/.bao-token` (created by `bao login` for OpenBao) + +### Vault Enterprise namespaces + +For Vault Enterprise, specify the namespace: + +```env-spec +# @initHcpVault(url="https://vault.example.com:8200", namespace="admin/team-a") +``` + +### Multiple instances + +If you need to connect to multiple Vault instances, register named instances: + +```env-spec +# @initHcpVault(id=prod, url="https://vault-prod.example.com:8200") +# @initHcpVault(id=dev, url="https://vault-dev.example.com:8200") +``` + +## Reading secrets + +This plugin introduces the `vaultSecret()` function to fetch secret values from Vault's KV v2 secrets engine. + +Since Vault KV v2 always stores key/value pairs, the item key (variable name) is automatically used as the JSON key to extract from the secret. You can override this with `#KEY` syntax or the `key` parameter. + +```env-spec title=".env.schema" +# @plugin(@varlock/hashicorp-vault-plugin) +# @initHcpVault(url="https://vault.example.com:8200") +# --- + +# Fetches "secret/db/config" and extracts "DB_HOST" key +DB_HOST=vaultSecret("secret/db/config") + +# Override the extracted key with # syntax +DB_PASSWORD=vaultSecret("secret/db/config#password") + +# Or use named "key" parameter +DB_PORT=vaultSecret("secret/db/config", key="PORT") + +# Fetch entire secret as JSON blob +DB_CONFIG=vaultSecret("secret/db/config", raw=true) + +# If using multiple instances +PROD_KEY=vaultSecret(prod, "secret/api/keys") +DEV_KEY=vaultSecret(dev, "secret/api/keys") +``` + +### Default path + +Use `defaultPath` to set a common path for secrets when no path argument is provided: + +```env-spec +# @initHcpVault(url="https://vault.example.com:8200", defaultPath=secret/myapp/config) +# --- + +# Both fetch from "secret/myapp/config" extracting item key +DB_PASSWORD=vaultSecret() +API_KEY=vaultSecret() + +# Override the inferred key using # syntax +STRIPE_KEY=vaultSecret("#stripe_api_key") + +# Explicit path still extracts item key by default +OTHER_SECRET=vaultSecret("secret/other/path") + +# Or override key on explicit path +OTHER_KEY=vaultSecret("secret/other/path#SPECIFIC_KEY") +``` + +### Path prefixing + +Use `pathPrefix` to automatically prefix all secret paths: + +```env-spec +# @initHcpVault(url="https://vault.example.com:8200", pathPrefix="secret/myapp") +# --- + +# Fetches from "secret/myapp/db/config" +DB_HOST=vaultSecret("db/config#HOST") +``` + +You can even use dynamic prefixes: + +```env-spec +# @initHcpVault(url="https://vault.example.com:8200", pathPrefix="secret/${ENV}") +# In prod: fetches from "secret/prod/..." +# In dev: fetches from "secret/dev/..." +DB_HOST=vaultSecret("db/config#HOST") +``` + +### Bulk loading secrets + +Use `raw=true` with `@setValuesBulk` to load all key/value pairs from a Vault path at once, instead of wiring up each secret individually: + +```env-spec +# @initHcpVault(url="https://vault.example.com:8200") +# @setValuesBulk(vaultSecret("secret/myapp/config", raw=true)) +# --- + +DB_HOST= +DB_PASSWORD= +API_KEY= +``` + +This fetches all keys from `secret/myapp/config` and maps them to matching item keys. + +--- + +## Reference + +### Root decorators + +#### `@initHcpVault()` + +Initialize a HashiCorp Vault plugin instance. + +**Parameters:** + +- `url: string` (required) - Vault server URL (e.g., `https://vault.example.com:8200`) +- `token?: string` - Explicit Vault authentication token +- `roleId?: string` - AppRole role ID for automated authentication +- `secretId?: string` - AppRole secret ID for automated authentication +- `namespace?: string` - Vault Enterprise namespace +- `defaultPath?: string` - Default secret path when no path argument is given to `vaultSecret()` +- `pathPrefix?: string` - Prefix automatically prepended to all secret paths +- `id?: string` - Instance identifier for multiple instances (defaults to `_default`) + +### Functions + +#### `vaultSecret()` + +Fetch a secret from HashiCorp Vault's KV v2 secrets engine. + +**Signatures:** + +- `vaultSecret()` - Uses `defaultPath`, extracts item key +- `vaultSecret(secretRef)` - Fetch by explicit path, extracts item key +- `vaultSecret(secretRef, key="jsonKey")` - Fetch and extract a specific key +- `vaultSecret(secretRef, raw=true)` - Fetch all key/value pairs as JSON blob (useful with `@setValuesBulk`) +- `vaultSecret(instanceId, secretRef)` - Fetch from a specific Vault instance + +**Key extraction:** + +By default, the item key (variable name) is used as the JSON key to extract from the secret. You can override this with `#KEY` syntax in the path or the named `key` parameter, or use `raw=true` to get the full key/value blob. + +**Secret Ref Formats:** + +- Path only: `"secret/myapp/config"` (extracts item key from the secret) +- Path with key override: `"secret/myapp/config#DB_PASSWORD"` (extracts specific key) + +**How paths work:** + +Vault KV v2 stores key/value pairs at a path. Given a path like `secret/myapp/config`, the plugin calls `GET /v1/secret/data/myapp/config` (the first path segment is the mount point, and `/data/` is inserted for the KV v2 API). + +### Data Types + +- `vaultToken` - HashiCorp Vault authentication token (sensitive) + +--- + +## Vault Setup + +### Enable KV v2 Secrets Engine + +```bash +# KV v2 is enabled by default at "secret/" in dev mode +# For production, enable it explicitly: +vault secrets enable -version=2 -path=secret kv +``` + +### Create a Policy + +```hcl +# policy.hcl - Allow reading secrets +path "secret/data/*" { + capabilities = ["read"] +} +``` + +```bash +vault policy write varlock-reader policy.hcl +``` + +### Set Up AppRole Auth (Recommended for CI/CD) + +AppRole is the recommended auth method for automated workflows: + +```bash +# Enable AppRole auth method +vault auth enable approle + +# Create a role +vault write auth/approle/role/varlock-role \ + secret_id_ttl=24h \ + token_ttl=1h \ + token_max_ttl=4h \ + token_policies=varlock-reader + +# Get the role ID +vault read auth/approle/role/varlock-role/role-id + +# Generate a secret ID +vault write -f auth/approle/role/varlock-role/secret-id +``` + +Save the `role_id` and `secret_id` from the output for your CI/CD configuration. + +### Create a Token (For simple setups) + +```bash +# Create a token with the reader policy +vault token create -policy=varlock-reader -ttl=24h +``` + +### Store Secrets + +```bash +# Store a single key/value +vault kv put secret/myapp/config DB_PASSWORD=supersecret + +# Store multiple keys +vault kv put secret/myapp/config \ + DB_HOST=db.example.com \ + DB_PASSWORD=supersecret \ + API_KEY=abc123 +``` + +## Troubleshooting + +### Secret not found +- Verify the secret exists: `vault kv get secret/myapp/config` +- Check the mount point is correct (first path segment, typically `secret`) +- Ensure you're using KV v2, not KV v1 (different API format) + +### Permission denied +- Check your token's policies: `vault token lookup` +- Ensure your policy includes `read` capability on `secret/data/*` (note the `/data/` prefix for KV v2) +- For AppRole: verify the role has the correct policies attached + +### Authentication failed +- **Local dev:** Run `vault login` (or `bao login` for OpenBao) and ensure `VAULT_ADDR` is set correctly +- **CI/CD:** Verify your token or AppRole credentials are properly wired up in `@initHcpVault()` +- Check if the token has expired: `vault token lookup` +- For AppRole: verify the secret ID hasn't expired and generate a new one if needed diff --git a/packages/plugins/hashicorp-vault/package.json b/packages/plugins/hashicorp-vault/package.json new file mode 100644 index 000000000..c72f9f623 --- /dev/null +++ b/packages/plugins/hashicorp-vault/package.json @@ -0,0 +1,55 @@ +{ + "name": "@varlock/hashicorp-vault-plugin", + "description": "Varlock plugin to load secrets from HashiCorp Vault (KV v2 secrets engine) / OpenBao", + "version": "0.0.1", + "type": "module", + "homepage": "https://varlock.dev/plugins/hashicorp-vault/", + "bugs": "https://github.com/dmno-dev/varlock/issues", + "repository": { + "type": "git", + "url": "https://github.com/dmno-dev/varlock.git", + "directory": "packages/plugins/hashicorp-vault" + }, + "main": "dist/index.js", + "types": "dist/index.d.ts", + "exports": { + "./plugin": "./dist/plugin.js" + }, + "files": ["dist"], + "scripts": { + "dev": "tsup --watch", + "build": "tsup", + "test": "vitest" + }, + "keywords": [ + "varlock", + "plugin", + "varlock-plugin", + "hashicorp", + "vault", + "secrets", + "kv", + "env", + ".env", + "dotenv", + "environment variables", + "env vars", + "config" + ], + "author": "dmno-dev", + "license": "MIT", + "engines": { + "node": ">=22" + }, + "peerDependencies": { + "varlock": "workspace:^" + }, + "devDependencies": { + "@env-spec/utils": "workspace:^", + "@types/node": "catalog:", + "ky": "catalog:", + "tsup": "catalog:", + "varlock": "workspace:^", + "vitest": "catalog:" + } +} diff --git a/packages/plugins/hashicorp-vault/src/plugin.ts b/packages/plugins/hashicorp-vault/src/plugin.ts new file mode 100644 index 000000000..183947dbc --- /dev/null +++ b/packages/plugins/hashicorp-vault/src/plugin.ts @@ -0,0 +1,543 @@ +import { Resolver } from 'varlock/plugin-lib'; +import ky from 'ky'; +import { readFile } from 'node:fs/promises'; +import { homedir } from 'node:os'; +import { join } from 'node:path'; + +const { SchemaError, ResolutionError } = plugin.ERRORS; + +const VAULT_ICON = 'simple-icons:vault'; + +plugin.name = 'vault'; +const { debug } = plugin; +debug('init - version =', plugin.version); +plugin.icon = VAULT_ICON; +plugin.standardVars = { + initDecorator: '@initHcpVault', + params: { + url: { key: 'VAULT_ADDR' }, + token: { key: 'VAULT_TOKEN', dataType: 'vaultToken' }, + namespace: { key: 'VAULT_NAMESPACE' }, + }, +}; + +const FIX_AUTH_TIP = [ + 'Verify your Vault credentials are configured correctly. Use one of the following options:', + ' 1. Provide a token explicitly via @initHcpVault(token=$VAULT_TOKEN)', + ' 2. Use AppRole auth via @initHcpVault(roleId=..., secretId=...)', + ' 3. Login via the vault CLI (vault login) - the ~/.vault-token file will be used automatically', +].join('\n'); + +interface CachedToken { + token: string; + expiresAt: number; +} + +class VaultPluginInstance { + private url?: string; + private namespace?: string; + private token?: string; + private roleId?: string; + private secretId?: string; + private defaultPath?: string; + private pathPrefix?: string; + private cachedToken?: CachedToken; + private secretCache = new Map>>(); + + constructor( + readonly id: string, + ) {} + + setAuth( + url?: any, + namespace?: any, + token?: any, + roleId?: any, + secretId?: any, + defaultPath?: any, + pathPrefix?: any, + ) { + this.url = url ? String(url).replace(/\/+$/, '') : undefined; + this.namespace = namespace ? String(namespace) : undefined; + this.token = token ? String(token) : undefined; + this.roleId = roleId ? String(roleId) : undefined; + this.secretId = secretId ? String(secretId) : undefined; + this.defaultPath = defaultPath ? String(defaultPath) : undefined; + this.pathPrefix = pathPrefix ? String(pathPrefix) : undefined; + debug( + 'vault instance', + this.id, + 'set auth - url:', + this.url, + 'hasToken:', + !!this.token, + 'hasRoleId:', + !!this.roleId, + 'namespace:', + this.namespace, + 'defaultPath:', + this.defaultPath, + 'pathPrefix:', + this.pathPrefix, + ); + } + + private async getVaultToken(): Promise { + // Check cached token (with 30s buffer) + if (this.cachedToken && this.cachedToken.expiresAt > Date.now() + 30_000) { + debug('Using cached Vault token'); + return this.cachedToken.token; + } + + // 1. Explicit token + if (this.token) { + debug('Using explicitly provided Vault token'); + return this.token; + } + + // 2. AppRole auth + if (this.roleId && this.secretId) { + return this.loginWithAppRole(); + } + + // 3. Vault/OpenBao CLI token helper files + for (const tokenFile of ['.vault-token', '.bao-token']) { + try { + const tokenPath = join(homedir(), tokenFile); + const fileToken = (await readFile(tokenPath, 'utf-8')).trim(); + if (fileToken) { + debug(`Using token from ~/${tokenFile}`); + return fileToken; + } + } catch { + debug(`Could not read ~/${tokenFile}`); + } + } + + throw new SchemaError('No Vault authentication found', { + tip: FIX_AUTH_TIP, + }); + } + + private async loginWithAppRole(): Promise { + if (!this.url) throw new SchemaError('Vault URL is required for AppRole auth'); + + try { + const headers: Record = {}; + if (this.namespace) headers['X-Vault-Namespace'] = this.namespace; + + const response = await ky.post(`${this.url}/v1/auth/approle/login`, { + json: { role_id: this.roleId, secret_id: this.secretId }, + headers, + }).json<{ auth: { client_token: string; lease_duration: number } }>(); + + const clientToken = response.auth.client_token; + const leaseDuration = response.auth.lease_duration; + + this.cachedToken = { + token: clientToken, + expiresAt: Date.now() + (leaseDuration * 1000), + }; + + debug('Successfully authenticated with AppRole'); + return clientToken; + } catch (err: any) { + let errorMessage = 'AppRole authentication failed'; + let errorTip: string | undefined; + + if (err.response) { + const status = err.response.status; + if (status === 400) { + errorMessage = 'AppRole login failed - invalid role_id or secret_id'; + errorTip = [ + 'Verify your AppRole credentials:', + ' - role_id: The UUID of the AppRole role', + ' - secret_id: A valid secret ID for the role', + 'Generate a new secret_id: vault write -f auth/approle/role//secret-id', + ].join('\n'); + } else if (status === 403) { + errorMessage = 'AppRole login denied - insufficient permissions'; + errorTip = 'Ensure the AppRole auth method is enabled and the role exists'; + } else { + errorMessage = `AppRole login failed (HTTP ${status})`; + } + } else if (err.message) { + errorMessage = `AppRole login error: ${err.message}`; + } + + throw new SchemaError(errorMessage, { tip: errorTip }); + } + } + + buildPath(explicitPath?: string): string { + let basePath: string; + if (explicitPath) { + basePath = explicitPath; + } else if (this.defaultPath) { + basePath = this.defaultPath; + } else { + throw new ResolutionError('No path specified and no defaultPath configured', { + tip: 'Either provide a path argument to vaultSecret() or set defaultPath in @initHcpVault()', + }); + } + + if (this.pathPrefix) { + const prefix = this.pathPrefix.replace(/\/+$/, ''); + const path = basePath.replace(/^\/+/, ''); + return `${prefix}/${path}`; + } + return basePath; + } + + private fetchSecretData(secretPath: string): Promise> { + // Deduplicate concurrent fetches for the same path + const cached = this.secretCache.get(secretPath); + if (cached) { + debug(`Using cached fetch for: ${secretPath}`); + return cached; + } + + const promise = this._fetchSecretData(secretPath); + this.secretCache.set(secretPath, promise); + // Clear cache entry on failure so retries can try again + promise.catch(() => this.secretCache.delete(secretPath)); + return promise; + } + + private async _fetchSecretData(secretPath: string): Promise> { + if (!this.url) { + throw new SchemaError('Vault URL is required'); + } + + const token = await this.getVaultToken(); + + // Split path into mount and secret path for KV v2 API + // e.g., "secret/apple/music" -> mount="secret", kvPath="apple/music" + // API endpoint: GET /v1/{mount}/data/{kvPath} + const slashIdx = secretPath.indexOf('/'); + let mount: string; + let kvPath: string; + if (slashIdx === -1) { + mount = secretPath; + kvPath = ''; + } else { + mount = secretPath.substring(0, slashIdx); + kvPath = secretPath.substring(slashIdx + 1); + } + + const apiUrl = kvPath + ? `${this.url}/v1/${mount}/data/${kvPath}` + : `${this.url}/v1/${mount}/data`; + + const headers: Record = { 'X-Vault-Token': token }; + if (this.namespace) headers['X-Vault-Namespace'] = this.namespace; + + try { + debug(`Fetching secret: ${secretPath}`); + + const response = await ky.get(apiUrl, { headers }) + .json<{ data: { data: Record; metadata: any } }>(); + + const secretData = response.data?.data; + if (!secretData) { + throw new ResolutionError('Secret data is empty'); + } + + return secretData; + } catch (err: any) { + if (err instanceof ResolutionError) throw err; + + let errorMessage = 'Failed to fetch secret'; + let errorTip: string | undefined; + + if (err.response) { + const status = err.response.status; + + if (status === 404) { + errorMessage = `Secret at path "${secretPath}" not found`; + errorTip = [ + 'Verify the secret exists in Vault:', + ` vault kv get ${secretPath}`, + '', + 'Common issues:', + ' - The mount point may be wrong (first path segment)', + ' - The secret path may not exist', + ' - KV v1 secrets use a different API format', + ].join('\n'); + } else if (status === 403) { + errorMessage = `Permission denied for path "${secretPath}"`; + errorTip = [ + 'Ensure your token has the correct policy. Required capability: read', + 'Example policy:', + ` path "${mount}/data/${kvPath || '*'}" {`, + ' capabilities = ["read"]', + ' }', + ].join('\n'); + } else if (status === 401) { + errorMessage = 'Vault authentication failed'; + errorTip = [ + 'Your token may be expired or invalid', + FIX_AUTH_TIP, + ].join('\n'); + } else if (status === 503) { + errorMessage = 'Vault server is sealed or unavailable'; + errorTip = 'Check that the Vault server is running and unsealed'; + } else { + try { + const errorBody = await err.response.json(); + const errors = errorBody.errors?.join('; ') || ''; + errorMessage = `Vault error (HTTP ${status}): ${errors || err.message}`; + } catch { + errorMessage = `Vault error (HTTP ${status})`; + } + } + } else if (err.message) { + errorMessage = `Network error: ${err.message}`; + errorTip = 'Verify the Vault URL is correct and the server is reachable'; + } + + throw new ResolutionError(errorMessage, { + tip: errorTip, + }); + } + } + + async getSecret(secretPath: string, jsonKey?: string): Promise { + const secretData = await this.fetchSecretData(secretPath); + + // If a JSON key is specified, extract it + if (jsonKey) { + if (!(jsonKey in secretData)) { + throw new ResolutionError(`Key "${jsonKey}" not found in secret`, { + tip: `Available keys: ${Object.keys(secretData).join(', ')}`, + }); + } + return String(secretData[jsonKey]); + } + + // No key specified: if single key, return its value; otherwise return JSON + const keys = Object.keys(secretData); + if (keys.length === 1) { + return String(secretData[keys[0]]); + } + return JSON.stringify(secretData); + } +} + +const pluginInstances: Record = {}; + +plugin.registerRootDecorator({ + name: 'initHcpVault', + description: 'Initialize a HashiCorp Vault plugin instance for vaultSecret() resolver', + isFunction: true, + async process(argsVal) { + const objArgs = argsVal.objArgs; + if (!objArgs) throw new SchemaError('Expected some args'); + + // Validate id is static + if (objArgs.id && !objArgs.id.isStatic) { + throw new SchemaError('Expected id to be static'); + } + const id = String(objArgs?.id?.staticValue || '_default'); + if (pluginInstances[id]) { + throw new SchemaError(`Instance with id "${id}" already initialized`); + } + + // url is required + if (!objArgs.url) { + throw new SchemaError('url parameter is required', { + tip: 'Provide your Vault server URL: @initHcpVault(url="https://vault.example.com:8200")', + }); + } + + pluginInstances[id] = new VaultPluginInstance(id); + + return { + id, + urlResolver: objArgs.url, + namespaceResolver: objArgs.namespace, + tokenResolver: objArgs.token, + roleIdResolver: objArgs.roleId, + secretIdResolver: objArgs.secretId, + defaultPathResolver: objArgs.defaultPath, + pathPrefixResolver: objArgs.pathPrefix, + }; + }, + async execute({ + id, + urlResolver, + namespaceResolver, + tokenResolver, + roleIdResolver, + secretIdResolver, + defaultPathResolver, + pathPrefixResolver, + }) { + const url = await urlResolver.resolve(); + const namespace = await namespaceResolver?.resolve(); + const token = await tokenResolver?.resolve(); + const roleId = await roleIdResolver?.resolve(); + const secretId = await secretIdResolver?.resolve(); + const defaultPath = await defaultPathResolver?.resolve(); + const pathPrefix = await pathPrefixResolver?.resolve(); + pluginInstances[id].setAuth(url, namespace, token, roleId, secretId, defaultPath, pathPrefix); + }, +}); + +plugin.registerDataType({ + name: 'vaultToken', + sensitive: true, + typeDescription: 'HashiCorp Vault authentication token', + icon: VAULT_ICON, + docs: [ + { + description: 'Vault Tokens', + url: 'https://developer.hashicorp.com/vault/docs/concepts/tokens', + }, + ], +}); + +plugin.registerResolverFunction({ + name: 'vaultSecret', + label: 'Fetch secret from HashiCorp Vault KV v2', + icon: VAULT_ICON, + argsSchema: { + type: 'mixed', + arrayMinLength: 0, + arrayMaxLength: 2, + }, + process() { + let instanceId: string; + let secretRefResolver: Resolver | undefined; + let keyResolver: Resolver | undefined; + let returnJson = false; + + // Check for named 'key' parameter + if (this.objArgs?.key) { + keyResolver = this.objArgs.key; + } + + // Check for named 'raw' flag - returns full secret as JSON blob instead of extracting a key + if (this.objArgs?.raw) { + if (!this.objArgs.raw.isStatic || this.objArgs.raw.staticValue !== true) { + throw new SchemaError('Expected raw=true'); + } + if (keyResolver) { + throw new SchemaError('Cannot use raw=true with key= parameter', { + tip: 'Remove the key= parameter, or remove raw=true', + }); + } + returnJson = true; + } + + // Get the item key for auto-inferring the JSON key + const parent = (this as any).parent; + const itemKey = parent?.key || ''; + + if (!this.arrArgs || this.arrArgs.length === 0) { + instanceId = '_default'; + } else if (this.arrArgs.length === 1) { + instanceId = '_default'; + secretRefResolver = this.arrArgs[0]; + } else if (this.arrArgs.length === 2) { + if (!(this.arrArgs[0].isStatic)) { + throw new SchemaError('Expected instance id to be a static value'); + } else { + instanceId = String(this.arrArgs[0].staticValue); + } + secretRefResolver = this.arrArgs[1]; + } else { + throw new SchemaError('Expected 0, 1, or 2 args'); + } + + if (!Object.values(pluginInstances).length) { + throw new SchemaError('No Vault plugin instances found', { + tip: 'Initialize at least one Vault instance using the @initHcpVault() root decorator', + }); + } + + // Make sure instance id is valid + const selectedInstance = pluginInstances[instanceId]; + if (!selectedInstance) { + if (instanceId === '_default') { + throw new SchemaError('Vault plugin instance (without id) not found', { + tip: [ + 'Either remove the `id` param from your @initHcpVault call', + 'or use `vaultSecret(id, secretRef)` to select an instance by id.', + `Possible ids are: ${Object.keys(pluginInstances).join(', ')}`, + ].join('\n'), + }); + } else { + throw new SchemaError(`Vault plugin instance id "${instanceId}" not found`, { + tip: [`Valid ids are: ${Object.keys(pluginInstances).join(', ')}`].join('\n'), + }); + } + } + + return { + instanceId, secretRefResolver, keyResolver, itemKey, returnJson, + }; + }, + async resolve({ + instanceId, secretRefResolver, keyResolver, itemKey, returnJson, + }) { + const selectedInstance = pluginInstances[instanceId]; + + // Resolve the secret ref (path, possibly with #KEY syntax) + let secretRef: string | undefined; + if (secretRefResolver) { + const resolved = await secretRefResolver.resolve(); + if (typeof resolved !== 'string') { + throw new SchemaError('Expected secret reference to resolve to a string'); + } + secretRef = resolved; + } + + // Parse for explicit JSON key using # syntax + // e.g., "secret/path#KEY_NAME" -> path="secret/path", explicitKey="KEY_NAME" + let secretPath: string; + let explicitKey: string | undefined; + if (secretRef) { + const hashIndex = secretRef.indexOf('#'); + if (hashIndex !== -1) { + secretPath = secretRef.substring(0, hashIndex); + explicitKey = secretRef.substring(hashIndex + 1); + } else { + secretPath = secretRef; + } + } else { + // No path arg — will use defaultPath via buildPath + secretPath = ''; + } + + // Validate conflicting options + if (returnJson && explicitKey) { + throw new SchemaError('Cannot use raw=true with #KEY syntax', { + tip: 'Remove the #KEY from the path, or remove raw=true', + }); + } + + // Determine the JSON key to extract: + // 1. Named 'key' parameter takes highest precedence + // 2. Explicit #KEY in path + // 3. json=true skips key extraction entirely + // 4. Default: use the item key name + let jsonKey: string | undefined; + if (keyResolver) { + const keyValue = await keyResolver.resolve(); + if (typeof keyValue !== 'string') { + throw new SchemaError('Expected key parameter to resolve to a string'); + } + jsonKey = keyValue; + } else if (explicitKey) { + jsonKey = explicitKey; + } else if (!returnJson) { + jsonKey = itemKey || undefined; + } + + // Build the full path using pathPrefix/defaultPath + const fullPath = selectedInstance.buildPath(secretPath || undefined); + + return await selectedInstance.getSecret(fullPath, jsonKey); + }, +}); diff --git a/packages/plugins/hashicorp-vault/tsconfig.json b/packages/plugins/hashicorp-vault/tsconfig.json new file mode 100644 index 000000000..2e3dad13d --- /dev/null +++ b/packages/plugins/hashicorp-vault/tsconfig.json @@ -0,0 +1,13 @@ +{ + "compilerOptions": { + "target": "es2021", + "moduleResolution": "bundler", + "strict": true, + "resolveJsonModule": true, + "esModuleInterop": true, + "skipLibCheck": false, + "customConditions": ["ts-src"] + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "dist"] +} diff --git a/packages/plugins/hashicorp-vault/tsup.config.ts b/packages/plugins/hashicorp-vault/tsup.config.ts new file mode 100644 index 000000000..a4ffa83a9 --- /dev/null +++ b/packages/plugins/hashicorp-vault/tsup.config.ts @@ -0,0 +1,14 @@ +import { defineConfig } from 'tsup'; + +export default defineConfig({ + entry: ['src/plugin.ts'], + dts: true, + sourcemap: true, + treeshake: true, + clean: false, + outDir: 'dist', + format: ['cjs'], + splitting: false, + target: 'esnext', + external: ['varlock'], +}); diff --git a/packages/varlock-website/astro.config.ts b/packages/varlock-website/astro.config.ts index 620e5d955..1e80ddcc4 100644 --- a/packages/varlock-website/astro.config.ts +++ b/packages/varlock-website/astro.config.ts @@ -184,6 +184,7 @@ export default defineConfig({ { label: 'Azure Key Vault', slug: 'plugins/azure-key-vault' }, { label: 'Bitwarden', slug: 'plugins/bitwarden' }, { label: 'GCP Secret Manager', slug: 'plugins/google-secret-manager' }, + { label: 'HashiCorp Vault', slug: 'plugins/hashicorp-vault' }, { label: 'Infisical', slug: 'plugins/infisical' }, { label: 'Pass', slug: 'plugins/pass' }, ], diff --git a/packages/varlock-website/src/content/docs/plugins/hashicorp-vault.mdx b/packages/varlock-website/src/content/docs/plugins/hashicorp-vault.mdx new file mode 100644 index 000000000..50081be85 --- /dev/null +++ b/packages/varlock-website/src/content/docs/plugins/hashicorp-vault.mdx @@ -0,0 +1,419 @@ +--- +title: HashiCorp Vault Plugin +description: Using HashiCorp Vault and OpenBao with Varlock +--- + +import { Steps, Icon } from '@astrojs/starlight/components'; +import Badge from '@/components/Badge.astro'; + +
+ +
+ +Our HashiCorp Vault plugin enables secure loading of secrets from [HashiCorp Vault](https://www.vaultproject.io/) (KV v2 secrets engine) and [OpenBao](https://openbao.org/) using declarative instructions within your `.env` files. + +The plugin supports multiple authentication methods including explicit tokens, AppRole for CI/CD, and automatic CLI token detection for local development. + +## Features + +- **Zero-config authentication** - Automatically uses Vault token from CLI login +- **AppRole authentication** - For automated and CI/CD workflows +- **Vault CLI integration** - Works seamlessly with `vault login` for local development +- **OpenBao compatible** - Works with OpenBao (detects `~/.bao-token` automatically) +- **Auto-infer secret keys** from environment variable names +- **JSON key extraction** from secrets using `#` syntax or named `key` parameter +- **Path prefixing** with `pathPrefix` option for organized secret management +- **Default path** support for sharing a common secret path across items +- Support for Vault Enterprise namespaces +- Support for multiple Vault instances + +## Installation and setup + +In a JS/TS project, you may install the `@varlock/hashicorp-vault-plugin` package as a normal dependency. +Otherwise you can just load it directly from your `.env.schema` file, as long as you add a version specifier. +See the [plugins guide](/guides/plugins/#installation) for more instructions on installing plugins. + +```env-spec title=".env.schema" +# 1. Load the plugin +# @plugin(@varlock/hashicorp-vault-plugin) +# +# 2. Initialize the plugin - see below for more details on options +# @initHcpVault(url="https://vault.example.com:8200") +``` + +### Authentication options + +The plugin tries authentication methods in this priority order: +1. **Explicit token** - If `token` is provided in `@initHcpVault()` +2. **AppRole** - If both `roleId` and `secretId` are provided +3. **CLI token file** - From `~/.vault-token` (created by `vault login`) or `~/.bao-token` (created by `bao login` for OpenBao) + +### Automatic authentication (Recommended for local dev) + +For local development, just provide the Vault URL - the plugin will pick up your CLI token automatically: + +```env-spec title=".env.schema" +# @plugin(@varlock/hashicorp-vault-plugin) +# @initHcpVault(url="https://vault.example.com:8200") +``` + +**How this works:** + +- **Local development:** Run `vault login` → automatically uses the token from `~/.vault-token` +- **OpenBao users:** Run `bao login` → automatically uses the token from `~/.bao-token` + +### AppRole auth (For CI/CD and automated workflows) + +AppRole is the recommended auth method for CI/CD and server environments: + + +1. **Set up AppRole in Vault** (see Vault Setup section below) + +2. **Wire up the credentials in your config**. Add config items for the role ID and secret ID, and reference them when initializing the plugin. + + ```env-spec title=".env.schema" + # @plugin(@varlock/hashicorp-vault-plugin) + # @initHcpVault( + # url="https://vault.example.com:8200", + # roleId=$VAULT_ROLE_ID, + # secretId=$VAULT_SECRET_ID + # ) + # --- + + VAULT_ROLE_ID= + # @sensitive + VAULT_SECRET_ID= + ``` + +3. **Set your credentials in deployed environments**. Use your platform's env var management UI to securely inject these values. + + +### Explicit token + +You can also provide a token directly: + +```env-spec title=".env.schema" +# @initHcpVault( +# url="https://vault.example.com:8200", +# token=$VAULT_TOKEN +# ) +# --- + +# @type=vaultToken @sensitive +VAULT_TOKEN= +``` + +### Vault Enterprise namespaces + +For Vault Enterprise, specify the namespace: + +```env-spec title=".env.schema" +# @initHcpVault(url="https://vault.example.com:8200", namespace="admin/team-a") +``` + +### Multiple instances + +If you need to connect to multiple Vault instances, register named instances: + +```env-spec title=".env.schema" +# @initHcpVault(id=prod, url="https://vault-prod.example.com:8200") +# @initHcpVault(id=dev, url="https://vault-dev.example.com:8200") +# --- + +PROD_KEY=vaultSecret(prod, "secret/api/keys#API_KEY") +DEV_KEY=vaultSecret(dev, "secret/api/keys#API_KEY") +``` + +## Loading secrets + +Once the plugin is installed and initialized, you can start adding config items that load values using the `vaultSecret()` resolver function. + +### Basic usage + +Since Vault KV v2 always stores key/value pairs, the item key (variable name) is automatically used as the JSON key to extract from the secret: + +```env-spec title=".env.schema" +# Fetches "secret/db/config" and extracts "DB_HOST" key +DB_HOST=vaultSecret("secret/db/config") + +# Override the extracted key with # syntax +DB_PASSWORD=vaultSecret("secret/db/config#password") + +# Or use named "key" parameter +DB_PORT=vaultSecret("secret/db/config", key="PORT") + +# Fetch entire secret as JSON blob +DB_CONFIG=vaultSecret("secret/db/config", raw=true) +``` + +### Default path + +Use `defaultPath` to set a common path for secrets when no path argument is provided: + +```env-spec title=".env.schema" +# @initHcpVault(url="https://vault.example.com:8200", defaultPath=secret/myapp/config) +# --- + +# Both fetch from "secret/myapp/config" extracting item key +DB_PASSWORD=vaultSecret() +API_KEY=vaultSecret() + +# Override the inferred key using # syntax +STRIPE_KEY=vaultSecret("#stripe_api_key") + +# Explicit path still extracts item key by default +OTHER_SECRET=vaultSecret("secret/other/path") +``` + +### Path prefixing + +Use `pathPrefix` to automatically prefix all secret paths for better organization: + +```env-spec title=".env.schema" +# @initHcpVault(url="https://vault.example.com:8200", pathPrefix="secret/myapp") +# --- + +# Fetches from "secret/myapp/db/config" +DB_HOST=vaultSecret("db/config#HOST") +``` + +You can even use dynamic prefixes: + +```env-spec title=".env.schema" +# @initHcpVault(url="https://vault.example.com:8200", pathPrefix="secret/${ENV}") +# --- + +# In prod: fetches from "secret/prod/db/config" +# In dev: fetches from "secret/dev/db/config" +DB_HOST=vaultSecret("db/config#HOST") +``` + +### Bulk loading secrets + +Use `raw=true` with `@setValuesBulk` to load all key/value pairs from a Vault path at once, instead of wiring up each secret individually: + +```env-spec title=".env.schema" +# @initHcpVault(url="https://vault.example.com:8200") +# @setValuesBulk(vaultSecret("secret/myapp/config", raw=true)) +# --- + +DB_HOST= +DB_PASSWORD= +API_KEY= +``` + +This fetches all keys stored at `secret/myapp/config` and maps them to matching item keys. Only items declared in your schema will be populated — any extra keys in the Vault secret are ignored. + +--- + +## Vault Setup + +### Enable KV v2 secrets engine + +```bash +# KV v2 is enabled by default at "secret/" in dev mode +# For production, enable it explicitly: +vault secrets enable -version=2 -path=secret kv +``` + +### Create a policy + +Create a policy that allows reading secrets: + +```hcl title="policy.hcl" +path "secret/data/*" { + capabilities = ["read"] +} +``` + +```bash +vault policy write varlock-reader policy.hcl +``` + +:::tip[Least privilege principle] +For production, scope down the `path` to only the specific secret paths your application needs. For example: `path "secret/data/myapp/*"` +::: + +### Set up AppRole auth (Recommended for CI/CD) + + +1. **Enable AppRole auth method** + ```bash + vault auth enable approle + ``` + +2. **Create a role** + ```bash + vault write auth/approle/role/varlock-role \ + secret_id_ttl=24h \ + token_ttl=1h \ + token_max_ttl=4h \ + token_policies=varlock-reader + ``` + +3. **Get the role ID and generate a secret ID** + ```bash + vault read auth/approle/role/varlock-role/role-id + vault write -f auth/approle/role/varlock-role/secret-id + ``` + Save the `role_id` and `secret_id` from the output for your CI/CD configuration. + + +### Create a token (For simple setups) + +```bash +vault token create -policy=varlock-reader -ttl=24h +``` + +### Store secrets + +```bash +# Store a single key/value +vault kv put secret/myapp/config DB_PASSWORD=supersecret + +# Store multiple keys +vault kv put secret/myapp/config \ + DB_HOST=db.example.com \ + DB_PASSWORD=supersecret \ + API_KEY=abc123 +``` + +### Vault CLI for local development + + +1. **Set the Vault address** + ```bash + export VAULT_ADDR="https://vault.example.com:8200" + ``` + +2. **Login to Vault** + ```bash + vault login + ``` + This writes a token to `~/.vault-token` which the plugin will automatically pick up. + +3. **Test the configuration** + ```bash + vault kv get secret/myapp/config + ``` + + +--- + +## Reference + +### Root decorators +
+
+#### `@initHcpVault()` + +Initialize a HashiCorp Vault / OpenBao plugin instance. + +**Key/value args:** +- `url` (required): Vault server URL (e.g., `https://vault.example.com:8200`) +- `token` (optional): Explicit Vault authentication token +- `roleId` (optional): AppRole role ID for automated authentication +- `secretId` (optional): AppRole secret ID for automated authentication +- `namespace` (optional): Vault Enterprise namespace +- `defaultPath` (optional): Default secret path when no path argument is given to `vaultSecret()` +- `pathPrefix` (optional): Prefix automatically prepended to all secret paths +- `id` (optional): Instance identifier for multiple instances + +```env-spec "@initHcpVault" +# @initHcpVault(url="https://vault.example.com:8200", defaultPath=secret/myapp/config) +``` +
+
+ +### Data types +
+
+#### `vaultToken` + +Represents a HashiCorp Vault authentication token. This type is marked as `@sensitive`. + +```env-spec "vaultToken" +# @type=vaultToken +VAULT_TOKEN= +``` +
+
+ +### Resolver functions +
+
+#### `vaultSecret()` + +Fetch a secret from HashiCorp Vault's KV v2 secrets engine. + +**Array args:** +- `instanceId` (optional): instance identifier to use when multiple plugin instances are initialized +- `secretRef` (optional): secret path, optionally with `#KEY` to override the extracted key. If omitted, uses `defaultPath`. Use `#key` (without a path) to override the inferred key while still using `defaultPath`. + +**Named args:** +- `key` (optional): JSON key to extract from the secret (overrides `#KEY` syntax and item key default) +- `raw` (optional): set to `true` to return all key/value pairs as a JSON blob instead of extracting a single key. Useful with `@setValuesBulk` for bulk loading. + +**Key extraction:** By default, the item key (variable name) is used as the JSON key. Override with `#KEY` or `key=`, or use `raw=true` to get everything. + +**How paths work:** + +Vault KV v2 stores key/value pairs at a path. Given a path like `secret/myapp/config`, the plugin calls `GET /v1/secret/data/myapp/config` (the first path segment is the mount point, and `/data/` is inserted for the KV v2 API). + +```env-spec /vaultSecret\\(.*\\)/ +# Uses defaultPath, extracts item key +DATABASE_URL=vaultSecret() + +# Override inferred key using # syntax (still uses defaultPath) +STRIPE_KEY=vaultSecret("#stripe_api_key") + +# Explicit path, extracts item key "DB_HOST" +DB_HOST=vaultSecret("secret/db/config") + +# Override key with # syntax +DB_PASSWORD=vaultSecret("secret/db/config#password") + +# Extract key (named parameter) +DB_PORT=vaultSecret("secret/db/config", key="PORT") + +# Fetch full secret as JSON +DB_CONFIG=vaultSecret("secret/db/config", raw=true) + +# With instance ID +PROD_SECRET=vaultSecret(prod, "secret/api/keys") +``` +
+
+ +--- + +## Troubleshooting + +### Secret not found +- Verify the secret exists: `vault kv get secret/myapp/config` +- Check the mount point is correct (first path segment, typically `secret`) +- Ensure you're using KV v2, not KV v1 (different API format) + +### Permission denied +- Check your token's policies: `vault token lookup` +- Ensure your policy includes `read` capability on `secret/data/*` (note the `/data/` prefix for KV v2) +- For AppRole: verify the role has the correct policies attached + +### Authentication failed +- **Local dev:** Run `vault login` (or `bao login` for OpenBao) and ensure `VAULT_ADDR` is set correctly +- **CI/CD:** Verify your token or AppRole credentials are properly wired up in `@initHcpVault()` +- Check if the token has expired: `vault token lookup` +- For AppRole: verify the secret ID hasn't expired and generate a new one if needed + +### JSON key not found +- Verify the key exists at the path: `vault kv get -field=MY_KEY secret/myapp/config` +- Key names are case-sensitive +- Check available keys: `vault kv get secret/myapp/config` + +## Resources + +- [HashiCorp Vault](https://www.vaultproject.io/) +- [OpenBao](https://openbao.org/) +- [Vault KV v2 Secrets Engine](https://developer.hashicorp.com/vault/docs/secrets/kv/kv-v2) +- [Vault AppRole Auth Method](https://developer.hashicorp.com/vault/docs/auth/approle) +- [Vault Tokens](https://developer.hashicorp.com/vault/docs/concepts/tokens) diff --git a/packages/varlock-website/src/content/docs/plugins/overview.mdx b/packages/varlock-website/src/content/docs/plugins/overview.mdx index 3704a8a57..9a6146cea 100644 --- a/packages/varlock-website/src/content/docs/plugins/overview.mdx +++ b/packages/varlock-website/src/content/docs/plugins/overview.mdx @@ -18,6 +18,7 @@ For now, only official Varlock plugins under the `@varlock` npm scope are suppor | [Azure Key Vault](/plugins/azure-key-vault/) | | | [Bitwarden](/plugins/bitwarden/) | | | [Google Secrets Manager](/plugins/google-secret-manager/) | | +| [HashiCorp Vault](/plugins/hashicorp-vault/)
_also compatible with OpenBao_ | | | [Infisical](/plugins/infisical/) | | | [Pass](/plugins/pass/)
_the standard unix password manager_ | |