From 6e77f3bff7b58f4f051595a87b2949cdb1c8c960 Mon Sep 17 00:00:00 2001 From: Tigran Khudaverdyan Date: Mon, 31 Aug 2026 14:13:38 +0400 Subject: [PATCH 1/2] docs: module cache (TF_DATA_DIR) for workspaces Documents how to keep downloaded modules and providers between jobs of the same workspace: the helm chart executor.cache block, the executor's TerraformDataDirCacheRoot setting, and the plain TF_DATA_DIR workspace environment variable that works with any version. Cross-links the provider cache page. Co-Authored-By: Claude Fable 5 --- SUMMARY.md | 1 + user-guide/workspaces/module-cache.md | 57 +++++++++++++++++++++++++ user-guide/workspaces/provider-cache.md | 2 +- 3 files changed, 59 insertions(+), 1 deletion(-) create mode 100644 user-guide/workspaces/module-cache.md diff --git a/SUMMARY.md b/SUMMARY.md index 5e5fc22..6c09042 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -78,6 +78,7 @@ * [Terraform State](user-guide/workspaces/terraform-state.md) * [Share Workspace State](user-guide/workspaces/share-workspace-state.md) * [Provider Cache](user-guide/workspaces/provider-cache.md) + * [Module Cache](user-guide/workspaces/module-cache.md) * [Variables](user-guide/workspaces/variables.md) * [Dynamic Provider Credentials](user-guide/workspaces/dynamic-provider-credentials/README.md) * [AWS Dynamic Provider Credentials](user-guide/workspaces/dynamic-provider-credentials/aws-dynamic-provider-credentials.md) diff --git a/user-guide/workspaces/module-cache.md b/user-guide/workspaces/module-cache.md new file mode 100644 index 0000000..0f6e03f --- /dev/null +++ b/user-guide/workspaces/module-cache.md @@ -0,0 +1,57 @@ +# Module Cache + +The executor runs every job in a fresh clone of the workspace repository and deletes it afterwards. Terraform and OpenTofu keep the modules they download and the providers they install inside the `.terraform` directory of that clone, so **each job downloads every module and provider again**. The [Provider Cache](provider-cache.md) covers the providers; this page explains how to also keep the modules (and providers) between jobs of the same workspace. + +Terraform and OpenTofu support relocating the `.terraform` directory with the `TF_DATA_DIR` environment variable. When it points at a directory that already contains the modules and providers of a previous run, `init` reuses them and downloads nothing: + +``` +Initializing modules... +Initializing provider plugins... +- Reusing previous version of keycloak/keycloak from the dependency lock file +- Using previously-installed keycloak/keycloak v5.8.0 +Terraform has been successfully initialized! +``` + +There are three ways to set it up. + +## Helm chart + +Chart version 4.8.0 and later has an `executor.cache` block that mounts a cache volume and sets every variable described below in one place: + +```yaml +executor: + cache: + enabled: true + path: /home/cnb/.terraform.d # mount point of the cache volume + sizeLimit: 2Gi # emptyDir size; or use existingClaim for a PVC + existingClaim: "" # RWX PVC when replicaCount > 1 + providers: true # TF_PLUGIN_CACHE_DIR=/plugin-cache + ignoreLockFile: false # TF_PLUGIN_CACHE_MAY_BREAK_DEPENDENCY_LOCK_FILE=1 + modules: true # TerraformDataDirCacheRoot=/data (see below) +``` + +## Executor setting (automatic, per workspace) + +Set the executor environment variable `TerraformDataDirCacheRoot` to a directory on a mounted volume, for example `/home/cnb/.terraform.d/data`. For every job the executor then creates `//` and runs terraform/tofu with `TF_DATA_DIR` pointing at it, so the next job of the same workspace finds its modules and providers already in place. A `TF_DATA_DIR` defined as a workspace environment variable always takes precedence. This setting is ignored by executors that predate it. + +## Workspace environment variable (any version) + +Add an environment variable to the workspace: + +``` +TF_DATA_DIR=/home/cnb/.terraform.d/data/ +``` + +The directory must be on a volume mounted in the executor (the same `cache-volume` used by the provider cache works) and must be unique per workspace: never share one data directory between workspaces, because it also stores the backend configuration and the module manifest of that configuration. + +## Things to know + +* `terraform init` is executed without `-upgrade`, so a cached module or provider is kept as long as it satisfies the version constraint in the configuration. Pin module and provider versions; with a loose constraint such as `~> 1.0` a newer release is only picked up after the cache directory is deleted. +* Providers are reused from the data directory when the workspace commits its `.terraform.lock.hcl`. Without a lock file terraform re-verifies them against the registry on every run; use the provider cache with `TF_PLUGIN_CACHE_MAY_BREAK_DEPENDENCY_LOCK_FILE=1`, or commit the lock file (`terraform providers lock -platform=linux_amd64`). +* Terrakube runs one job at a time per workspace, so no two jobs write to the same data directory. With several executor replicas sharing one volume the claim must be `ReadWriteMany`. +* Space: one copy of each module call plus one copy of each provider version per workspace. An emptyDir is emptied when the pod restarts; use a PersistentVolumeClaim to keep the cache across restarts. +* To reset a workspace's cache simply delete its directory under the cache root. + +{% hint style="info" %} +This only speeds up module and provider downloads. Modules published in the Terrakube private registry are additionally cached in the Terrakube storage and served from there, see [Private Registry](../private-registry/README.md). +{% endhint %} diff --git a/user-guide/workspaces/provider-cache.md b/user-guide/workspaces/provider-cache.md index 235e977..d438b95 100644 --- a/user-guide/workspaces/provider-cache.md +++ b/user-guide/workspaces/provider-cache.md @@ -8,7 +8,7 @@ TF_PLUGIN_CACHE_DIR=/home/cnb/.terraform.d/plugin-cache
-When deploying the helm chart the following can be used to add an emptyDir volume to cache the providers. +When deploying the helm chart the following can be used to add an emptyDir volume to cache the providers (chart 4.8.0 and later can do this, plus the [module cache](module-cache.md), with the `executor.cache` block). ``` executor: From c20fbb6953306aa5f440f9f5f54363c784ded05c Mon Sep 17 00:00:00 2001 From: Tigran Khudaverdyan Date: Mon, 31 Aug 2026 15:10:27 +0400 Subject: [PATCH 2/2] docs(module-cache): note that TF_PLUGIN_CACHE_DIR must exist Co-Authored-By: Claude Fable 5 --- user-guide/workspaces/module-cache.md | 1 + 1 file changed, 1 insertion(+) diff --git a/user-guide/workspaces/module-cache.md b/user-guide/workspaces/module-cache.md index 0f6e03f..2b7930d 100644 --- a/user-guide/workspaces/module-cache.md +++ b/user-guide/workspaces/module-cache.md @@ -51,6 +51,7 @@ The directory must be on a volume mounted in the executor (the same `cache-volum * Terrakube runs one job at a time per workspace, so no two jobs write to the same data directory. With several executor replicas sharing one volume the claim must be `ReadWriteMany`. * Space: one copy of each module call plus one copy of each provider version per workspace. An emptyDir is emptied when the pod restarts; use a PersistentVolumeClaim to keep the cache across restarts. * To reset a workspace's cache simply delete its directory under the cache root. +* Terraform does not create `TF_PLUGIN_CACHE_DIR` itself ("the directory must already exist"), and a freshly mounted volume is empty, so `init` reports `The specified plugin cache dir ... cannot be opened`. The chart's `executor.cache` block creates the sub-directories with an init container; if you mount the volume by hand, add an init container (or a pre-init script) that runs `mkdir -p` for them. `TF_DATA_DIR` directories are created by Terraform on demand. {% hint style="info" %} This only speeds up module and provider downloads. Modules published in the Terrakube private registry are additionally cached in the Terrakube storage and served from there, see [Private Registry](../private-registry/README.md).