From 3ed08282bb2abbce5789553737663ce0da180815 Mon Sep 17 00:00:00 2001 From: Emma <98318442+spiceemma@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:05:05 -0700 Subject: [PATCH 1/8] docs(partitioning): Cayenne refuses stored data written under a different partition_by --- .../data-accelerators/cayenne/index.md | 1 + .../data-acceleration/partitioning.md | 22 +++++++++++++------ 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/website/docs/components/data-accelerators/cayenne/index.md b/website/docs/components/data-accelerators/cayenne/index.md index 894ca7c04..cb4221c6f 100644 --- a/website/docs/components/data-accelerators/cayenne/index.md +++ b/website/docs/components/data-accelerators/cayenne/index.md @@ -1027,6 +1027,7 @@ Before a source deletion, writes must stop and pending keys must reach zero **wh Consider the following limitations when using Spice Cayenne acceleration: - **Memory Mode Constraints**: `mode: memory` (fully in-RAM, ephemeral) is supported alongside `mode: file`, but it does not persist any data (the dataset reloads from its source on restart), does not support partitioned tables (`partition_by`), and enforces a hard per-table RAM bound instead of spilling to disk — a breach returns an error rather than growing without limit. Use `mode: file` when persistence across restarts is required. DML and [`retention_sql`](../../reference/spicepod/datasets#accelerationretention_sql) are not among the constraints — see [Writes in memory mode](#writes-in-memory-mode). +- **No Re-partitioning**: Cayenne does not re-partition stored data. A `mode: file` dataset whose stored data was written with a different `partition_by` fails to load. To change `partition_by`, stop Spice and delete the dataset's data directory and the Cayenne metastore directory. See [Changing `partition_by` after refresh](../../features/data-acceleration/partitioning.md#changing-partition_by-after-refresh). - **S3 Express Only**: Standard S3 buckets are not supported for remote storage. Only S3 Express One Zone directory buckets are supported. - **Unsupported Data Types**: `Interval`, `Duration`, `FixedSizeBinary`, `Union`, and `RunEndEncoded` types require `unsupported_type_action` configuration. - **Indexes**: `indexes` builds a read-path [secondary index](#secondary-indexes), not a database index — a `unique` entry does not constrain writes, and registration logs a warning saying so. Deduplication requires only `primary_key`, and the `on_conflict` the warning mentions is not needed. See [Duplicate primary keys in one write](#duplicate-primary-keys-in-one-write). diff --git a/website/docs/features/data-acceleration/partitioning.md b/website/docs/features/data-acceleration/partitioning.md index 878d56438..3899e7862 100644 --- a/website/docs/features/data-acceleration/partitioning.md +++ b/website/docs/features/data-acceleration/partitioning.md @@ -211,18 +211,26 @@ A `refresh_mode: full` refresh — and any other whole-table overwrite — repla ## Changing `partition_by` after refresh -Once partitions exist on disk, changing `partition_by` is rejected: +Spice does not re-partition existing acceleration data. An Arrow acceleration lives in memory and is rebuilt from the source on every start, so a new `partition_by` takes effect on restart. A Cayenne acceleration in `mode: file` keeps its data and catalog on disk, so its stored layout must match `partition_by` when the dataset loads. -> The `partition_by` expressions are different from the expressions used to create the existing partition files. Revert the `partition_by` expressions, delete the partition files, or change the location the partition files are stored to create new partitions. +A file-mode Cayenne dataset fails to load when its stored data was written with a different partitioning than its current `partition_by`. This covers adding `partition_by` to unpartitioned data, removing it from partitioned data, and changing the partition expressions. The runtime logs one error and does not retry. The error has this form, with the values in angle brackets filled in: -To re-partition: +```text +Failed to load dataset (cayenne): its acceleration data was written , but the dataset now sets . Cayenne does not re-partition existing acceleration data. To keep using the existing data, . To apply the new 'partition_by' instead, stop Spice, delete the acceleration data directory '' and the Cayenne metastore directory '', then start Spice to reload the dataset from the source. Other Cayenne datasets that use this metastore also reload from their sources. See: https://spiceai.org/docs/features/data-acceleration/partitioning +``` + +The stored partitioning reads `without 'partition_by'` or `partitioned by []`, and the configured value reads `no 'partition_by'` or `'partition_by: []'`. + +To keep the existing data, restore the `partition_by` the data was written with, or remove `partition_by` if the data was written without it. + +To apply the new `partition_by`: -1. Stop the runtime. -2. Delete the partitioned acceleration directory or point `accelerator_dir` at a fresh location. +1. Stop Spice. +2. Delete the acceleration data directory and the Cayenne metastore directory that the error names. The metastore directory is set by [`cayenne_metadata_dir`](../../components/data-accelerators/cayenne/index.md#metastore-location). Deleting only the data directory is not enough, because the metastore keeps the dataset's file list and refresh checkpoint. 3. Update `partition_by`. -4. Restart — Spice will rebuild from the source. +4. Start Spice. The dataset reloads from its source. -There is no automatic in-place re-partitioning. +The metastore holds the catalog for every Cayenne dataset that shares it, so deleting it also makes those datasets reload from their sources. ## Validation rules From 9e69702dd45695e86578239740ff002e5e798649 Mon Sep 17 00:00:00 2001 From: Emma <98318442+spiceemma@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:20:35 -0700 Subject: [PATCH 2/8] docs(cayenne): fix the partitioning link path --- website/docs/components/data-accelerators/cayenne/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/components/data-accelerators/cayenne/index.md b/website/docs/components/data-accelerators/cayenne/index.md index cb4221c6f..0edf2487c 100644 --- a/website/docs/components/data-accelerators/cayenne/index.md +++ b/website/docs/components/data-accelerators/cayenne/index.md @@ -1027,7 +1027,7 @@ Before a source deletion, writes must stop and pending keys must reach zero **wh Consider the following limitations when using Spice Cayenne acceleration: - **Memory Mode Constraints**: `mode: memory` (fully in-RAM, ephemeral) is supported alongside `mode: file`, but it does not persist any data (the dataset reloads from its source on restart), does not support partitioned tables (`partition_by`), and enforces a hard per-table RAM bound instead of spilling to disk — a breach returns an error rather than growing without limit. Use `mode: file` when persistence across restarts is required. DML and [`retention_sql`](../../reference/spicepod/datasets#accelerationretention_sql) are not among the constraints — see [Writes in memory mode](#writes-in-memory-mode). -- **No Re-partitioning**: Cayenne does not re-partition stored data. A `mode: file` dataset whose stored data was written with a different `partition_by` fails to load. To change `partition_by`, stop Spice and delete the dataset's data directory and the Cayenne metastore directory. See [Changing `partition_by` after refresh](../../features/data-acceleration/partitioning.md#changing-partition_by-after-refresh). +- **No Re-partitioning**: Cayenne does not re-partition stored data. A `mode: file` dataset whose stored data was written with a different `partition_by` fails to load. To change `partition_by`, stop Spice and delete the dataset's data directory and the Cayenne metastore directory. See [Changing `partition_by` after refresh](../../../features/data-acceleration/partitioning.md#changing-partition_by-after-refresh). - **S3 Express Only**: Standard S3 buckets are not supported for remote storage. Only S3 Express One Zone directory buckets are supported. - **Unsupported Data Types**: `Interval`, `Duration`, `FixedSizeBinary`, `Union`, and `RunEndEncoded` types require `unsupported_type_action` configuration. - **Indexes**: `indexes` builds a read-path [secondary index](#secondary-indexes), not a database index — a `unique` entry does not constrain writes, and registration logs a warning saying so. Deduplication requires only `primary_key`, and the `on_conflict` the warning mentions is not needed. See [Duplicate primary keys in one write](#duplicate-primary-keys-in-one-write). From 100a3c736b8ac44237f2de3ba2055b73f0f14ddd Mon Sep 17 00:00:00 2001 From: Emma <98318442+spiceemma@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:33:56 -0700 Subject: [PATCH 3/8] docs(partitioning): the check compares partition names; back up before deleting a shared metastore --- .../docs/components/data-accelerators/cayenne/index.md | 2 +- website/docs/features/data-acceleration/partitioning.md | 8 +++++--- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/website/docs/components/data-accelerators/cayenne/index.md b/website/docs/components/data-accelerators/cayenne/index.md index 0edf2487c..b7ee67631 100644 --- a/website/docs/components/data-accelerators/cayenne/index.md +++ b/website/docs/components/data-accelerators/cayenne/index.md @@ -1027,7 +1027,7 @@ Before a source deletion, writes must stop and pending keys must reach zero **wh Consider the following limitations when using Spice Cayenne acceleration: - **Memory Mode Constraints**: `mode: memory` (fully in-RAM, ephemeral) is supported alongside `mode: file`, but it does not persist any data (the dataset reloads from its source on restart), does not support partitioned tables (`partition_by`), and enforces a hard per-table RAM bound instead of spilling to disk — a breach returns an error rather than growing without limit. Use `mode: file` when persistence across restarts is required. DML and [`retention_sql`](../../reference/spicepod/datasets#accelerationretention_sql) are not among the constraints — see [Writes in memory mode](#writes-in-memory-mode). -- **No Re-partitioning**: Cayenne does not re-partition stored data. A `mode: file` dataset whose stored data was written with a different `partition_by` fails to load. To change `partition_by`, stop Spice and delete the dataset's data directory and the Cayenne metastore directory. See [Changing `partition_by` after refresh](../../../features/data-acceleration/partitioning.md#changing-partition_by-after-refresh). +- **No Re-partitioning**: Cayenne does not re-partition stored data. A `mode: file` dataset whose stored data was written with different partition columns fails to load. Changing `partition_by` requires rebuilding the acceleration, which can lose rows that exist only in Cayenne. See [Changing `partition_by` after refresh](../../../features/data-acceleration/partitioning.md#changing-partition_by-after-refresh). - **S3 Express Only**: Standard S3 buckets are not supported for remote storage. Only S3 Express One Zone directory buckets are supported. - **Unsupported Data Types**: `Interval`, `Duration`, `FixedSizeBinary`, `Union`, and `RunEndEncoded` types require `unsupported_type_action` configuration. - **Indexes**: `indexes` builds a read-path [secondary index](#secondary-indexes), not a database index — a `unique` entry does not constrain writes, and registration logs a warning saying so. Deduplication requires only `primary_key`, and the `on_conflict` the warning mentions is not needed. See [Duplicate primary keys in one write](#duplicate-primary-keys-in-one-write). diff --git a/website/docs/features/data-acceleration/partitioning.md b/website/docs/features/data-acceleration/partitioning.md index 3899e7862..c50589b30 100644 --- a/website/docs/features/data-acceleration/partitioning.md +++ b/website/docs/features/data-acceleration/partitioning.md @@ -213,7 +213,7 @@ A `refresh_mode: full` refresh — and any other whole-table overwrite — repla Spice does not re-partition existing acceleration data. An Arrow acceleration lives in memory and is rebuilt from the source on every start, so a new `partition_by` takes effect on restart. A Cayenne acceleration in `mode: file` keeps its data and catalog on disk, so its stored layout must match `partition_by` when the dataset loads. -A file-mode Cayenne dataset fails to load when its stored data was written with a different partitioning than its current `partition_by`. This covers adding `partition_by` to unpartitioned data, removing it from partitioned data, and changing the partition expressions. The runtime logs one error and does not retry. The error has this form, with the values in angle brackets filled in: +A file-mode Cayenne dataset fails to load when its stored data was written with a different partitioning than its current `partition_by`. This covers adding `partition_by` to unpartitioned data, removing it from partitioned data, and changing the partition column names. The runtime logs one error and does not retry. The error has this form, with the values in angle brackets filled in: ```text Failed to load dataset (cayenne): its acceleration data was written , but the dataset now sets . Cayenne does not re-partition existing acceleration data. To keep using the existing data, . To apply the new 'partition_by' instead, stop Spice, delete the acceleration data directory '' and the Cayenne metastore directory '', then start Spice to reload the dataset from the source. Other Cayenne datasets that use this metastore also reload from their sources. See: https://spiceai.org/docs/features/data-acceleration/partitioning @@ -221,8 +221,12 @@ Failed to load dataset (cayenne): its acceleration data was written ]`, and the configured value reads `no 'partition_by'` or `'partition_by: []'`. +The check compares partition column names, not expressions. A plain column entry is named after the column, a named entry after its key, and an anonymous expression `expr0`, `expr1`, and so on. Changing an expression while its name stays the same, such as `bucket(10, id)` to `bucket(20, id)` as `expr0`, is not detected: the dataset loads, and its existing partitions keep the layout of the old expression. Follow the steps below for any change to a partition expression, not only for changes the error reports. + To keep the existing data, restore the `partition_by` the data was written with, or remove `partition_by` if the data was written without it. +Deleting the metastore removes the catalog for every Cayenne dataset that shares it, and those datasets reload from their sources. Rows that exist only in Cayenne cannot be reloaded and are lost: rows in accelerator-only datasets ([`acceleration.write_mode: acceleration`](../../reference/spicepod/datasets.md#accelerationwrite_mode) or `sink` datasets), and write-back rows not yet written to the source. Before deleting it, back up the metastore and data directories, or confirm that every dataset sharing the metastore can be fully reloaded from its source. + To apply the new `partition_by`: 1. Stop Spice. @@ -230,8 +234,6 @@ To apply the new `partition_by`: 3. Update `partition_by`. 4. Start Spice. The dataset reloads from its source. -The metastore holds the catalog for every Cayenne dataset that shares it, so deleting it also makes those datasets reload from their sources. - ## Validation rules A `partition_by` expression is rejected at startup if any of the following hold: From 3857923465c0b8d272fb2e8d4c9ce6a69afc14af Mon Sep 17 00:00:00 2001 From: Emma <98318442+spiceemma@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:45:20 -0700 Subject: [PATCH 4/8] docs(partitioning): plain-column entries compare by column name --- website/docs/features/data-acceleration/partitioning.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/features/data-acceleration/partitioning.md b/website/docs/features/data-acceleration/partitioning.md index c50589b30..328cbf248 100644 --- a/website/docs/features/data-acceleration/partitioning.md +++ b/website/docs/features/data-acceleration/partitioning.md @@ -221,7 +221,7 @@ Failed to load dataset (cayenne): its acceleration data was written ]`, and the configured value reads `no 'partition_by'` or `'partition_by: []'`. -The check compares partition column names, not expressions. A plain column entry is named after the column, a named entry after its key, and an anonymous expression `expr0`, `expr1`, and so on. Changing an expression while its name stays the same, such as `bucket(10, id)` to `bucket(20, id)` as `expr0`, is not detected: the dataset loads, and its existing partitions keep the layout of the old expression. Follow the steps below for any change to a partition expression, not only for changes the error reports. +The check compares partition column names, not expressions. An entry whose expression is a plain column is compared by the column name, even when it is written as a `{ name: column }` mapping. Any other expression is compared by its mapping key, or by `expr0`, `expr1`, and so on when it has none. A change that keeps these names is not detected, such as `bucket(10, id)` to `bucket(20, id)` as `expr0`, or renaming the key of a `{ name: id }` entry: the dataset loads, and its existing partitions keep the layout of the old expression. Follow the steps below for any change to a partition expression, not only for changes the error reports. To keep the existing data, restore the `partition_by` the data was written with, or remove `partition_by` if the data was written without it. From bd6cfd4ed2b48d3f0865653f69859b2127844ab4 Mon Sep 17 00:00:00 2001 From: Emma <98318442+spiceemma@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:55:27 -0700 Subject: [PATCH 5/8] docs(partitioning): back up after shutdown; name the active metastore --- .../docs/features/data-acceleration/partitioning.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/website/docs/features/data-acceleration/partitioning.md b/website/docs/features/data-acceleration/partitioning.md index 328cbf248..916a0a03b 100644 --- a/website/docs/features/data-acceleration/partitioning.md +++ b/website/docs/features/data-acceleration/partitioning.md @@ -225,14 +225,15 @@ The check compares partition column names, not expressions. An entry whose expre To keep the existing data, restore the `partition_by` the data was written with, or remove `partition_by` if the data was written without it. -Deleting the metastore removes the catalog for every Cayenne dataset that shares it, and those datasets reload from their sources. Rows that exist only in Cayenne cannot be reloaded and are lost: rows in accelerator-only datasets ([`acceleration.write_mode: acceleration`](../../reference/spicepod/datasets.md#accelerationwrite_mode) or `sink` datasets), and write-back rows not yet written to the source. Before deleting it, back up the metastore and data directories, or confirm that every dataset sharing the metastore can be fully reloaded from its source. +Deleting the metastore removes the catalog for every Cayenne dataset that shares it, and those datasets reload from their sources. Rows that exist only in Cayenne cannot be reloaded and are lost: rows in accelerator-only datasets ([`acceleration.write_mode: acceleration`](../../reference/spicepod/datasets.md#accelerationwrite_mode) or `sink` datasets), and write-back rows not yet written to the source. To apply the new `partition_by`: -1. Stop Spice. -2. Delete the acceleration data directory and the Cayenne metastore directory that the error names. The metastore directory is set by [`cayenne_metadata_dir`](../../components/data-accelerators/cayenne/index.md#metastore-location). Deleting only the data directory is not enough, because the metastore keeps the dataset's file list and refresh checkpoint. -3. Update `partition_by`. -4. Start Spice. The dataset reloads from its source. +1. Stop Spice with a graceful shutdown (`SIGTERM`), so the metastore's SQLite WAL is checkpointed. +2. Unless every dataset sharing the metastore can be fully reloaded from its source, back up the metastore directory (`cayenne.db` with its `-wal` and `-shm` files) and every data directory it catalogs. Take the backup only after Spice has stopped. +3. Delete the acceleration data directory and the Cayenne metastore directory. Deleting only the data directory is not enough, because the metastore keeps the dataset's file list and refresh checkpoint. The error names the metastore directory resolved for the failing dataset. A process opens one shared catalog, from whichever Cayenne dataset initializes first, so when Cayenne datasets resolve different directories the active catalog can be another one (see [Metastore location](../../components/data-accelerators/cayenne/index.md#metastore-location)). Setting the same explicit `cayenne_metadata_dir` on every Cayenne dataset makes the named directory the active one. +4. Update `partition_by`. +5. Start Spice. The dataset reloads from its source. ## Validation rules From c76bd5f8a09b2b9751e698eef76a94b5ce6fd1ae Mon Sep 17 00:00:00 2001 From: Emma <98318442+spiceemma@users.noreply.github.com> Date: Sun, 11 Oct 2026 02:04:54 -0700 Subject: [PATCH 6/8] docs(partitioning): back up the whole metastore directory; align metastore paths before deleting --- website/docs/features/data-acceleration/partitioning.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/website/docs/features/data-acceleration/partitioning.md b/website/docs/features/data-acceleration/partitioning.md index 916a0a03b..f43b41480 100644 --- a/website/docs/features/data-acceleration/partitioning.md +++ b/website/docs/features/data-acceleration/partitioning.md @@ -230,10 +230,11 @@ Deleting the metastore removes the catalog for every Cayenne dataset that shares To apply the new `partition_by`: 1. Stop Spice with a graceful shutdown (`SIGTERM`), so the metastore's SQLite WAL is checkpointed. -2. Unless every dataset sharing the metastore can be fully reloaded from its source, back up the metastore directory (`cayenne.db` with its `-wal` and `-shm` files) and every data directory it catalogs. Take the backup only after Spice has stopped. -3. Delete the acceleration data directory and the Cayenne metastore directory. Deleting only the data directory is not enough, because the metastore keeps the dataset's file list and refresh checkpoint. The error names the metastore directory resolved for the failing dataset. A process opens one shared catalog, from whichever Cayenne dataset initializes first, so when Cayenne datasets resolve different directories the active catalog can be another one (see [Metastore location](../../components/data-accelerators/cayenne/index.md#metastore-location)). Setting the same explicit `cayenne_metadata_dir` on every Cayenne dataset makes the named directory the active one. -4. Update `partition_by`. -5. Start Spice. The dataset reloads from its source. +2. Unless every dataset sharing the metastore can be fully reloaded from its source, back up the whole metastore directory, including every sidecar file next to `cayenne.db`, and every data directory it catalogs. Take the backup only after Spice has stopped. +3. Make sure every Cayenne dataset resolves the same metastore directory. A process opens one shared catalog, from whichever Cayenne dataset initializes first, while the error names the directory resolved for the failing dataset, so the two can differ when datasets resolve different directories (see [Metastore location](../../components/data-accelerators/cayenne/index.md#metastore-location)). If they can differ, set the same explicit `cayenne_metadata_dir` on every Cayenne dataset first. +4. Delete the acceleration data directory and that metastore directory. Deleting only the data directory is not enough, because the metastore keeps the dataset's file list and refresh checkpoint. +5. Update `partition_by`. +6. Start Spice. The dataset reloads from its source. ## Validation rules From 459887673cf9f899ed67dcaf504b3036807a317c Mon Sep 17 00:00:00 2001 From: Emma <98318442+spiceemma@users.noreply.github.com> Date: Sun, 11 Oct 2026 02:11:36 -0700 Subject: [PATCH 7/8] docs(partitioning): identify the active metastore before deleting --- website/docs/features/data-acceleration/partitioning.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/features/data-acceleration/partitioning.md b/website/docs/features/data-acceleration/partitioning.md index f43b41480..575887f68 100644 --- a/website/docs/features/data-acceleration/partitioning.md +++ b/website/docs/features/data-acceleration/partitioning.md @@ -229,10 +229,10 @@ Deleting the metastore removes the catalog for every Cayenne dataset that shares To apply the new `partition_by`: -1. Stop Spice with a graceful shutdown (`SIGTERM`), so the metastore's SQLite WAL is checkpointed. +1. Stop Spice with a graceful shutdown (`SIGTERM`). 2. Unless every dataset sharing the metastore can be fully reloaded from its source, back up the whole metastore directory, including every sidecar file next to `cayenne.db`, and every data directory it catalogs. Take the backup only after Spice has stopped. -3. Make sure every Cayenne dataset resolves the same metastore directory. A process opens one shared catalog, from whichever Cayenne dataset initializes first, while the error names the directory resolved for the failing dataset, so the two can differ when datasets resolve different directories (see [Metastore location](../../components/data-accelerators/cayenne/index.md#metastore-location)). If they can differ, set the same explicit `cayenne_metadata_dir` on every Cayenne dataset first. -4. Delete the acceleration data directory and that metastore directory. Deleting only the data directory is not enough, because the metastore keeps the dataset's file list and refresh checkpoint. +3. Identify the active metastore directory. A process opens one shared catalog, from whichever Cayenne dataset initializes first, and the error names the directory resolved for the failing dataset, which can be a different one when datasets resolve different directories (see [Metastore location](../../components/data-accelerators/cayenne/index.md#metastore-location)). When every Cayenne dataset resolves the same directory, that directory is the active one. Otherwise, treat every directory a Cayenne dataset resolves as a candidate, include each in the backup, and set the same explicit `cayenne_metadata_dir` on every Cayenne dataset before restarting. +4. Delete the acceleration data directory and the active metastore directory, or every candidate directory. Deleting only the data directory is not enough, because the metastore keeps the dataset's file list and refresh checkpoint. 5. Update `partition_by`. 6. Start Spice. The dataset reloads from its source. From f3075b9c36d0f71545a7733a30c01aeebbf75882 Mon Sep 17 00:00:00 2001 From: Emma <98318442+spiceemma@users.noreply.github.com> Date: Sun, 11 Oct 2026 02:17:55 -0700 Subject: [PATCH 8/8] docs(partitioning): delete every data directory the removed metastore catalogs --- website/docs/features/data-acceleration/partitioning.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/features/data-acceleration/partitioning.md b/website/docs/features/data-acceleration/partitioning.md index 575887f68..dcb4fe2e8 100644 --- a/website/docs/features/data-acceleration/partitioning.md +++ b/website/docs/features/data-acceleration/partitioning.md @@ -232,7 +232,7 @@ To apply the new `partition_by`: 1. Stop Spice with a graceful shutdown (`SIGTERM`). 2. Unless every dataset sharing the metastore can be fully reloaded from its source, back up the whole metastore directory, including every sidecar file next to `cayenne.db`, and every data directory it catalogs. Take the backup only after Spice has stopped. 3. Identify the active metastore directory. A process opens one shared catalog, from whichever Cayenne dataset initializes first, and the error names the directory resolved for the failing dataset, which can be a different one when datasets resolve different directories (see [Metastore location](../../components/data-accelerators/cayenne/index.md#metastore-location)). When every Cayenne dataset resolves the same directory, that directory is the active one. Otherwise, treat every directory a Cayenne dataset resolves as a candidate, include each in the backup, and set the same explicit `cayenne_metadata_dir` on every Cayenne dataset before restarting. -4. Delete the acceleration data directory and the active metastore directory, or every candidate directory. Deleting only the data directory is not enough, because the metastore keeps the dataset's file list and refresh checkpoint. +4. Delete the active metastore directory, or every candidate directory, and every data directory the deleted metastore catalogs, not only the failing dataset's. Deleting only the data directory is not enough, because the metastore keeps the dataset's file list and refresh checkpoint. A data directory left behind is no longer referenced by any catalog, so its files stay on disk while the dataset reloads into new files. 5. Update `partition_by`. 6. Start Spice. The dataset reloads from its source.