Skip to content

Clean up orphaned data in destructively_trim_db for single-tenant exports - #634

Open
bbliem wants to merge 1 commit into
mainfrom
feature/trim-db-single-tenant-export-cleanup
Open

bbliem wants to merge 1 commit into
mainfrom
feature/trim-db-single-tenant-export-cleanup

Conversation

@bbliem

@bbliem bbliem commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Trimming the DB to a single plan and running dumpdata left behind data that is not scoped to the retained plan, leaking other clients' content into the export. Add three cleanups to the trim command:

  • delete_orphaned_translation_data() (always): sweep wagtail_localize rows whose translated object no longer exists. The trim deletes inside mute_signals(post_delete), which suppresses wagtail_localize's own cleanup_translation_on_delete handler, so this mirrors it after the fact.
  • delete_orphaned_plan_pages() (always): delete PlanRootPage trees (any locale) and their Sites not belonging to a retained plan. Plan.delete() uses a bulk PageQuerySet.delete() that leaves a deleted plan's translated locale-tree pages (and Site) behind as live orphans.
  • prune_unused_common_indicators() (behind --prune-shared-reference-data): delete common indicators not linked to a retained plan or a surviving indicator, then empty frameworks.

Extract the deletion-summary printing into helpers to keep handle() under the complexity limits.

While this removes some data that should not be in single-tenant exports, it is not sufficient. When I tested this with a churned client's data, there were still several records included that belonged to some other tenant for one reason or another. For cases like this, where we hand off data to customers, we need to be extra sure that we do not include data from other customers. A destructively_trim_db -> dumpdata approach seems to risky to me. Therefore, for data hand-off cases like this, I decided to go with a constructive approach that reuses some code from the copying app in order to only export data that is in fact reachable from the tenant's plan. There is a different PR for this: #632


✅ Pre-Merge Checklist

Type of Change

  • Set the PR's label to match the nature of this change

Testing

  • Built Unit tests (unit tests added/updated)
  • Manually tested locally (functionality verified)

@kausal-code-coverage

kausal-code-coverage Bot commented Jul 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #634   +/-   ##
=======================================
  Coverage   63.22%   63.22%           
=======================================
  Files         387      387           
  Lines       41608    41608           
  Branches     5371     5371           
=======================================
  Hits        26307    26307           
  Misses      13615    13615           
  Partials     1686     1686           
Flag Coverage Δ
e2e-tests 8.53% <ø> (ø)
unittests 60.56% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted file tree graph

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0c6707f2b1

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread actions/management/commands/destructively_trim_db.py Outdated
Comment thread actions/management/commands/destructively_trim_db.py Outdated
…orts

Trimming the DB to a single plan and running dumpdata left behind data
that is not scoped to the retained plan, leaking other clients' content
into the export. Add three cleanups to the trim command:

- delete_orphaned_translation_data() (always): sweep wagtail_localize
  rows whose translated object no longer exists. The trim deletes inside
  mute_signals(post_delete), which suppresses wagtail_localize's own
  cleanup_translation_on_delete handler, so this mirrors it after the fact.

- delete_orphaned_plan_pages() (always): delete PlanRootPage trees (any
  locale) and their Sites not belonging to a retained plan. Plan.delete()
  uses a bulk PageQuerySet.delete() that leaves a deleted plan's translated
  locale-tree pages (and Site) behind as live orphans.

- prune_unused_common_indicators() (behind --prune-shared-reference-data):
  delete common indicators not linked to a retained plan or a surviving
  indicator, then empty frameworks.

Extract the deletion-summary printing into helpers to keep handle() under
the complexity limits.
@bbliem
bbliem force-pushed the feature/trim-db-single-tenant-export-cleanup branch from 0c6707f to 7ebc4bf Compare July 3, 2026 12:16
@Bhavatu

Bhavatu commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Some merge conflicts to be resolved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants