Cdd 3451 be make sure all 6 permission fields get passed with every charts api call by dandammann · Pull Request #3269 · UKHSA-Internal/data-dashboard-api

dandammann · 2026-06-30T15:33:15Z

Description

This PR includes the following:

See ticket: https://ukhsa.atlassian.net/browse/CDD-3451

Type of change

Please select the options that are relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Tech debt item

Checklist:

I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added tests at the right levels to prove my change is effective
I have added screenshots or screen grabs where appropriate
I have added docstrings in the correct style (google)

…opying the most recent wagtail.fields.StreamField and inserting theme and sub_theme into it)

…e_topic_and_metric_id_by_name() and get_geography_type_id_and_code_by_name() so that any inconsistent combination is denied access

* Update theme functionality to pull available themes from the db via the metrics interface. * CDD-3175: added endpoints for retrieving subtheme/topics/metrics and wired up javascript to call endpoints. * Update the model and the permission_set javascript when handling wildcard selection * Update to add serializer to handle request and response for subthemes and update to subthemes to handle querying db * CDD-3175: updated the JS to add wildcard and empty object options * CDD-3175: Updated the topics and metrics endpoints to retrieve data from the DB * CDD-3175: wired up the logic for selecting geography types * CDD-3175: update permission set for geographies * CDD-3175: updates for limiting the creation of duplicate permission sets * CDD-3175: updates for handling the naming of permission sets * CDD-3085: updated validations and wildcard functionality * CDD-3175: update migrations and add tidy up javascript and validation * CDD-3175: remove console logs from javascript * CDD-3175: Update PermissionSet model * CDD-3175: Update wagtail hooks * CDD-3175: remove print statements and tidy up field_choice_callables * CDD-3175: Update method descriptions * CDD-3175: tidied up the geography serializer * CDD-3175: formatting * Update documentation * CDD-3175: linting * CDD-3175: tests * CDD-3176: Add initial model * CDD-3175: Initial Commit * Create initial permission set * Add conditional sub_theme dropdown * Update migration file and tidy up child_theme.js * pip: (deps): bump python-dotenv from 1.2.1 to 1.2.2 Bumps [python-dotenv](https://github.com/theskumar/python-dotenv) from 1.2.1 to 1.2.2. - [Release notes](https://github.com/theskumar/python-dotenv/releases) - [Changelog](https://github.com/theskumar/python-dotenv/blob/main/CHANGELOG.md) - [Commits](theskumar/python-dotenv@v1.2.1...v1.2.2) --- updated-dependencies: - dependency-name: python-dotenv dependency-version: 1.2.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Testing dummy secret with gitleaks * Testing dummy secret with gitleaks * Added secret scan to the existing action.yaml * Reverted to script installation of gitleaks * Changed job name * Changed ubuntu version * Using official gitleaks action * Updated ubuntu version * Gitleaks arg removed * CDD-3175: populate the Topic dropdown * Update theme functionality to pull available themes from the db via the metrics interface. * Update to add serializer to handle request and response for subthemes and update to subthemes to handle querying db * CDD-3175: updated the JS to add wildcard and empty object options * CDD-3175: Updated the topics and metrics endpoints to retrieve data from the DB * CDD-3085: updated validations and wildcard functionality * WIP: Separate model files and create permission set block * add name back in * working draft * Split models into two files * CDD-3175: removed duplicate functionality * CDD-3175: refactored naming of endpoints * Add unit testing * CDD-3175: update to fix wildcard selection * Simplified version * remove old code * Remove old code * CDD-3175: update for PR comments * CDD-3175: update for PR comments * remove old file * CDD-3175: Update method annotation * CDD-3175: Update method annotation * Update checkboxes * Linting fixes * remove merge issue * linting things * CDD-3175: Update urls for permission set endpoints * CDD-3176: remove duplicated tests * CDD-3172: Update to add the functionality for retrieving user permission sets * CDD-3172: linting * CDD-3175: Update to add ability to get by id and to create initial permission set hierarchy * CDD-3175: add group by functionality * CDD-3172: small refactor of permission_hierarchy and users and topics * Remove testing changes to truncated_dataset * Remove group by geography * refactor permission grouping to group by id rather than name * CDD-3172: linting * CDD-3172: tests * CDD-3172: tests * CDD-3172: tests and refactoring * CDD-3172: Update response format * sonar feedback: update based on sonarqube output * sonar feedback: update based on sonarqube output * Linting * CDD-3171: Update permission set form now it's a page not a snippet * Update topic page to include theme/subtheme/topic fields * WIP: filter getPages based on is_public field * Move auth content underneath CMS * Expose themes/subthemes/topics on topic and metric doc child pages * CDD-3172: move class for blocks * WIP: Add theme/subtheme/topic to pages * linting and permission set url changes * CDD-3172: refactored naming of geography method and updated the tests based on feedback. * CDD-3172: updated test to better name test and updated permission hierarchy error * CDD-2172: Add examples for each of the potential responses for get permissions sets hierarchy requests * CDD-2172: linting * CDD-3147: Update Cognito User for permission sets Add permission sets to request.user object Make cognito user ephemeral for speed - DB access is not needed Move auth header name to settings for flexibility * CDD-3147: Improve logging of JWT * CDD-3147: Update readme for using JWT locally * CDD-3147: Update readme for using JWT locally * CDD-3119 Add a new SimpleMenu model. This is a simplified version of the current Menu model where menus are now just simple links with a title. * CDD-3119 Add panels attribute to SimpleMenu model. * CDD-3119 Beef up the SimpleMenu serializer tests. * CDD-3232 Update chart response styles. Bar and line charts now have borders and grid lines on both axis. * pip dev: (deps-dev): bump pre-commit from 4.5.1 to 4.6.0 Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 4.5.1 to 4.6.0. - [Release notes](https://github.com/pre-commit/pre-commit/releases) - [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md) - [Commits](pre-commit/pre-commit@v4.5.1...v4.6.0) --- updated-dependencies: - dependency-name: pre-commit dependency-version: 4.6.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * pip: (deps): bump idna from 3.11 to 3.12 Bumps [idna](https://github.com/kjd/idna) from 3.11 to 3.12. - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst) - [Commits](kjd/idna@v3.11...v3.12) --- updated-dependencies: - dependency-name: idna dependency-version: '3.12' dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * pip dev: (deps-dev): bump gitpython from 3.1.46 to 3.1.47 Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.46 to 3.1.47. - [Release notes](https://github.com/gitpython-developers/GitPython/releases) - [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES) - [Commits](gitpython-developers/GitPython@3.1.46...3.1.47) --- updated-dependencies: - dependency-name: gitpython dependency-version: 3.1.47 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> * pip: (deps): bump click from 8.3.2 to 8.3.3 Bumps [click](https://github.com/pallets/click) from 8.3.2 to 8.3.3. - [Release notes](https://github.com/pallets/click/releases) - [Changelog](https://github.com/pallets/click/blob/main/CHANGES.rst) - [Commits](pallets/click@8.3.2...8.3.3) --- updated-dependencies: - dependency-name: click dependency-version: 8.3.3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * pip: (deps): bump psycopg2-binary from 2.9.10 to 2.9.12 Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.10 to 2.9.12. - [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS) - [Commits](psycopg/psycopg2@2.9.10...2.9.12) --- updated-dependencies: - dependency-name: psycopg2-binary dependency-version: 2.9.12 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * pip: (deps): bump pydantic from 2.13.2 to 2.13.3 Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.13.2 to 2.13.3. - [Release notes](https://github.com/pydantic/pydantic/releases) - [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md) - [Commits](pydantic/pydantic@v2.13.2...v2.13.3) --- updated-dependencies: - dependency-name: pydantic dependency-version: 2.13.3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * build: remove simplejson dependency Saw a dependabot update for this package and in investigating if it was safe to update realised we're not actually using it... * CDD-3313: Add topic page link to headline metrics card (#3151) * pip: (deps): bump idna from 3.12 to 3.13 Bumps [idna](https://github.com/kjd/idna) from 3.12 to 3.13. - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst) - [Commits](kjd/idna@v3.12...v3.13) --- updated-dependencies: - dependency-name: idna dependency-version: '3.13' dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * topics: add HIV topic * CDD-3087: new CMS page for logged-out functionality (#3163) * pip: (deps): bump filelock from 3.28.0 to 3.29.0 Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.28.0 to 3.29.0. - [Release notes](https://github.com/tox-dev/py-filelock/releases) - [Changelog](https://github.com/tox-dev/filelock/blob/main/docs/changelog.rst) - [Commits](tox-dev/filelock@3.28.0...3.29.0) --- updated-dependencies: - dependency-name: filelock dependency-version: 3.29.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * WIP: pseudo code / note form of solution * WIP: filter pages on permission sets * Permission check updates and form handling * WIP: Fix comparison function * Finish getPages endpoint * CDD-3172: Remove permission_sets from CMS API * CDD-3172: Update docstring * remove redundant code * fixes for existing unit tests * Update imports * add endpoint back in for testing * fix import * New tests, and fixes and updates to existing tests * Naming fixes * test coverage * CDD-3171: Tweaks * CDD-3171: Add display name to permission sets * remove log file * Fix js file * Linting * Update migration * refactor for sonarqube checks * Fix constraints on permission sets * update unit tests * fix allowed_pages overwrite * Fix unit test * Fix test * linting * CDD-3171: Move permission_set.js insert to Media class * CDD-3171: Add ignores for importlint This could do with some more time spent on it to simplify the imports being ignored * Update architectural constraints * linting * Update wildcard value in viewsets * Update test * remove comment * Update import * fix test name * linting * combine imports * refactor for simplicity * linting * CDD-3173: prototype authorization curl call on /api/downloads/v2 * CDD-3173: get rid of check_permissions_by_name() and make /api/downloads/v2 use check_permissions() instead * CDD-3173: let cms/dashboard/viewsets.py from CDD-3171 use my fully equivalent check_permissions() function instead * CDD-3173: add debugging code to user_manager.py to be able to test this JIRA ticket in isolation * CDD-3173: evaluate metric- and geography-related permissions separately * CDD-3173: lint * CDD-3173: lint * CDD-3173: convert permission function arguments into named arguments (safer) * CDD-3173: move things around for architectural constraints * CDD-3173: reduce noisy comments * CDD-3173: remove rbac_permissions parameter (easy to do, cos doesn't cause other failures) * Pass permission set array to check permissions * CDD-3174: re-added noisy comment * CDD-3174: formalize comment * CDD-3174: revert rbac_permissions warning fix * CDD-3174: revert every logger.info("Entered function ...) * Add logs for permission sets * CDD-3174: revert SQL printing * CDD-3174: revert permission set debugging * CDD-3174: add type hints to permission functions & vars * CDD-3174: make permissions disallow empty "" requests * Update test mock and linting * CDD-3174: centralize WILDCARD_ID_VALUE * CDD-3174: use MetricsAPIInterface to access data mappers from common folder * CDD-3174: remove duplicate check_metric_related_permissions() call * CDD-3174: add BaseRequestParams() class that all other request classes inherit from * Update log * Update log line * CDD-3174: add named arguments to check_permissions() * CDD-3174: fix viewsets.py error since merge * CDD-3174: simplify convoluted WHERE clauses in SQL * CDD-3174: add _get_id_string_or_none() to normalize strings to prevent unintentional "None" == "None" string comparisons * CDD-3174: separate check_page_permissions() and check_chart_permissions() which allows all arguments to be mandatory (makes behaviour more predictable) * CDD-3174: add tests for permission filtering functionality * CDD-3174: move TestCheckPagePermissions() from test_viewsets.py to test_permissions.py * CDD-3174: log user permissions for every API call that comes with a JWT * CDD-3174: make comment more prominent (so it can't be missed) * CDD-3174: simplify permission logging * CDD-3174: avoid illogical ChartRequestParams(ChartRequestParams) class name * CDD-3174: simplify to "if permission_sets and check_chart_permissions_by_name()" * CDD-3174: remove 2 redundant check_chart_permissions() tests * CDD-3174: add docstrings to functions * CDD-3174: remove duplicate sanity check * CDD-3174: build permission set from the non-public instead of the public recordset * CDD-3174: fix type hints * CDD-3174: call log_user_permission_summary() not on every API call anymore * CDD-3174: fix lint * CDD-3174: protect against geographies with the same name across geography types * CDD-3174: fix geography code bug * CDD-3174: remove old RBAC @require_authorisation decorator from DownloadsView.post --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Luke Towell <luke.towell@burendo.com> Co-authored-by: Luke Towell <luke.towell@ukhsa.gov.uk> Co-authored-by: Kathryn Dale <kathryn.dale@burendo.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: abdihakim92x1 <AbdiHakim.Mohamed@ukhsa.gov.uk> Co-authored-by: Matt Reynolds <18287679+mattjreynolds@users.noreply.github.com> Co-authored-by: David Logie <david.logie@ukha.gov.uk> Co-authored-by: Josh Humphries <josh.humphries@ukhsa.gov.uk> Co-authored-by: Taiwo Kareem <13158672+tushortz@users.noreply.github.com> Co-authored-by: Aidan Skinner <aidan@skinner.me.uk> Co-authored-by: itsthatianguy <ian.j.rufus@gmail.com> Co-authored-by: kathryn-dale <161315272+kathryn-dale@users.noreply.github.com> Co-authored-by: sahmed06 <58435820+sahmed06@users.noreply.github.com>

…fields & the new strict permission name-to-id requirements

…API endpoints

…s-get-passed-with-every-charts-API-call

jrdh

Changes look good. I've included a couple of non-blocking comments about naming and docs but the change itself looks solid to me.

jrdh · 2026-08-14T10:29:31Z

+def backfill_pages_with_theme_and_subtheme(apps, model_section, model_name):
+    topic_map = _build_topic_map(apps)
+
+    LandingPage = apps.get_model(model_section, model_name)


This var is confusingly named as the function is called with

backfill_pages_with_theme_and_subtheme(apps, "home", "LandingPage")

backfill_pages_with_theme_and_subtheme(apps, "topic", "TopicPage")

backfill_pages_with_theme_and_subtheme(apps, "topics_list", "TopicsListPage")

so sometimes it's a LandingPage but other times it's a TopicPage or a TopicsListPage.

Non-blocking issue, just confusing to read through.

Ah good catch - missed during copy and paste 🙈
app_model seemed to be an appropriate name

jrdh · 2026-08-14T13:21:27Z

@@ -0,0 +1,57 @@
+from wagtail.fields import StreamValue


Thinking forward to someone looking at this in 6 months time with no awareness of what we're doing now, I think it'd be good to have a block comment at the top of this file explaining what these functions were used for and why.

Added some info - let me know that's clear or if more/less info is needed

…s-get-passed-with-every-charts-API-call

sonarqubecloud · 2026-09-17T17:14:49Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
2.1% Duplication on New Code

See analysis details on SonarQube Cloud

dandammann requested a review from a team as a code owner June 30, 2026 15:33

itsthatianguy mentioned this pull request Jul 6, 2026

Cherry pick/cdd 3451 be make sure all 6 permission fields get passed with every charts api call #3277

Closed

9 tasks

dandammann and others added 9 commits July 7, 2026 09:47

CDD-3451: add theme and sub_theme DB storage to a CMS page body (by c…

de06239

…opying the most recent wagtail.fields.StreamField and inserting theme and sub_theme into it)

CDD-3451: add theme and sub_theme drop-downs to the CMS page body

47a739f

CDD-3451: refactor get_id_by_name() functions into get_theme_sub_them…

8f9564e

…e_topic_and_metric_id_by_name() and get_geography_type_id_and_code_by_name() so that any inconsistent combination is denied access

CDD-3451: amend tests to match the new mandatory theme and sub_theme …

e8e6960

…fields & the new strict permission name-to-id requirements

CDD-3451: get test coverage up

6998429

CDD-3451: remove unused import

bc024fa

CDD-3451: get test coverage up to 100%

dc561f4

CDD-3451: make geography and geography_type mandatory across those 8 …

6b7023c

…API endpoints

itsthatianguy force-pushed the CDD-3451-BE-Make-sure-all-6-permission-fields-get-passed-with-every-charts-API-call branch from f0f44de to 6b7023c Compare July 7, 2026 09:22

MeganB-2 reviewed Jul 9, 2026

View reviewed changes

Comment thread metrics/data/managers/core_models/topic.py Outdated

MeganB-2 reviewed Jul 9, 2026

View reviewed changes

Comment thread metrics/api/serializers/plots.py

MeganB-2 reviewed Jul 9, 2026

View reviewed changes

Comment thread metrics/data/managers/core_models/geography.py Outdated

PR comments

2ec2403

MeganB-2 previously approved these changes Jul 15, 2026

View reviewed changes

sahmed06 and others added 4 commits July 20, 2026 11:55

Merge branch 'main' into CDD-3451-BE-Make-sure-all-6-permission-field…

09bf75a

…s-get-passed-with-every-charts-API-call

Merge branch 'main' into CDD-3451-BE-Make-sure-all-6-permission-field…

bcad387

…s-get-passed-with-every-charts-API-call

Updating migration order following merge with main

64690d9

Fixing tests

60112db

dandammann dismissed MeganB-2’s stale review via 60112db August 5, 2026 09:08

itsthatianguy added 4 commits August 5, 2026 15:28

Migrations to backfill theme and sub_theme, updates to bootstrap data

e613f0d

Lint fix

c81abd5

Merge branch 'main' into CDD-3451-BE-Make-sure-all-6-permission-field…

0c6ffc0

…s-get-passed-with-every-charts-API-call

Merge branch 'main' into CDD-3451-BE-Make-sure-all-6-permission-field…

a516548

…s-get-passed-with-every-charts-API-call

jrdh previously approved these changes Aug 14, 2026

View reviewed changes

itsthatianguy added 3 commits August 17, 2026 09:09

Merge branch 'main' into CDD-3451-BE-Make-sure-all-6-permission-field…

d6623e8

…s-get-passed-with-every-charts-API-call

Fixing up migrations

21c0753

Addressing PR comments

c409c01

itsthatianguy dismissed jrdh’s stale review via c409c01 August 17, 2026 08:32

Linting

3826fdc

itsthatianguy requested a review from jrdh August 17, 2026 10:41

Merge branch 'main' into CDD-3451-BE-Make-sure-all-6-permission-field…

bbf6ada

…s-get-passed-with-every-charts-API-call

mattjreynolds previously approved these changes Sep 2, 2026

View reviewed changes

sahmed06 and others added 3 commits September 3, 2026 16:08

Merge branch 'main' into CDD-3451-BE-Make-sure-all-6-permission-field…

57692d2

…s-get-passed-with-every-charts-API-call

Updating backfill script to handle revisions

e9f5b32

Formatting

cee3f7d

itsthatianguy dismissed mattjreynolds’s stale review via cee3f7d September 4, 2026 14:12

Merge branch 'main' into CDD-3451-BE-Make-sure-all-6-permission-field…

d143281

…s-get-passed-with-every-charts-API-call

mattjreynolds approved these changes Sep 7, 2026

View reviewed changes

sahmed06 and others added 2 commits September 10, 2026 14:06

Merge branch 'main' into CDD-3451-BE-Make-sure-all-6-permission-field…

b0442fd

…s-get-passed-with-every-charts-API-call

Merge branch 'main' into CDD-3451-BE-Make-sure-all-6-permission-field…

37a71eb

…s-get-passed-with-every-charts-API-call

Navigation Menu

Cdd 3451 be make sure all 6 permission fields get passed with every charts api call - #3269

Cdd 3451 be make sure all 6 permission fields get passed with every charts api call#3269
dandammann wants to merge 29 commits into
mainfrom
CDD-3451-BE-Make-sure-all-6-permission-fields-get-passed-with-every-charts-API-call

dandammann commented Jun 30, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jrdh left a comment

Uh oh!

jrdh Aug 14, 2026

Uh oh!

itsthatianguy Aug 17, 2026

Uh oh!

jrdh Aug 14, 2026

Uh oh!

itsthatianguy Aug 17, 2026

Uh oh!

sonarqubecloud Bot commented Sep 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

dandammann commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Checklist:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jrdh left a comment

Choose a reason for hiding this comment

Uh oh!

jrdh Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

itsthatianguy Aug 17, 2026

Choose a reason for hiding this comment

Uh oh!

jrdh Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

itsthatianguy Aug 17, 2026

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud Bot commented Sep 17, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

dandammann commented Jun 30, 2026 •

edited

Loading