Skip to content

[SYNPY-1869] Add SearchIndex entity + search-management APIs; rename SchemaOrganization to Organization#1422

Draft
BryanFauble wants to merge 20 commits into
developfrom
synpy-1869-search-index
Draft

[SYNPY-1869] Add SearchIndex entity + search-management APIs; rename SchemaOrganization to Organization#1422
BryanFauble wants to merge 20 commits into
developfrom
synpy-1869-search-index

Conversation

@BryanFauble

@BryanFauble BryanFauble commented Jul 13, 2026

Copy link
Copy Markdown
Member

Problem:

Two related gaps in the OOP model layer:

  1. SearchIndex / search-management surface was incomplete. The SearchManagementController in Synapse-Repository-Services exposes a family of resources — SearchIndex (an OpenSearch index built from a defining SQL query), plus the configuration resources that control how indexes are built: TextAnalyzer, ColumnAnalyzerOverride, SynonymSet, SearchConfiguration, and SearchConfigBinding — as well as index query and autocomplete endpoints. The Python client only had a partial, not-fully-correct starting point for these.

  2. SchemaOrganization naming was inconsistent. The model that holds JSON-schema resources was named SchemaOrganization, but conceptually it's just an "Organization" that holds resources. The name should read consistently.

Solution:

Search management (new):

  • Added the SearchIndex entity model (models/search_index.py) following the async-first @async_to_sync pattern — store/get/delete/autocomplete, columns derived read-only from defining_sql, registered concrete type org.sagebionetworks.repo.model.search.table.SearchIndex.
  • Added the search-management resource models (models/search_management.py): TextAnalyzer, ColumnAnalyzerOverride, SynonymSet, SearchConfiguration, SearchConfigBinding, plus query/autocomplete request+hit types.
  • Added the REST API layer (api/search_services.py) and wired it into api/__init__.py.
  • Registered the new entity in concrete_types.py, entity_factory.py, mixins/asynchronous_job.py, and marked SearchIndex read-only-schema in mixins/table_components.py.

Rename (hard, no back-compat alias):

  • SchemaOrganizationOrganization (class + SchemaOrganizationProtocolOrganizationProtocol), module helper list_json_schema_organizationslist_organizations.
  • Renamed files via git mv to preserve history: models/schema_organization.pymodels/organization.py, the unit/integration test modules, and docs/reference/experimental/{async,sync}/schema_organization.mdorganization.md; updated mkdocs.yml nav.
  • Updated all references across models/__init__.py, operations/{store,delete}_operations.py, mixins, curator extension, and the deprecated services/json_schema.py docstring pointers.
  • Note: the legacy public JsonSchemaOrganization class in services/json_schema.py is intentionally left unchanged — it is a separate deprecated API, not the model being renamed.

Testing:

  • New async unit tests for the search-management models and Organization; existing store/delete/factory routing unit tests updated for the rename.
  • Focused unit run green: 142 passed across json_schema services, organization, search-management, and store/delete operations.
  • Integration tests added for SearchIndex (create/retrieve, match-all query, autocomplete, delete) — kept minimal to limit CI/Synapse-server load; index-creation tests skip gracefully on a 403 where creation is restricted.
  • ruff and black clean on all changed files (the 2 remaining F841 flags in the renamed org test module and the json_schema tutorial are pre-existing debt, left untouched per surgical-change scope).

BryanFauble and others added 20 commits July 13, 2026 23:36
…SchemaOrganization to Organization

Build the OOP method surface for search-management resources (TextAnalyzer,
ColumnAnalyzerOverride, SynonymSet, SearchConfiguration, SearchConfigBinding)
and the SearchIndex entity with autocomplete/query support, following the
async-first @async_to_sync pattern with protocol classes and unit tests.

Hard-rename the SchemaOrganization model to Organization (class, protocol,
module helper list_json_schema_organizations -> list_organizations, files,
docs, and mkdocs nav) so it consistently reads as an "Organization" that
holds resources. No backwards-compatible alias.
…ne_keys

- Return SearchIndex children from sync_from_synapse/walk/get_children by
  adding searchindex dispatch to StorableContainer and searchindexes fields
  on Project and Folder
- Include searchindex in access_control ENTITY_TYPE_MAPPING so ACL/permission
  cascades reach SearchIndex children
- Replace `{k: v ... if v is not None}` comprehensions with delete_none_keys()
  in search_services list functions
- Fix async docstring examples in search_index to use async form
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants