Skip to content

Preserve one-shot column iterables in full-text search queries - #943

Open
pengpengyi92 wants to merge 1 commit into
simonw:mainfrom
pengpengyi92:fix/fts-search-column-iterables-20261114
Open

pengpengyi92 wants to merge 1 commit into
simonw:mainfrom
pengpengyi92:fix/fts-search-column-iterables-20261114

Conversation

@pengpengyi92

@pengpengyi92 pengpengyi92 commented Nov 13, 2026 •

Copy link
Copy Markdown

Fixes #942.

search_sql iterates columns twice, for the CTE and outer SELECT projections. A generator is exhausted after the first pass, producing an empty outer SELECT list. Materialize the iterable once before the existing projection logic; None, lists and tuples keep their behavior, and empty iterators now follow the existing empty-list default.

The new tests execute real FTS4/FTS5 queries through both search_sql and search. They cover empty/single/multiple column selections, list/tuple/iterator/generator inputs, output column order, filtering/pagination, rank inclusion, escaped column names and the original alias collision case.

Validation on Python 3.12.13 / SQLite 3.53.1:

  • Before the source change: 28 new cases fail, 26 controls pass.
  • After the change: all 54 new cases pass; 158 tests pass across the new module, test_fts.py and test_query.py, in both default and --sqlite-autocommit modes.
  • Black and flake8 pass on both changed files; mypy passes on all 10 source files.
python -m pytest -q tests/test_fts_search_iterables.py tests/test_fts.py tests/test_query.py
python -m pytest -q tests/test_fts_search_iterables.py tests/test_fts.py tests/test_query.py --sqlite-autocommit

Only the search projection is changed, not index creation, query matching, SQL escaping or transaction handling. A consumed generator is not made reusable across separate calls. The complete optional-extension/OS matrix was not run.

AI assistance: prepared and checked with OpenAI Codex; no independent human review or hosted CI success is claimed.


📚 Documentation preview 📚: https://sqlite-utils--943.org.readthedocs.build/en/943/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

search_sql() exhausts iterator columns before building the outer projection

1 participant