You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Avoid quadratic suffix scans when deduplicating query column names #912
sqlite_utils.utils.dedupe_keys reserves all original column names upfront, then searches for each duplicate name by restarting suffix generation at _2. With many repeated names, this repeatedly checks the same suffixes and performs quadratic work.
Add an optimization that avoids restarting the suffix search while preserving the exact existing output names and order, including collisions with preexisting suffixed names. The implementation must continue to accept generators and other iterables. Add correctness coverage for duplicate names, preexisting suffixes, adversarial collisions, and generator inputs; include a benchmark for 500, 2000, and 5000 duplicates without making timing-dependent tests flaky.
Source baseline: 9c66bafa52ae23d40499c74b0db3b635bcddb876, CPython 3.12.15 on macOS arm64. A local synthetic benchmark measured 2,000 identical names at approximately 0.115 seconds and 5,000 at 0.746 seconds (best of three); when original suffixed names are also reserved, the corresponding times were approximately 0.382 and 2.504 seconds. These are helper benchmarks, not end-to-end database timings.
I plan a small patch with correctness coverage and an independently runnable benchmark. AI assistance: MiniMax is analyzing/implementing this contribution; Codex runs the validation and submits the results.
Activity