viz: --geojson auto for NAME-keyed region columns (city-name works via geocode; county-name needs a new mechanism)

[jqnatividad]

Amended. As originally filed, this issue claimed forward_geocode_regions could resolve a county-NAME column ("Allegheny County") to a county FIPS. That is wrong, and the correction changes what this issue is actually asking for. The original mechanism is sound — but for a city-name column, not a county-name one. Details below.

Summary

--geojson auto (#4397, PR #4418) resolves boundaries from region codes (county FIPS, ZCTA, tract, place GEOID). Extend it to region columns that hold names rather than codes, by resolving them to FIPS through the Geonames engine qsv geocode already ships.

Why this is a real gap

REGION_CODE_LEAVES (src/cmd/viz.rs) already includes "county" as distinct from "county_fips"/"fips", and match_region_code already case-folds alphabetic region codes — so name-keyed region columns are a supported shape today whenever the user supplies their own GeoJSON. They are exactly the columns auto cannot serve.

What the geocode engine can and cannot do (the correction)

forward_geocode_regions (src/cmd/geocode.rs) calls:

engine.suggest::<String>(name, 1, None, None)

That searches the cities index — the Geonames citiesNNNNN gazetteer. It matches populated places.

admin2Codes.txt (DEFAULT_ADMIN2_CODES_URL) is loaded only to decorate a matched city record with cityrecord.admin2_division. Counties are not independently searchable, and the --admin1 option is a filter applied to city results, not a name search over administrative divisions.

So:

column holds	resolvable today?	how
a city name (Pittsburgh)	yes	suggest → city record → admin2_division → us_fips_from_codes → 42003
a county name (Allegheny County)	no	suggest fuzzy-matches it against city names; it does not reach a county record

The original issue conflated these. A literal "Allegheny County" value does not resolve through this path.

Two separable pieces of work

A. City-name columns → county FIPS (the mechanism that works)

suggest → matched city → its admin2_division → county FIPS → the existing county boundary path from #4397. This is a lookup, not a spatial approximation, so the nearest-populated-place caveat below does not apply — the answer is "what county is this city in", which Geonames states directly.

Needs GeoRegion to expose the FIPS pair: it currently carries admin1_code / us_state_code but not admin2 or the FIPS codes, even though us_fips_from_codes(admin1_code, admin2_code) already exists as a pure function and us_fips_strings shows the admin2 access pattern. GeoRegion's own doc comment says the struct is "a complete region-identifier surface … populated for reuse by future callers" — this is that caller. ~5 lines.

Geonames index resolution is the binding constraint here. The default is cities15000 (~26k cities worldwide, ~800 US), which omits most US small towns, so a city-name column will under-resolve badly. cities1000 (~140k) or cities500 (~200k) largely fixes it.

But do not change the global default to fix this: it is a cost imposed on every geocode user (13mb → 56mb index, ~26k → ~200k records, slower lookups, more memory) and the help text explicitly warns that "search results will be different". The prebuilt index published from the qsv repo is English-only cities15000, so changing the default also means shipping a much larger prebuilt or pushing users to build locally.

Instead, detect and report — the same pattern #4397 uses for a vintage mismatch, which turns a dead end into an instruction:

--geojson auto: resolved 40 of 300 place names. The default cities15000 Geonames index
omits places under 15,000 population; rebuild with `qsv geocode index-update --cities-url 1000`.

B. County-name columns (what this issue originally asked for)

Needs a mechanism that does not exist yet. Options, roughly in order of appeal:

1. Make admin2 records searchable — build a lookup over admin2Codes.txt (name → US.PA.003), which is already downloaded. Cheap, offline, and exactly keyed to the question. Ambiguity (Washington County exists ~30 times) is resolved by a state column, or reported.
2. Match TIGERweb's NAME/BASENAME — the county layer carries both (Beaver County / Beaver). Tempting, but bare names are ambiguous across states, so it needs a state from somewhere regardless.
3. Require the user to supply a state column alongside, and disambiguate with it.

Whichever is chosen, the honesty check is the same as the code path: resolution rate over distinct names, with ambiguous bare names reported rather than silently resolved to an arbitrary state.

Important: do NOT extend either of these to lat/lon reverse geocoding

Investigated and rejected during #4397. reverse_geocode_regions returns the nearest populated place's county, not the containing county. A point in rural Fayette County PA whose nearest city record sits in Westmoreland yields the wrong county — and nothing downstream detects it, because every derived FIPS is a real county, so any resolution-rate check reads 100%.

A denser index does not rescue this. It makes the error rarer without making it detectable, which is a worse failure mode than the common one: it survives testing, so you stop looking for it. The correct mechanism for bare lat/lon boundaries is point-in-polygon against fetched polygons, which #4397 also defers.

Reverse geocoding remains safe for deriving a state set (a nearest-city match essentially never crosses a state line, and when it does the set merely widens, which is harmless).

Acceptance

• A city-name column resolves to county boundaries with nothing supplied but the CSV, and an under-resolving index is reported with the actionable rebuild hint rather than silently producing a sparse map.
• A county-name column resolves via whichever mechanism is chosen in (B), with ambiguous bare names reported, not silently resolved.
• Resolution rate over distinct names is the honesty check, given the same threshold treatment as the code path in viz: --geojson auto — fetch US standard-jurisdiction boundaries (county/ZCTA/tract/place) from Census TIGERweb automatically, vintage-aware + state-scoped + cached (companion to #4394/#4395) #4397.

[jqnatividad]

Amended above. The correction, briefly: forward_geocode_regions calls engine.suggest() against the cities index, and admin2Codes.txt only decorates a matched city record with admin2_division — counties are not independently searchable, and --admin1 is a filter on city results rather than a name search. So the mechanism as originally described resolves a city-name column (city → its admin2 → county FIPS), not a county-name one; "Allegheny County" fuzzy-matches against city names and does not reach a county record.

Split into (A) city-name columns, which the existing engine handles once GeoRegion exposes the FIPS pair, and (B) county-name columns, which need a new lookup — most likely an index over the already-downloaded admin2Codes.txt.

Also recorded there: Geonames index resolution is the binding constraint for (A) — the default cities15000 omits most US small towns — but the fix is to detect and report an under-resolving index, not to raise the global default, which would cost every geocode user a 13mb → 56mb index and different search results.

[jqnatividad]

The premise of this issue has shifted. Findings from #4427 / #4430, which came out of measuring whether a denser Geonames index would make Part A viable.

The binding constraint was never index resolution

This issue records the index floor as the constraint on Part A ("the default cities15000 omits most US small towns"). That is true of coverage, but it was not what made city-name resolution wrong. forward_geocode_regions passed no country filter, so a US city name was matched against every populated place on Earth:

input	resolved to
Montgomery	Sahiwal District, Pakistan
Birmingham	United Kingdom

Identical under cities15000 and a locally built cities1000. A denser index does not fix this and plausibly worsens it — ~114k more foreign places compete for the same bare names.

That is fixed in #4430: per-row --geocode-country / --geocode-admin1 hint columns, an implicit us for usa-states, and strict semantics (a hint that excludes every candidate reports the miss rather than falling back to the top hit, which is what geocode suggest --admin1 does by design).

What that leaves for Part A

Part A is now a coverage question, not a correctness one, and two of its stated details need correcting:

• GeoRegion exposing the FIPS pair is necessary but nowhere near sufficient. The "~5 lines" estimate assumed a foothold that isn't there: viz.rs's existing forward_geocode_regions call serves the plotly-geo path (LocationMode::Iso3/UsaStates), not the --geojson custom-boundary path that --geojson auto feeds. match_region_code joins the column's own values to feature ids, so a city-name column needs a synthesized per-row FIPS that also scopes the auto-fetch. That plumbing is the actual work.
• SUGGEST_ADMIN1_LIMIT = 10 is too small once a hint is involved, and this is measured, not theoretical. A prefix match scores exactly 1.0 and ties break by population, so a small town in the hinted state sits below every same-named larger city: Springfield + US.CO returned Springfield Missouri under cities1000. fix(viz,geocode): constrain --geocode with per-row country/admin1 hints (#4427) #4430 added a separate, much larger limit for the hinted path (cheap — the engine sorts and dedupes the full candidate set before applying the limit) and left geocode suggest's constant alone.

The under-resolution report this issue proposes for Part A remains the right shape, and it now has a cheap remedy to point at — see below.

Part B is unchanged, and the blocker is confirmed

The amendment above is correct: admin2_division is only ever reachable as decoration on a matched city record (geocode.rs:2650, 3820, 3864); counties are not independently searchable. Additionally, admin2Codes.txt is not retained — admin2_codes_url is handed to geosuggest's IndexUpdater, which folds it into the rkyv index, and the common path downloads a prebuilt index, so that file is never fetched locally at all. So option 1 owns a fetch+cache path, not just a parse.

One point in Part B's favor that is easy to miss: us_fips_from_codes already takes (admin1_code, admin2_code), so a state column disambiguates Washington County through machinery that exists. The genuinely new part is only name → US.PA.003.

The index remedy, measured

Numbers for the "detect and report, don't change the global default" decision, which still looks right:

• shipped prebuilt (cities15000): 22.6 MiB rkyv / 9.06 MiB .sz
• locally built cities1000: 82.7 MiB — 3.66×

Two things make a published cities1000 prebuilt cheap, which would turn Part A's under-resolution report into a one-command fix (qsv geocode index-load 1000) instead of a ~200 MB Geonames rebuild:

• the download URL is already parameterized by resolution — …/{VERSION}/qsv-{VERSION}-geocode-index.rkyv.cities + {shortcut}.sz — so …rkyv.cities1000.sz would fetch correctly today;
• the only code blocking it is the equality guard at geocode.rs:2197 (shortcut != DEFAULT_GEONAMES_CITIES_INDEX).

Stability was better than feared, at least at the easy end: across 332 major US cities with a country filter, only 4 rows (1.2%) resolved differently between the two indexes, and the visible differences were gains (Alaska boroughs that previously resolved to nothing). US coverage per population floor is still unmeasured.