Updated Why qsv (markdown)
ec9f84c
Update Benchmarks page for the 22.0.1 run
Regenerated from benchmarks/Benchmarks.wiki.md (qsv 22.0.1, aarch64-apple-darwin).
- 259 -> 277 benchmarks
- "dashboard" -> "Data Schematic" throughout, matching the 22.0.1 terminology
- two flagship deep-dives (stats, frequency) -> four (adds validate and
moarstats, which landed in d95e09c9c and ab9be8860 but never reached this page)
- adds the callout linking the classic tabular benchmarks at qsv.dathere.com
The interactive Data Schematic itself redeploys to GitHub Pages from
dathere/qsv master (73fd16a9d).
1cb3fcf
Updated Visualization (markdown)
8aecb90
docs: catch the wiki up for the 22.0.1 "Data Schematic" release
Wiki was last updated Jul 23, mid-cycle; 22.0.1 shipped Aug 8.
- Home: new 22.0.1 release blurb (was still 21.1.0 "Leaner & Faster")
- Visualization: introduce the Data Schematic as the ARTIFACT `viz smart`
produces (command stays "auto-dashboard", matching viz.rs/FEATURES.md);
new data-viewer-drawer + cross-linking + localized-schematics sections;
dictionary-declared pipeline funnel panel; funnel chart example;
viz shipping/big-endian caveat
- Transform-and-Reshape: new `fixedwidth` section (help declares this page
as its wiki home)
- Indexing-Compression-Diff: new `clean` section (Command-Reference linked
a page with no such section)
- Breaking changes documented: minijinja 2.23 True/False/None rendering,
cached 2->3 sled->redb (incl. the TTL=0 inversion), describegpt prompt
file 8.0.0 -> 9.0.0
- describegpt: document local whatlang content-language auto-detection and
the dual-typed --language flag
- Binary-Variants: per-variant command counts verified against 22.0.1
(qsv 78 / qsvpy 79 / qsvmcp 69 / qsvlite 51 / qsvdp 44)
- MCP-Server: replace "~65 tools" with 69 commands / 55 skill-based tools
Fixes found en route:
- `describegpt --lang` does not exist (rejected by docopt); corrected to
`--language` in 2 places
- 5 broken anchors into "(cross-reference)" headings in Command-Reference
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6e01caf
docs: catch the wiki up on changes since Jul 12
Audited the wiki against qsv master (~50 non-dependency commits since the
last content update). Most of that window was viz work.
Added:
- Visualization: `splom` + `parcats` subcommands (decision table + examples);
a new "Animation, range & annotation" section for `--slider`,
`--slider-speed`, `--slider-cumulative`, `--rangeslider`, `--y-range` and
`--annotation`; `viz smart` sections for skew-aware encoding (Gini/Lorenz,
log-scale boxes), judicious animated panels, and `--photos` hover imagery;
the `--dict-info` sidecar download row
- Aggregation-and-Statistics: `stats --jsonl` / `--pretty-json` stdout output
- Validation-and-Schema: `validate --split-ragged`, incl. QSV_COMMENT_CHAR
- Installation: the twice-a-month background update check and its opt-out
Fixed stale facts:
- Visualization said the smart downsample cap was 50,000 points in three
places; it is 150,000, which the wiki's own Environment-Variables page
already stated correctly
- FAQ gave the MSRV as 1.96; it is 1.97
- Security-and-Dependencies listed forks qsv no longer patches (`sled`,
`calamine`) and a stale plotly pin, and omitted `self_update`; added a note
on the sled -> redb migration that dropped the forked crate entirely
- Command-Reference's viz chart list was missing 8 subcommands
Every added example was executed against a current build before documenting;
that caught an animated-bar example using a column sales_sample.csv lacks.
4200617
Update Benchmarks dashboard page (qsv 21.1.0)
9996f28
docs(viz): add "Hand-tuning the Data Dictionary" section
Document how to hand-edit the describegpt JSON Schema that drives
viz smart --dictionary:
- how to get an editable <stem>.schema.json (infer or describegpt),
view it with --dict-info, and re-run
- panel routing precedence concept -> role -> content_type -> stats
- x-qsv role/concept/content_type valid values & routing (namespace/bucket depth)
- KPI tuning: gauge_range (auto-emitted) and target (hand-authored), and how
role/concept make a column KPI-eligible
- caveats: edits not re-validated, guardrails, HTML/smart-only
Verified against source and a live viz smart run.
a0e6db9
docs(describegpt): document x-qsv KPI hints (gauge_range/target) + role/concept
Add the x-qsv KPI-hint fields to the JSON Schema output section:
- x-qsv.gauge_range (auto-emitted for canonical-scale measures) -> viz smart gauge
- x-qsv.target (hand-authored, never emitted) -> "vs target" delta
- note x-qsv.role/concept and null_values/null_candidates also land here
Cross-linked to the viz smart KPI overview row.
Verified describegpt is otherwise current (content-type vocab = 49, okf,
multimodal --context-file, --infer-null-values all already documented).
582b3bd
docs(viz): document yesterday's changes — icicle, KPI overview row, legend toggle
- viz icicle: new part-to-whole hierarchy subcommand (level-aligned bars);
quick-decision row, smart --hierarchy-style option, body section, examples
- viz smart KPI overview row: big-number tiles, x-qsv.gauge_range GAUGE and
x-qsv.target DELTA hints (describegpt now emits gauge_range)
- "Toggle legend" modebar button alongside fullscreen
- smart summary choropleths from a region-code column (recognizes county_fips/
state_fips and other geo codes)
e631563
docs: document denull, describegpt --infer-null-values & viz env vars; fix qsv command count
Fill wiki gaps for changes since 21.1.0:
- denull: new command (Command-Reference row + Transform-and-Reshape section)
- describegpt --infer-null-values (AI-and-Documentation)
- Visualization env-var section: QSV_VIZ_MAX_POINTS/CDN/NO_COMPRESS, QSV_GEOJSON_SHORTCUTS
- Home: full-binary command count 76 -> 78
ec16e36
docs: bump qsvmcp tool count ~60 -> ~65 in MCP-Server
`qsvmcp --list` (prebuilt 21.1.0) reports 65 commands; the MCP tool
surface (~55 skill tools + ~10 meta-tools) is likewise ~65.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
27e5d97
docs: fix qsvmcp feature list in Binary-Variants
Verified against `qsvmcp --list` (prebuilt 21.1.0) + `cargo tree -i qsv`
and Cargo.toml:
- qsvmcp does NOT include `luau` (blocked by issue #3937, needs panic=unwind)
or `apply` (standalone NLP feature not enabled; ships feature_capable only)
- Corrected the feature cell to the canonical qsvmcp meta-feature set
- Corrected the `apply` availability row (qsv/qsvpy* only, not qsvmcp)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5c9d3eb
docs: wiki drift audit — fix stale counts, flags, viz/MapLibre, dep forks
Audited all wiki pages against current qsv source (v21.1.0):
- Command counts, version strings, stats/vocab/token counts corrected
- Removed non-existent flags (diff --select, fetch --redis-cache-conn,
QSV_TMPDIR, QSV_MEMORY_HEADROOM_MB) and fictional apply ops (cast,
wkt_point) / luau helpers; fixed luau col[] access + positional script arg
- viz page migrated Mapbox->MapLibre (DensityMap, token-free --style list,
--dict-info/--dataset-pid/--cluster/--heatmap-density)
- Security page dep-fork updates (csvlens YS-L, calamine/plotly/spreadsheet-ods),
geoconvert formats, validate keywords, Integrations release URL
- Recipe fixes: template arg order, Tieske date TAT, joinp --asof, --language
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
02a1764
viz: document violin, choropleth, treemap & sunburst
Close Visualization page drift against the current viz subcommands:
- violin: new decision-table row, subcommand example, and a rewritten
smart-dashboard "Distribution panels (violins by default)" section
covering --violin auto|on|off, the non-null point-overlay heuristic,
dense-grid demotion, the --box-points none interaction, and the
sampled-violin downsampling behavior
- choropleth: decision-table row, two examples, a "Choropleths"
subsection (--locations/--value/--agg/--location-mode/--geojson
point-in-polygon/--geocode/--map), and a static-export tile note
- treemap & sunburst: two decision-table rows, two examples, a
"Hierarchies" subsection, and a smart part-to-whole hierarchy panel
bullet documenting --hierarchy-style
All prose transcribed from docs/help/viz.md. The decision table now
covers all 20 viz subcommands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
578cb64
docs(viz): document choropleth, smart choropleth panel & --dictionary
- add viz choropleth to the quick decision table (2 rows: built-in
countries/states & custom-GeoJSON point-in-polygon)
- add a smart auto-dashboard bullet for the geographic choropleth panel
(reverse-geocoded per-state/per-country fill + point-in-polygon binning,
snap-distance / dropped-points behavior, geocode-feature gating)
- add an EXPERIMENTAL --dictionary / --dictionary-context subsection
- add choropleth sample-dataset downloads & command examples
- add a 'Choropleths in depth' workflow section
- note the new fullscreen modebar button for interactive HTML
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
5c68d80
viz: document radial sunburst in-sector text
Note the radial sunburst labels in the Gallery section and the
"Hierarchy charts in depth" sunburst description (plotly.js 3.6).
111e707
Visualization: document viz treemap & sunburst
Add the part-to-whole hierarchy charts that were missing from the page:
- decision-table rows for treemap (sized tiles) and sunburst (rings)
- a smart-dashboard bullet for the auto hierarchy panel, incl. the
Cramér's V independence screen and --hierarchy-style override
- treemap/sunburst subcommand examples (sales_sample.csv)
- a "Hierarchy charts in depth" section covering --cols/--value and the
additive-only --agg (sum/count) constraint
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7e08812
viz: point gallery link at GitHub Pages instead of githack
The gallery is now served from GitHub Pages (published from examples/viz/
by a workflow on master), so it renders with the correct content type and
the embedded smart-dashboard iframes load — no third-party proxy.
bfdcd38
viz: refresh for static export, geo outliers, --smarter, --theme
Bring the Visualization page up to date with recent viz work on master:
- Fix stale static-export claims: smart dashboards now export ALL panels
(beyond 8, via domain-positioned axes) and the offline ScatterGeo map;
only the Mapbox tile map and 3D scatter stay HTML-only (was: "capped at
8 panels" / "static export skips them").
- Document viz smart geographic-map enrichments: outlier flagging,
reverse-geocoded core extent + bounding box/location summary, and the
"Core extent"/"Full extent" buttons.
- Add the --smarter flag (one-step moarstats --advanced).
- Add a Theming section for --theme (Plotly built-in themes, all chart
types) and distinguish it from --style (map basemap).
c8f38dd
Document describegpt --context-file multimodal support
Expand the --context-file docs: file type is sniffed from contents, not
extension; text/CSV/spreadsheet inlined as {{ context }} while PDF/image
files are base64-encoded and sent as a multimodal attachment (requires a
vision-capable model/endpoint). Note the user-message default, custom
template placement, unsupported docx/pptx, the ~32 MB cap, and add a PDF
codebook example.
95733e4
Add Security & Dependency Audits page
Document the daily cargo audit (security-audit.yml + .cargo/audit.toml),
the patched-fork strategy (including the sled fork that drops unmaintained
instant + fxhash), and the current vetted ignore list for unmaintained
transitive-dep advisories. Link from the Tuning & Internals sidebar.
4522735
viz: document (NULL)/Other freq buckets, auto log-scale, and hatching cue
b6f94c7
Updated Command Reference (markdown)
e1bcda9
Updated Visualization (markdown)
33a987d
viz: document viz smart downsampling algorithms and thresholds
Add a 'Data size & downsampling' subsection covering the 50,000-point
cap, LTTB (shape-preserving) for the time-series panel, uniform stride
for scatter/map/histogram panels, and which panels summarize instead of
plotting raw points. Note downsampling is viz smart-only.
b15d4be
wiki: document describegpt --format okf (Open Knowledge Format)
Adds an "Open Knowledge Format output" section to AI-and-Documentation
(Schema table with Content Type/Enumeration columns, single-sentence
frontmatter description, --okf-type / ds-source / ds-updated mappings),
plus quick-reference + output-format list entries and a Glossary entry.
b9979a6
wiki: document viz geo/contour/scatter3d + smart auto-wiring
Visualization: add scatter3d/contour/geo to the decision table and the chart
subcommands block; document the smart-dashboard additions (correlated-pair
contour for big data, 3D triple panel, geo projection world-overview for
global extents, the box-points size heuristic + --box-points on smart); add a
viz geo paragraph to "Maps in depth".
Command-Reference: extend the viz chart-type list with scatter3d/contour/geo.
Geospatial: add a viz geo row + see-also note.
f7080a4
wiki: document moarstats-informed viz smart dashboards
The Visualization page didn't cover how `qsv moarstats` informs `viz
smart`. Add a "moarstats-informed dashboards" subsection (bimodal ->
histogram, skew/outlier box hints, concentrated high-card bars) and a
"Feeds viz smart" cross-link from the moarstats section of Aggregation
& Statistics.
4f52514
viz: document the new map subcommand
- Visualization: decision-table row, smart-dashboard map panel (plus the
time-series + correlated-pair panels the list was missing), a 'Maps in depth'
section (encodings, --density, --style/token), quakes.csv dataset, and a
Geospatial cross-link.
- Geospatial: decision-table row + See-also link to viz map.
- Command-Reference: add map to the viz chart list.
8b764a6
wiki: fix Visualization gallery link (githack, not htmlpreview)
htmlpreview.github.io renders the Plotly gallery blank (it drops external CDN
scripts and can't handle the inline bundle). Point the "View the gallery" link
at raw.githack.com, which serves the raw file as text/html so the charts render.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQL6f5DLKFHc2SM7XW5HQD
ad2d096