Skip to content

Comments

feat: classification dataset support (v1.2)#2

Open
ortizeg wants to merge 37 commits intodevelopfrom
feat/stats-class-filter
Open

feat: classification dataset support (v1.2)#2
ortizeg wants to merge 37 commits intodevelopfrom
feat/stats-class-filter

Conversation

@ortizeg
Copy link
Owner

@ortizeg ortizeg commented Feb 16, 2026 •
edited
Loading

Summary

  • Add class filter checkboxes to statistics tab for slicing data by label
  • First-class single-label classification dataset support with full feature parity to detection workflows
  • Classification JSONL parser with auto-detection, multi-split ingestion, and sentinel bbox pattern
  • Evaluation metrics: accuracy, macro/weighted F1, per-class P/R/F1, clickable confusion matrix
  • Error analysis categorizing images as correct, misclassified, or missing prediction
  • Confusion matrix polish with threshold filtering and overflow scroll for 43+ classes
  • Embedding scatter color modes: GT class, predicted class, correct/incorrect (Tableau 20 palette)
  • Most-confused class pairs summary and F1 bars with color-coded thresholds

Milestone v1.2 — 3 phases, 6 plans, 34 commits

Phase Scope
15 Classification ingestion & display (parser, grid badges, modal editor, stats)
16 Classification evaluation (predictions, metrics, confusion matrix, error analysis)
17 Classification polish (threshold filtering, most-confused pairs, embedding colors)

Stats: 70 files changed, +8,035 / -3,883 lines

Test plan

  • Import a classification JSONL dataset (auto-detects type)
  • Verify grid shows class label badges (no bbox overlays)
  • Open detail modal — class dropdown editor works
  • Import classification predictions — evaluation panel shows accuracy/F1/confusion matrix
  • Click confusion matrix cell — grid filters to that GT/pred pair
  • Error analysis shows correct/misclassified/missing categories
  • Confusion matrix threshold slider filters noise at 43+ classes
  • Embedding scatter color modes work (GT class, predicted, correct/incorrect)
  • Class filter checkboxes in statistics tab filter metrics
  • Detection datasets unaffected (no regression)

🤖 Generated with Claude Code

ortizeg and others added 30 commits February 14, 2026 12:28
@ortizeg @claude
chore: archive v1.1 milestone
cbae356 
Archive Deployment, Workflow & Competitive Parity milestone:
- 7 phases (8-14), 20 plans, 26/26 requirements complete
- Roadmap and requirements archived to .planning/milestones/
- PROJECT.md evolved with validated v1.1 requirements
- ROADMAP.md collapsed v1.1 into details tag

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
feat: add class filter checkboxes to statistics overview tab
928cfb2 
Allow users to exclude specific classes from statistics computation
to analyze data slices by label. Collapsible checkbox panel with
select-all/deselect-all controls filters class distribution chart
and recomputes summary stats client-side.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
feat: apply class filter to evaluation tab with cached recomputation
a468c62 
Lift ClassFilter above sub-tab navigation so it's shared across all
tabs. EvaluationPanel now receives excludedClasses and uses a new
useFilteredEvaluation hook that:
- Filters PR curves, per-class metrics, and confusion matrix rows/cols
- Recomputes mAP as mean of filtered per-class AP values
- Synthesizes a new "all" PR curve from included classes (COCO 101-pt)
- Caches results in a Map keyed by serialized excluded set, so revisiting
  the same combination is O(1)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg
docs: start milestone v1.2 Classification Dataset Support
aec9d6b
@ortizeg
docs: complete project research
7f9043d
@ortizeg
docs: define milestone v1.2 requirements
927afab
@ortizeg
docs: create milestone v1.2 roadmap (3 phases)
0b196c1
@ortizeg
docs(15): research phase domain
8754346
@ortizeg
docs(15): create phase plan
6868104
@ortizeg
fix(15): revise plans based on checker feedback
9286e8e
@ortizeg @claude
feat(15-01): add ClassificationJSONLParser, dataset_type schema migra…
5264e51 
…tion, and model updates

- Create ClassificationJSONLParser with sentinel bbox values (0.0) for classification annotations
- Add dataset_type column migration to DuckDB schema (default 'detection')
- Add dataset_type field to DatasetResponse Pydantic model
- Update BaseParser.build_image_batches signature with image_dir parameter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
feat(15-01): add scanner detection, parser dispatch, dataset_type API…
8af8a11 
…, and category update endpoint

- Add classification JSONL layout detectors (D: split dirs, E: flat) to FolderScanner
- Add GCS classification detection support
- Dispatch to ClassificationJSONLParser in IngestionService based on format
- Store dataset_type (classification/detection) on dataset INSERT
- Thread format through ImportRequest -> ingest_splits_with_progress -> ingest_with_progress
- Add dataset_type to GET /datasets and GET /datasets/{id} responses
- Add PATCH /annotations/{id}/category endpoint for classification label editing
- Make statistics gt_annotations classification-aware (distinct labeled images)
- Show .jsonl files in browse endpoint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
docs(15-01): complete classification ingestion backend plan
0f93248 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
feat(15-02): add dataset_type threading, grid class badges, and scan …
b96ce5e 
…format badge

- Add dataset_type field to Dataset TypeScript interface
- Thread datasetType prop from dataset page to ImageGrid, SampleModal, StatsDashboard
- Add ClassBadge component to GridCell for classification datasets (replaces bbox overlay)
- Show "Classification JSONL" format badge in scan results for classification_jsonl format
- Add datasetType prop signatures to SampleModal and StatsDashboard for Task 2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
feat(15-02): classification-aware modal, annotation list, and statistics
e7ad776 
- Add class label dropdown editor in SampleModal for classification datasets
- Wire PATCH /annotations/{id}/category mutation for category updates
- Hide bbox overlay, annotation editor, and triage toolbar for classification
- Show predicted class with confidence in classification modal
- Hide bbox/area columns in AnnotationList for classification datasets
- Hide Evaluation, Error Analysis, Worst Images, Intelligence tabs for classification
- Swap summary card labels: "Labeled Images" and "Classes" for classification

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
docs(15-02): complete classification frontend display plan
bfac740 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg
docs(phase-15): complete phase execution
f522fda
@ortizeg @claude
docs(16): research phase domain
e148018 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg
docs(16): create phase plan
d6eef42
@ortizeg @claude
feat(16-01): add classification prediction parser and import endpoint
76bda2a 
- Create ClassificationPredictionParser with flexible key lookup for filename, label, confidence
- Add classification_jsonl format to PredictionImportRequest
- Route classification_jsonl predictions through new parser in import_predictions
- Handle classification_jsonl in run_name derivation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
feat(16-01): add classification evaluation, confusion cell, and error…
9f38741 
… analysis services

- Create ClassificationEvaluationResponse and ClassificationPerClassMetrics models
- Implement compute_classification_evaluation with accuracy, F1, confusion matrix
- Implement get_classification_confusion_cell_samples without IoU matching
- Implement classify_errors for correct/misclassified/missing categorization
- Route evaluation, confusion-cell-samples, and error-analysis endpoints by dataset_type
- Detection evaluation paths remain completely untouched

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
docs(16-01): complete classification evaluation backend plan
5d6c2ee 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
feat(16-02): add classification types, prediction import format, and …
7dca67e 
…grid predicted label badge

- Add ClassificationEvaluationResponse and AnyEvaluationResponse union type
- Add classification_jsonl format to PredictionImportRequest
- Update useEvaluation hook to return AnyEvaluationResponse
- Update useFilteredEvaluation to pass through classification responses
- Add Classification JSONL option to prediction import dialog
- Add predicted class badge (green=correct, red=mismatch) on grid cells

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
feat(16-02): classification evaluation panel and error analysis panel
5d1433e 
- Un-hide Evaluation and Error Analysis tabs for classification datasets
- Classification evaluation shows accuracy, macro F1, weighted F1 metric cards
- Classification evaluation renders confusion matrix with click-to-filter
- Classification evaluation renders per-class P/R/F1/Support table
- Classification error analysis shows correct/misclassified/missing categories
- IoU slider hidden for classification in both panels
- Detection evaluation and error analysis completely unchanged

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
docs(16-02): complete classification evaluation frontend plan
1bce172 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
fix(16-02): use misclassified key for classification error samples grid
3bbeed2 
The classification error analysis backend populates samples_by_type with
"misclassified" key, but the frontend was reading "label_error" (detection key).
Fall back to label_error for detection compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg
docs(phase-16): complete phase execution
79ff323
@ortizeg
docs(classification-polish): research phase domain
a5f9460
@ortizeg
docs(17): create phase plan
2893b39
@ortizeg @claude
feat(17-01): add threshold filtering and overflow scroll to confusion…
10a3230 
… matrix

- Add threshold slider (0-50%, default 1%) to hide low-value off-diagonal cells
- Wrap table in overflow-auto max-h-[500px] scroll container
- Compact mode for >20 classes: smaller cells, truncated labels, 10px text
- Show hidden cell count next to slider
- Diagonal cells always visible regardless of threshold

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ortizeg and others added 7 commits February 18, 2026 22:57
@ortizeg @claude
feat(17-02): enrich coordinates endpoint with GT/pred labels
4ff366a 
- LEFT JOIN annotations for ground_truth and prediction labels
- MIN() + GROUP BY to handle multi-annotation edge cases
- Add optional gtLabel/predLabel fields to EmbeddingPoint type

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
feat(17-01): add most-confused pairs and F1 bars to classification eval
660d287 
- Add MostConfusedPairs component: top 10 off-diagonal pairs ranked by count
- Clickable rows filter grid to show misclassified samples
- Add F1Bar component: color-coded bar (green/yellow/red) for per-class F1
- Add Performance column to ClassificationPerClassTable with F1Bar
- Integrate MostConfusedPairs between confusion matrix and per-class table

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
feat(17-02): add color mode dropdown and categorical coloring to embe…
1f4c858 
…dding scatter

- Add ColorMode type and Tableau 20 categorical palette to scatter
- getFillColor branches on colorMode (default/gt_class/pred_class/correctness)
- Color mode dropdown in toolbar with prediction-dependent options disabled
- Thread datasetType prop to EmbeddingPanel from page
- Invalidate embedding-coordinates cache after prediction import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
docs(17-01): complete confusion matrix polish plan
af2607e 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg @claude
docs(17-02): complete embedding color modes plan
fc0e2d1 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg
docs(phase-17): complete phase execution — milestone v1.2 complete
67a7a9c
@ortizeg @claude
chore: archive v1.2 Classification Dataset Support milestone
09da917 
Archive roadmap and requirements to milestones/, update PROJECT.md with
validated requirements and new decisions, collapse v1.2 in ROADMAP.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ortizeg ortizeg changed the title feat: add class filter to statistics overview tab feat: classification dataset support (v1.2) Feb 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ortizeg