KNN output type mismatch in full_stack_prediction

[bleedblack1]

Adds a transformation layer between knn_batch() and full_stack_prediction() to resolve a structural type mismatch that causes TypeError: unhashable type: 'dict' when KNN refinement is enabled (KNN_VALUE_DEFAULT > 0).

---

Problem

knn_batch() returns:

list[list[dict[str, Any]] | None]

where each non-None element is a list of record dicts:

[
    {"ACCESSION": "...", "COSINE_SIMILARITY": 0.9, "BIOME": "root:Environmental:..."},
    {"ACCESSION": "...", "COSINE_SIMILARITY": 0.85, "BIOME": "root:Environmental:..."},
]

However, full_stack_prediction() consumes the KNN results as if they were:

dict[str, float]  # {biome_lineage_string: score}

This is the same format used by the deep learning predictions (unambig_pred, unambig_constr_pred, etc.). The mismatch surfaces at two points:

1. Heuristic selection :

best_heuristic_gold = biome_herarchy_dct_reversed.get(
    list(best_heuristic)[0] if best_heuristic is not None else None,
    None,
)

When best_heuristic is a list[dict], list(best_heuristic)[0] returns a dict, not a string. Using a dict as a dict key raises TypeError: unhashable type: 'dict'.

2. Score extraction :

best_heuristic_gold = {best_heuristic_gold: best_heuristic[list(best_heuristic)[0]]}

Indexing a list with a dict raises TypeError.

3. normalize_to_ontology :

normalize_to_ontology(best_heuristic, target_ontology="AMENDED")

Calls .items() on the prediction — fails on list[dict].

Why this is currently hidden

KNN_VALUE_DEFAULT is set to -1, which causes knn_batch() to early-return [None] * len(predictions). The bug is only triggered when KNN is re-enabled, which the codebase's own TODO indicates is intended:

KNN_VALUE_DEFAULT = -1  # THE FUNCTION IS DEPRECATED. TODO: Transform function to KNN tool

---

Solution

New bridge function: _knn_records_to_prediction()

Converts the raw KNN record list into the {biome: score} dict that all downstream code expects:

def _knn_records_to_prediction(
    records: list[dict[str, Any]] | None,
) -> dict[str, float] | None:

Logic:

1. Groups all returned KNN neighbours by their BIOME value
2. Computes the mean cosine similarity per biome across all neighbours
3. Returns a single-entry dict {best_biome: mean_score} for the biome with the highest mean similarity
4. Returns None if input is None or empty

This matches the single-entry dict pattern used by the unambiguous prediction paths:

# Existing pattern in the codebase:
top_dominant = {top_dominant_term: _top_pred.get(top_dominant_term, 0)}

Transformation applied at both KNN call sites

# Before (unconstrained)
refined_predictions = knn_batch(...)
After (unconstrained)
refined_predictions_raw = knn_batch(...)

refined_predictions = [_knn_records_to_prediction(rp) for rp in refined_predictions_raw]
Before (constrained)
constrained_refined_predictions = knn_batch(...)
After (constrained)
constrained_refined_predictions_raw = knn_batch(...)

constrained_refined_predictions = [

_knn_records_to_prediction(crp) for crp in constrained_refined_predictions_raw

]

---

Files Changed

File	Change
trapiche/taxonomy_prediction.py	Added _knn_records_to_prediction() bridge function; applied transformation at both KNN call sites in full_stack_prediction()

---

Heuristic Priority (unchanged)

The fix preserves the intended priority ordering documented in the code:

1. refined_constrained_prediction (KNN + text constraints)
2. unambig_constr_pred (deep model + text constraints)
3. refined_prediction (KNN unconstrained)
4. unambig_pred (deep model unconstrained)
5. None

---