Summary
When the in-game item hover tooltip is showing, it covers part of the item icon. Icon scanning then matches against occluded pixels and returns a confident wrong item rather than failing.
Observed in the stash: hovering Army bandage renders the game's dark name tooltip over the icon grid. The scan returned an unrelated item and the overlay showed its price (292 ₽) as if the match were good.
Why it produces a wrong answer rather than no answer
Icon scanning is template matching against the static icon set in Data/icons (RatEye.Config.Processing.Icon.UseStaticIcons). When the tooltip occludes the real icon, the best available template is still some template, so the engine returns its nearest match. RatConfig.IconScan.MinAcceptConfidence cannot filter this, because the reported confidence stays high.
From a traced session, repeated icon scans returned the same item at confidence=0.94 while different items were being clicked. The manifest for the current data bundle also records uniqueIconSourceCount 4976 against iconCount 4986 with 10 shared-source groups, so visually similar items (food especially) are exactly where a confident wrong match comes from.
Silent misidentification is the real defect here. Even without a full fix, turning this into "could not read that icon" would be a clear improvement.
Current workarounds
Both are awkward and rely on the user knowing about the problem:
- Hover so the tooltip renders above the icon, leaving the icon itself unobstructed.
- Scan quickly, before the tooltip appears.
Relevant timing facts
IconScan has no settle delay. NameScan sleeps 50 ms to let the game UI update; IconScan captures immediately. So whether the tooltip lands in the capture depends entirely on how long the user hovered before pressing the modifier, which is why "scan it fast" works.- Icon template matching costs ~202-227 ms per scan (
scan.locate_icon, split roughly evenly between ratEye.icon.template_match and ratEye.icon.template_match_rotated).
- Inspect-window OCR costs ~17-54 ms (
ratEye.inspection.title_scan / ocr_recognize).
OCR is both faster and less ambiguous than template matching here.
Candidate fixes
Not deciding an approach in this issue, but the two directions are:
- Detect the occluding tooltip and mask it out, or refuse to match. Converts a confident wrong answer into an honest failure. Useful on its own.
- Read the hover tooltip text instead of matching the icon. The tooltip contains the item name, and the engine already has the OCR path used for inspect-window titles. Potentially more accurate and cheaper than template matching.
Repository split
- RatScanner owns orchestration: when to capture, and what to do when the capture is occluded (reject, retry, fall back to a different identification strategy).
- RatEye owns the image work: occlusion detection, masking, or tooltip OCR.
A companion RatEye issue should be opened once an approach is chosen. Filing it now would mean an engine issue for an undecided design.
Related
Same family as the icon-scan limitations already documented in README.md ("Shared icons (especially keys) can match incorrectly", "Stash lighting can interfere with the top-left stash area") and FAQ.md → "Icon scanning gets a lot of wrong matches".
Summary
When the in-game item hover tooltip is showing, it covers part of the item icon. Icon scanning then matches against occluded pixels and returns a confident wrong item rather than failing.
Observed in the stash: hovering
Army bandagerenders the game's dark name tooltip over the icon grid. The scan returned an unrelated item and the overlay showed its price (292 ₽) as if the match were good.Why it produces a wrong answer rather than no answer
Icon scanning is template matching against the static icon set in
Data/icons(RatEye.Config.Processing.Icon.UseStaticIcons). When the tooltip occludes the real icon, the best available template is still some template, so the engine returns its nearest match.RatConfig.IconScan.MinAcceptConfidencecannot filter this, because the reported confidence stays high.From a traced session, repeated icon scans returned the same item at
confidence=0.94while different items were being clicked. The manifest for the current data bundle also recordsuniqueIconSourceCount 4976againsticonCount 4986with 10 shared-source groups, so visually similar items (food especially) are exactly where a confident wrong match comes from.Silent misidentification is the real defect here. Even without a full fix, turning this into "could not read that icon" would be a clear improvement.
Current workarounds
Both are awkward and rely on the user knowing about the problem:
Relevant timing facts
IconScanhas no settle delay.NameScansleeps 50 ms to let the game UI update;IconScancaptures immediately. So whether the tooltip lands in the capture depends entirely on how long the user hovered before pressing the modifier, which is why "scan it fast" works.scan.locate_icon, split roughly evenly betweenratEye.icon.template_matchandratEye.icon.template_match_rotated).ratEye.inspection.title_scan/ocr_recognize).OCR is both faster and less ambiguous than template matching here.
Candidate fixes
Not deciding an approach in this issue, but the two directions are:
Repository split
A companion RatEye issue should be opened once an approach is chosen. Filing it now would mean an engine issue for an undecided design.
Related
Same family as the icon-scan limitations already documented in
README.md("Shared icons (especially keys) can match incorrectly", "Stash lighting can interfere with the top-left stash area") andFAQ.md→ "Icon scanning gets a lot of wrong matches".