Integrate data viz using spotlight

[bw4sz]

I'm looking at more tightly connected huggingface and spotlight to allow users to get a better handle on their data.

[bw4sz]

https://github.com/Renumics/spotlight

[bw4sz]

https://docs.lightly.ai/studio/about-us/

[bw4sz]

We really need a gallery view, where you can look at crops of many images at once, not on a per-image basis.

[Surjasa]

in progress

[Surjasa]

@bw4sz for Spotlight integration, do you prefer we produce a Lightly-style dataset or manifest (Lightly-style fields/API) or a package that follows Spotlight’s own ingest schema or validator?

[bw4sz]

Thanks for your interest. Let's make a roadmap here.

Motivation

DeepForest users need more insight into annotations and visualization is time consuming. We have simple tools like DeepForest.visualize.plot_annotations, but they only provide static overlays and not much UI. Spotlight looks like a nice way of seeing input images and exploring predictions to find bad annotations and cluster outputs.

Use-cases

1. Read a dataset using deepforest.utilities.read_file so that it has the DeepForest columns and schema. I'm open to ideas on how a user invokes spotlight.

One idea is that we create a DeepForest data class that subclasses a geopandas dataframe (the output of read_file) and add a method.

from deepforest import get_data
from deepforest.utilities import read_file

# sample package data
path = get_data("OSBS_029.csv")
df = read_file(path)
df.spotlight()

another is that we make a small wrapper function that converts the dataframe to spotlight, probably easier

from deepforest import get_data
from deepforest.utilities import read_file
from deepforest.visualize import view_with_spotlight

# sample package data
path = get_data("OSBS_029.csv")
df = read_file(path)
view_with_spotlight(df)

2. Add in predictions or embeddings. This is more advanced and might take a couple iterations. Predictions will be easier.

from deepforest import get_data
from deepforest.utilities import read_file
from deepforest.visualize import view_with_spotlight
from deepforest.main import deepforest

# sample package data
model = deepforest.main()
model.load_model("Weecology/deepforest-tree")
path = get_data("OSBS_029.csv")
df = read_file(path)
results = model.predict_tile(df)
view_with_spotlight(results)

That will be predictions. We have not yet gotten to embeddings, we'd need to discuss (@jveitchmichaelis ) what layers we might extract from, etc. Let's leave that until a future PR.

Roadmap

• update dependencies and install
• add tests to cover basic behavior
• Decide on a wrapper versus class approach to getting data into spotlight format. I don't know the format well enough to comment on your question above
• Add to docs an example and a screenshot.
• Tests and CI must pass to review the PR

Thanks for your contribution to open-source.

[Surjasa]

Thanks for your roadmap and guidance.

According to my understanding, the initial goal is to enable interactive visualization of DeepForest annotations/predictions in Spotlight (embeddings deferred to a future PR). This PR implements the requested functionality:

• view_with_spotlight(df) wrapper, and df.spotlight() accessor
• basic Spotlight format conversion and manifest export
• CLI command for packaging galleries, tests, and docs with screenshots

All tests and CI pass on my branch. Next I can address any reviewer feedback, add small refinements to the docs if requested, or follow up later with embeddings after we decide which model layers to use. Please let me know if you want me to split or reduce anything for review.
Thank you.