Does it make sense reducing outliers before reducing topics? Is it possible?

[diegodlh]

I want to do both outlier and topic reduction. Intuitively I would think that the right order would be outlier reduction first (to accommodate some of the outlier documents in one of the original topics available) and topic reduction afterwards.

However, I've seen the warning in the docs that running topic reduction after outlier reduction (and corresponding topic update) may lead to errors: https://maartengr.github.io/BERTopic/getting_started/outlier_reduction/outlier_reduction.html#update-topics

According to this comment, running update_topics should be the last step. Does this mean:

1. running reduce_outliers() first,
2. then reduce_topics(),
3. and finally update_topics() with the new_topics from step 1?

Won't the new_topics from step 1 refer to topics already reduced in step 2?

Or does the comment mean running reduce_outliers() and corresponding update_topics() after reduce_topics()?

Thanks!

[MaartenGr]

When you reduce topics, it does so on the full topics along with all mappings that might exist. However, if you reduce outliers, some specific documents from the same topic might be ssigned to different topics, which makes the mapping not a 1:1 mapping anymore when you reduce them. As such, I think you might run into issues if you would do outlier reduction first before you run the reduction. In other words, I would advise running the reduce_topics first.