Implementation of Handling Arbitrary Miscalibrations in Ratings#185
Open
AvitalZar wants to merge 10 commits into
Open
Implementation of Handling Arbitrary Miscalibrations in Ratings#185AvitalZar wants to merge 10 commits into
AvitalZar wants to merge 10 commits into
Conversation
…a. it doesn't what they did in the lib, b. it's a probabilistic method.
tests: created. all fails as well.
…cking for w's validation. a little fixing in the tests.
…t matter how theoretically it beats every other estimator when practically non of them get to the right answer even once:)
…of uniform dominance based on kendall tau destination.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi :)
I have implemented the core algorithm from the paper "Handling Arbitrary Miscalibrations in Ratings" (https://arxiv.org/abs/1806.05085). This algorithm provides a probabilistic method for ranking a set of items based on their scores, combining preference-based ranking and numerical ratings.
Here is a summary of the changes introduced in this PR:
New Feature Implementation:
I added the Randomized Grade Calibrated Ranking (RGCR) function. I placed it in the stochastic_methods file, as it relies on a probabilistic approach to generate the final ranking. I didn't place it under probabilistic methods, since computing the exact probability for each item is computationally impractical here, stochastic_methods felt like the most logical home. However, I am completely open to relocating it if you feel another file is more appropriate.
Comprehensive Testing:
I added a set of comprehensive tests for the new function in a new file: test/test_rgcr_method. Since I could not find an existing location for function-specific tests, I created this separate file. The tests thoroughly verify the algorithm's properties as described in the original paper.
Dependencies:
I updated the requirements file to include 'sortedcontainers'. This library was required for a specific data structure used during one of the internal checks within the function.
Everything has been tested and works as expected.
I would appreciate your feedback and review!
Best regards,
Avital Zar.