Pasted from the slack chat:
Hey @faisal Alquaddoomi and @vincent
I spoke with Arjun yesterday to nail down the feedback features we want to include in the first version of the web server. All of the following are described in-detail in the user-stories document under the PHASE 1 FEEDBACK SYSTEM FOR META2ONTO v1 section. Additionally, these are all features that we would like to implement in the perfect scenario. If there are features that are too difficult to implement, we can try to include them later or skip them entirely.
Study-level feedback - Thumbs up/down to indicate if a search result is good/bad along with a contextual reason selection (i.e. someone would click "thumbs down" and then have to give a reason why by clicking a button for "wrong disease/tissue" or "metadata is incomplete so I can't tell").
Word-level feedback - For each study returned from a search, words in the description that were deemed informative by our model when making that prediction will be highlighted. There should be some sort of popup or dropdown menu that allows someone to say this is a poor prediction and why.
Missing-information feedback - Allow users to highlight the text in the study descriptions themselves and tell us that those words should have been used by our model to make that prediction and why (i.e., a word should have been highlighted, but was not).
For example, if someone searched "kidney" and got a list of kidney studies returned, but the word "kidney" in one of the study descriptions is not highlighted, then they should tell us because our model is missing key information in the text.
Sample-level feedback - Thumbs up/down buttons (same as for study-level).
Feedback progress tracking - Let the user know how many studies they've provided feedback for
Please feel free to take creative liberty on these features if you think there is a better way to implement them. Happy to discuss more.
Pasted from the slack chat:
Hey @faisal Alquaddoomi and @vincent
I spoke with Arjun yesterday to nail down the feedback features we want to include in the first version of the web server. All of the following are described in-detail in the user-stories document under the PHASE 1 FEEDBACK SYSTEM FOR META2ONTO v1 section. Additionally, these are all features that we would like to implement in the perfect scenario. If there are features that are too difficult to implement, we can try to include them later or skip them entirely.
Study-level feedback - Thumbs up/down to indicate if a search result is good/bad along with a contextual reason selection (i.e. someone would click "thumbs down" and then have to give a reason why by clicking a button for "wrong disease/tissue" or "metadata is incomplete so I can't tell").
Word-level feedback - For each study returned from a search, words in the description that were deemed informative by our model when making that prediction will be highlighted. There should be some sort of popup or dropdown menu that allows someone to say this is a poor prediction and why.
Missing-information feedback - Allow users to highlight the text in the study descriptions themselves and tell us that those words should have been used by our model to make that prediction and why (i.e., a word should have been highlighted, but was not).
For example, if someone searched "kidney" and got a list of kidney studies returned, but the word "kidney" in one of the study descriptions is not highlighted, then they should tell us because our model is missing key information in the text.
Sample-level feedback - Thumbs up/down buttons (same as for study-level).
Feedback progress tracking - Let the user know how many studies they've provided feedback for
Please feel free to take creative liberty on these features if you think there is a better way to implement them. Happy to discuss more.