Only plot head-specific test set in results for multihead models#1445
Conversation
|
@vue1999 Could you check that? |
vue1999
left a comment
There was a problem hiding this comment.
Thanks for this, I have checked the code and tested it, looks good to me. Could you please sync the branch with main and ensure the unit tests pass?
7bbbef0 to
9d8ef53
Compare
Thanks @vue1999 for testing. I've rebased now. lt seems the errors are coming from files i have not touched. any advice? |
vue1999
left a comment
There was a problem hiding this comment.
Looks good, tested and unit tests pass✅
Thanks, the unit tests were fixed in the main branch changes, so after rebasing they have succeeded as expected |
Amazing, thanks! |
Issue:
When training a multi-head model, all heads' test sets are plotted on every head's results panel. For a 3-head
model, each panel shows test data from all 3 heads (each evaluated with its own readout), all labelled "Test".
This has two consequences:
Solution:
run_train.py: Label each test set with its head name before passing to the plottervisualise_train.py: only plot the test data for the current head + add the head name to the legend labelTesting
2 epochs with: single head, multihead, naive FT, mutlihead FT
Multihead before (3 heads) (cropped out training plots as not relevant here):