Skip to content

Fix #356 compatibility#361

Open
KartikP wants to merge 16 commits intomainfrom
epflneuroailab-changes
Open

Fix #356 compatibility#361
KartikP wants to merge 16 commits intomainfrom
epflneuroailab-changes

Conversation

@KartikP
Copy link
Contributor

@KartikP KartikP commented Feb 11, 2026

From #356 but with other changes to get it to work with testing infrastructure.

@KartikP KartikP force-pushed the epflneuroailab-changes branch from ec07cd5 to 27f9d2e Compare February 11, 2026 20:48
@KartikP KartikP force-pushed the epflneuroailab-changes branch from bb6ba01 to b913b91 Compare February 11, 2026 21:07
KartikP added 10 commits February 11, 2026 16:08
@KartikP
Merge branch 'main' into epflneuroailab-changes
3fa2a36
@KartikP
restore plugin submission orchestrator
4f6bf08
@KartikP
import numpy
deb1899
@KartikP
Add layer to dummy model neuroidassembly
1a544ae
@KartikP
return score object with mean of list of score
6a65f52
@KartikP
Update blank2014 test_ceiling expected value for GroupKFold CV
6738606 
The new cross-validation kwargs (split_coord="story", kfold="group")
change the ceiling from 0.2103 to 0.1446.
@KartikP
Propagate split_coord to predictions and fix MultiIndex detection
8090665 
- blank2014: add story coord to predictions (needed for split_coord="story")
- fedorenko2016: add sentence_id coord to predictions (needed for split_coord="sentence_id")
- pereira2018: replace .coords check with try/except to handle xarray
  MultiIndex levels not appearing in .coords (xarray 2022.3.0 behavior)
- pereira2018/test.py: update test_dummy_bad expected scores for new
  GroupKFold CV strategy (243: 0.0186->0.0, 384: 0.0334->0.0168)
- linear_predictivity/test.py: update expected score (0.0283->0.0410)
@KartikP
remove cv_kwargs for linear
9adee94
@KartikP
account for difference in pearsonr correlation methods
dbe92a8
@KartikP
Propogate raw and ceiling values for database
16e896a
@mike-ferguson mike-ferguson added the submission_prepared Attached to a PR is metadata and layer mapping is successful. label Feb 12, 2026
@KartikP KartikP removed the submission_prepared Attached to a PR is metadata and layer mapping is successful. label Feb 12, 2026
@mike-ferguson mike-ferguson added the submission_prepared Attached to a PR is metadata and layer mapping is successful. label Feb 12, 2026
@KartikP KartikP changed the title Group-based Splitting for Benchmarks + Batch-layer Evaluation + GPU Support Fix #356 compatibility Feb 12, 2026
This was referenced Feb 12, 2026
Open
Merged
@KartikP
Copy link
Contributor Author

KartikP commented Feb 13, 2026

@BKHMSI Would like your feedback on my changes for your PR when you have the chance.

@BKHMSI
Copy link
Contributor

BKHMSI commented Feb 18, 2026

Hi @KartikP, thanks for making it work with the testing infrastructure.

The only comment I have is that the benchmarks with linear regression (e.g., Blank2014-linear) should use the same cross validation kwargs as the ridge regression one (i.e., group-based splitting). If you want to keep the old method, then I propose to give it the identifier Blank2014-linear-legacy or something similar.

@KartikP
Copy link
Contributor Author

KartikP commented Feb 18, 2026

Hi @KartikP, thanks for making it work with the testing infrastructure.

The only comment I have is that the benchmarks with linear regression (e.g., Blank2014-linear) should use the same cross validation kwargs as the ridge regression one (i.e., group-based splitting). If you want to keep the old method, then I propose to give it the identifier Blank2014-linear-legacy or something similar.

I'll verify that your original implement can reproduce previous scores. Otherwise, I'll mark the linear benchmarks as linear-legacy and then add the kwargs back in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

submission_prepared Attached to a PR is metadata and layer mapping is successful.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@KartikP @BKHMSI @mike-ferguson

Comments