Validity and Stability Pipeline Development FS Benchmark#288
Validity and Stability Pipeline Development FS Benchmark#288schaeferbasti wants to merge 9 commits intoautogluon:fe_benchmark_mainfrom
Conversation
|
|
||
| print(result) | ||
| result = pd.DataFrame([result.__dict__]) | ||
| path = f"results/{args.mode}_{args.method_name}_{args.data_foundry_task_id.split('|')[3].split('/')[0]}_{args.repeat}.csv" |
There was a problem hiding this comment.
Consider adding code that checks if the cache exists and then skips unless we pass --ignore-cache.
There was a problem hiding this comment.
Also maybe make this logic a function so that we can easily go from argumetns to cache path
|
|
||
|
|
||
| @dataclass | ||
| class ExtraBenchmarkSetup2026: |
There was a problem hiding this comment.
Let us not have this in the TabArena TabFlow code, but in the experimental part. Also, I think this kind of setup won't work the same way, might be a good starting point, but likely you can make it much simpler and hardcode a lot of choices. In theory, you just need to generate a loop + array job with the values of this loop. For example, check out some suggestions from Claude Code or ChatGPT for how to do this
SLURM does not need all this setup loigc but most importantly the batch file. There is also submitit as a python package as an alternative
2 participants
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.