Sjohn/gen eval

[sjohn4]

Here is the bare minimum changes needed so we can evaluate text generation_tasks plus the perplexity eval metric.

[github-actions]

^{( % to main)
( % to main)}

[codecov]

Codecov Report

❌ Patch coverage is 85.05747% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.61%. Comparing base (c961257) to head (f46636c).

Files with missing lines	Patch %	Lines
...n/evaluator/internal/dataset/evaluation_dataset.py	76.92%	9 Missing ⚠️
modyn/evaluator/internal/core_evaluation.py	40.00%	3 Missing ⚠️
modyn/evaluator/internal/metrics/perplexity.py	96.87%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #662      +/-   ##
==========================================
+ Coverage   85.58%   85.61%   +0.02%     
==========================================
  Files         258      259       +1     
  Lines       11378    11450      +72     
==========================================
+ Hits         9738     9803      +65     
- Misses       1640     1647       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[MaxiBoether]

I think we can only review/merge this as soon as the storage PR that adds support for non-labeled is merged, no? Not sure how you did this right now without having it merged