Fine-tuning guidance

[ahrm]

I don't know if this can be considered an issue, but I have trouble fine-tuning needle on (what I consider to be) a very simple data extraction task. The goal is to extract author and paper title data from a citation like: Bantilan, N. (2018). Themis-ml: A Fairness-Aware Machine Learning Interface for End-To-End Discrimination Discovery and Mitigation. Journal of Technology in Human Services, 36(1), 15-30. https://doi.org/10.1080/15228835.2017.1416512

I have prepared a dataset of 200 datapoints which looks like this:

...
{"query": "Bantilan, N. (2018). Themis-ml: A Fairness-Aware Machine Learning Interface for End-To-End Discrimination Discovery and Mitigation. Journal of Technology in Human Services, 36(1), 15-30. https://doi.org/10.1080/15228835.2017.1416512", "tools": [{"name": "extract_citation_data", "parameters": {"type": "object", "properties": {"authors": {"type": "string"}, "title": {"type": "string"}, "publisher": {"type": "string"}}, "required": ["authors", "title"]}}], "answers": [{"name": "extract_citation_data", "arguments": {"authors": "Bantilan, N.", "title": "Themis-ml: A Fairness-Aware Machine Learning Interface for End-To-End Discrimination Discovery and Mitigation", "publisher": "Journal of Technology in Human Services, 36(1), 15-30."}}]}
{"query": "Mishra, Asit K., et al. \u201cTowards characterizing cloud backend workloads: insights from Google compute clusters.\u201d SIGMETRICS Perform. Eval. Rev., vol. 37, no. 4, 2010, pp. 34\u201341. https://doi.org/10.1145/1773394.1773400", "tools": [{"name": "extract_citation_data", "parameters": {"type": "object", "properties": {"authors": {"type": "string"}, "title": {"type": "string"}, "publisher": {"type": "string"}}, "required": ["authors", "title"]}}], "answers": [{"name": "extract_citation_data", "arguments": {"authors": "Mishra, Asit K., et al.", "title": "Towards characterizing cloud backend workloads: insights from Google compute clusters", "publisher": "SIGMETRICS Perform. Eval. Rev., vol. 37, no. 4, 2010, pp. 34\u201341."}}]}
...

However, when I run needle finetune data.jsonl --epochs 3, the loss doesn't seem to decrease:

checkpoints/needle2.pkl not found locally, downloading from HF...

checkpoints/needle2.pkl: downloading bytes:   0% 0.00/90.4M [00:00<?, ?B/s]
checkpoints/needle2.pkl: downloading bytes:  98% 88.9M/90.4M [00:01<00:00, 86.7MB/s, 7.27MB/s  ]
checkpoints/needle2.pkl: downloading bytes: 100% 88.9M/88.9M [00:01<00:00, 46.0MB/s, 8.30MB/s  ]
checkpoints/needle2.pkl: reconstructing file: 100% 90.4M/90.4M [00:01<00:00, 46.8MB/s, 8.63MB/s  ]
Downloading tokenizer from HuggingFace...
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.

tokenizer/tokenizer.model: downloading bytes:   0% 0.00/132k [00:00<?, ?B/s]
tokenizer/tokenizer.model: downloading bytes: 100% 85.3k/85.3k [00:00<00:00, 162kB/s, 8.46kB/s  ]
tokenizer/tokenizer.model: reconstructing file: 100% 132k/132k [00:00<00:00, 251kB/s, 13.1kB/s  ]
tokenizer.vocab: 100% 113k/113k [00:00<00:00, 42.0MB/s]
training on 200 examples, seq_len 1024
LoRA rank 16 on 5 weight groups (compiling...)
epoch 1/3  step 1/39  loss 1.0496
epoch 1/3  step 2/39  loss 1.1264
epoch 1/3  step 3/39  loss 1.3048
epoch 1/3  step 4/39  loss 1.3791
epoch 1/3  step 5/39  loss 1.1333
epoch 1/3  step 6/39  loss 1.0731
epoch 1/3  step 7/39  loss 0.9701
epoch 1/3  step 8/39  loss 1.0592
epoch 1/3  step 9/39  loss 1.1624
epoch 1/3  step 10/39  loss 1.1403
epoch 1/3  step 11/39  loss 1.0086
epoch 1/3  step 12/39  loss 1.0423
epoch 1/3  step 13/39  loss 1.0337
epoch 1/3  loss 1.0337
epoch 2/3  step 14/39  loss 1.0248
epoch 2/3  step 15/39  loss 1.0500
epoch 2/3  step 16/39  loss 0.9756
epoch 2/3  step 17/39  loss 1.0285
...

And when I use the final fine-tuned model, it just always says "sorry, I can't help you with that".

Am I doing something wrong?

[HenryNdubuaku]

This issue has already been fixed, you simply need to pip install --upgrade cactus-needle

[ahrm]

What issue? Note that this is not about the NaN issue, which is indeed fixed, as you see the loss is not NaN, but nevertheless it is not being fine-tuned effectively. I installed the latest version from the github directly.

[HenryNdubuaku]

The issue about model saying "sorry, I can't help you with that", just waiting for other fixes and the python package will be updated in 1hr. That was a gating in the old binary for when model confidence is low, and it has been removed, now.

[HenryNdubuaku]

As for fine-tuning guidance, I will create a proper write-up and notify you, until then, I'll leave the issue open.

[HenryNdubuaku]

@ahrm here: