What system prompt do you recommend?

[th789]

Hello Meditron team,

Happy New Year! Hope you are doing well. Thank you so much for releasing Meditron! I have the following questions on the recommended system prompt and how to input it to the model.

1. It seems that during evaluation, different system prompts are used for different datasets (https://github.com/epfLLM/meditron/blob/main/evaluation/inference.py#L121). In general, when using the meditron models posted on huggingface (https://huggingface.co/epfl-llm/meditron-7b and https://huggingface.co/epfl-llm/meditron-70b), what system prompt do you recommend?

2. Given a system prompt, is the following the proper way to input it to the model?

import torch
from transformers import pipeline

model_path = "epfl-llm/meditron-7b"
prompt = "What are the symptoms of diabetes?"
prompt_template = f"system prompt beginning... {prompt} ... system prompt end"

#load model and tokenizer
model = transformers.AutoModelForCausalLM.from_pretrained(
   model_path,
   torch_dtype=torch.float16,
   device_map="auto",
   )
tokenizer = transformers.AutoTokenizer.from_pretrained(model_path)

#tokenize input
prompt_tokens = tokenizer(prompt_template, return_tensors='pt')["input_ids"].to(device)

#generate output
output = model.generate(
   inputs=prompt_tokens,
   temperature=0.1,
   do_sample=True,
   eos_token_id=tokenizer.eos_token_id,
   pad_token_id=tokenizer.pad_token_id,
   max_new_tokens=512
   )

print(tokenizer.decode(output[0]))

Thank you very much!