I am working in a project where I have a dataset of model responses tagged with “thumbs up” or “thumbs down” by the user. That’s all the info I have and I cannot pop up new generations to the user, I have to make use only of the dataset.

Is there any literature on the best ways to evaluate the model who generated those responses and/or fine tune the model?

The most obvious thing I can think of is calculating the % of responses that got thumbs up for performance, and for fine tuning training a reward model on the dataset I have and later applying RLHF to the model.

Is there any publication exploring some better ways of doing that?