Is RLHF fundamentally broken? Paid labelers rating synthetic scenarios doesn’t seem like real human feedback to me

Every major AI model goes through RLHF — thousands of paid contractors rating AI outputs to teach models what good looks like.

But here’s what bothers me:

These contractors are paid per task — incentivized to finish fast not feel deeply. They’re rating synthetic scenarios not real emotional situations. They burn out after thousands of repetitive evaluations.

The result is AI that passes every benchmark but fails every real human moment.

OpenAI spent $100M+ on this process. And GPT-4 still can’t pass as human in a genuine emotional conversation.

My question for this community:

Is the problem the method — RLHF itself? Or the implementation — who they hire as labelers?

And what would genuinely authentic human feedback even look like at scale?

Genuinely curious what ML practitioners here think.