[D] What is even the point of these LLM benchmarking papers?

Lately, NeurIPS and ICLR are flooded with these LLM benchmarking papers. All they do is take a problem X and benchmark a bunch of propriety LLMs on this problem. My main question is these proprietary LLMs are updated almost every month. The previous models are deprecated and are sometimes no longer available. By the time these papers are published, the models they benchmark on are already dead.

So, what is the point of such papers? Are these big tech companies actually using the results from these papers to improve their models?