[D] Papers with no code

I can’t believe the amount of papers in major conferences that are accepted without providing any code or evidence to back up their claims. A lot of these papers claim to train huge models and present SOTA performance in the results section/tables but provide no way for anyone to try the model out themselves. Since the models are so expensive/labor intensive to train from scratch, there is no way for anyone to check whether: (1) the results are entirely fabricated; (2) they trained on the test data or (3) there is some other evaluation error in the methodology.

Worse yet is when they provide a link to the code in the text and Openreview page that leads to an inexistent or empty GH repo. For example, this paper presents a method to generate protein MSAs using RAG at orders magnitude the speed of traditional software; something that would be insanely useful to thousands of BioML researchers. However, while they provide a link to a GH repo, it’s completely empty and the authors haven’t responded to a single issue or provide a timeline of when they’ll release the code.