You will never know the "generative process" of a paper, which makes it nearly impossible to properly evaluate it, even as an expert reviewer. If you are not a

You will never know the "generative process" of a paper, which makes it nearly impossible to properly evaluate it, even as an expert reviewer. If you are not a reviewer (or working on a competing paper), you are much better off NOT reading the newest papers. Instead, rely on the best proxy metric of all: Time. Wait until the paper has been battle-tested. See if people on social media are talking about it, what critics are saying, wait for reproduction results and open-source implementations, conference acceptances, and citations and comparisons. These give you a much better idea of the validity of the paper than its content can. When reading the newest ML papers, I found it useful to not judge them, but instead use them as inspiration. Forget about the results. The paper may contain interesting ideas or viewpoints you didn't consider before, and those are probably much more valuable than the result table.