It should be unsurprising if comments here, made in public, capable of being challenged are far higher quality than official "peer review" which has really become a bit of a joke. Has been for quite a long time, to be fair. (And unsurprising if not, it's a web forum).
Peer reviewed and published means precisely excrement in a world where papers are not retracted and failures to replicate are refused publication by the "prestigious" "peer-reviewed" journals that published the original.
Intelligent, reasoned, knowledgeable review going back and forth with the author in public makes "peer review" seem just as silly as it usually is. Review by peer professor's indentured servants aka PhD students is often actually higher quality, if wildly dishonest and quite frankly disgraceful.
We now live in a world where LLMs can produce papers. Actual research is a little more interesting, with or without LLM. The journal peer review system doesn't seem to be helping.
[–] hasley 4 days ago
[–] harry8 3 days ago
This is nothing like academic journal peer review, where you can't see the failed submissions for the paper and why. You have no clue if the reviewers were interested in the topic, diligent, drunk or playing politics "support papers from these institutions because they're the good guys, undermine those because they are or aren't from Europe." You have no idea if the paper has since failed to replicate on multiple occasions. You have no idea if you are the first person to actually read the paper and when you have you thoughts, no matter how insightful, well articulated and supported by evidence, those thoughts do not matter to the publication process of that paper, which has been peer reviewed.
Public review by the knowledgeable who care is a very different thing to so called "peer review."
[–] elendilm 4 days ago
It's fairly basic. For a work published on a public platform, the same guys paid by the journal can also review it if they so choose to in addition to those people in the public who were not paid by anyone.
[–] mkremins 3 days ago
[–] elendilm 3 days ago
The logic stands. A reviewer, even those who get zero compensation, can always review publicly published papers just like others.
For AI, the borders seem to have blurred a bit between academic scientists (e.g. at universities) and external scientists at companies or even private people). The problem with the latter is that it is much harder to enforce good scientific practice. Think only about the time it takes to write the state of the art of an ordinary conference paper.
I am also a bit drawn between giving everyone the chance to contribute to research and ensuring good scientific practice. Just writing down your latest ideas and successes is not the same as writing a scientific paper that shows the research in the context of past research and needs, for instance, to be much more mathematically rigorous.