Peer review is an absolute necessity for recognizing good science and rejecting the false, but the traditional method of journal based peer review is not keeping up with modern needs. The more prestigious journals are more concerned with the potential a paper has to enhance their impact factor. With so many papers to choose from they can happily reject many, not because they are wrong but because they are not sufficiently mainstream to attract quick citations. When they do accept they place the final version behind a paywall and charge the taxpayers who funded it $30 to read each paper. Is this right?
Different areas of science have different needs and the resilience of journal based peer review can in part be attributed to its flexibility. The needs of maths and physics surpass what the journals offer and as a result the peer review process has been largely replaced by internal reviews, submission to open archives. Where work is more theoretical and speculative the journals do little to decide the validity. This is determined by citations, open discussions and further research leading eventually to experimental tests (we hope). But even here the journals have not disappeared. They remain because students and postdocs need the official stamp of approval that the journal offers in order to move to their next job. Can this role be replaced?
The role of open discussion on the web is surprisingly controversial. In a recent post I queried a response to a question put to Brian Cox and Jeff Foreshaw in the Guardian. They were basically saying that blogging about science that is not yet peer-reviewed undermines the system. a littler later there was a similar article in the Guardian itself in which astronomer Sarah Kendrew defends blogging. But Cox and Foreshaw are far from isolated in their opinion. A link from that article leads to an interesting story about a question in a course about “Responsible Conduct of Research”. The question was as follows:
A good alternative to the current peer review process would be web logs (BLOGS) where papers would be posted and reviewed by those who have an interest in the work, true or false?
The correct answer according to the course is false. Lose a point if you thought otherwise. Well it is indeed the case that blogs alone cannot replace the current peer review system, but they are becoming increasingly important in discussing and judging some questions. Could it be possible to construct a system of peer review based on open web-based appraisals that would replace the journals? Nearly a year and a half ago I asked this question and suggested that a system based on something like stack exchange might be possible. It would not be easy and one thing is clear: It would have to be backed by people with more clout and credibility than me.
Happily some people who do have that kind of clout are now starting to think of the same idea. In particular Tim Gowers has been asking similar questions for peer review in mathematics (see here and here) As we have seen above, such a system is likely to be highly controversial as well as difficult to put together effectively, but at least it is starting to be discussed by people who matter. Mathematics is an area where it might work most easily because correctness in mathematics is very cut-and-dried. This is one reason why MathOverflow has been so much more successful than Physics Stack Exchnge. But as I said earlier, journal based peer-review holds its place because it is so flexible. To replace it we need a web-based peer-review system that can work across all disciplines.