User talk:Anachronist/Reliable sources (university presses)

Comparison to peer review by journals

@Anachronist: Thanks for this, some really good points. I agree with most of it and I'd support putting it back in mainspace.

Regarding the comparison of press peer review to journal peer review, I don't think this is quite accurate:

With a scholarly journal, peer reviewers and authors don't know one another's identities. This bidirectional anonymity typically doesn't exist during a book's peer-review; the reviewers often know the identity of the author, which can result in a favorable bias toward a reviewer's fellow colleague.

The first line describes double blind peer review which is one model used by journals but by no means universal. I don't know the statistics but I've certainly done my share of single blind reviews (where I know the author's name but they don't) and open peer review is increasingly common.

In general, I wonder how much of what's in the peer review section are specific to books, as opposed to just being weaknesses with scholarly peer review as a whole – IMO Wikipedians put entirely too much faith in it, but that's another discussion. – Joe (talk) 16:07, 17 June 2024 (UTC)

Hmm. The point I was trying to make with this may be a bit muddy. A journal generally has peer reviewers from all over the place, whereas a university press draws its reviewers from its smaller network of universities. So if I submit a journal article for single-blind review, the peer reviewer may see my name, but would be extremely unlikely to know who I am. If I submit a book to a university press, the reviewer not only would see my name, but may be familiar with my work, or even know me personally — especially if the university press uses reviewers recommended by the author of the book being reviewed.

Can you suggest a way to revise it? ~Anachronist (talk) 18:56, 17 June 2024 (UTC)

Probably this varies from field to field, but if I'm asked to review something, it's extremely unlikely that I won't know who they are. I don't think that takes away from your point, but I'd perhaps phrase it as weaknesses of peer review (Scholarly peer review#Criticism is surely long enough that it could be its own article by now) rather than . On the other hand, the point you make about reviewers only reading part of the work is specific to books. I can make some edits directly, if you don't mind? – Joe (talk) 08:26, 18 June 2024 (UTC)

No, I don't mind at all. I never considered myself the "owner" of this, and when I first wrote it, some other editors who had advised me, I invited to contribute, but they didn't. One of those editors told me about his experience with a university press using peer reviewers recommended by the author, so I put it in. ~Anachronist (talk) 01:30, 19 June 2024 (UTC)

This should be in project space, probably as (part of) a guideline

While the essay could surely use a re-write so as to be clearer and more on-point, I wholeheartedly agree with its message, which I not only think is a very important one, but one that should be obvious to any experienced Wikipedian and that should therefore have community consensus.

I find it highly concerning that this was moved out of project space as an "extreme minority opinion". Anyone who has read a dozen or more books published by university presses knows that all respectable academic publishers do publish stinking rubbish from time to time, ranging from simply bad scholarship to highly original and well-researched theses which are nevertheless irrevocably refuted and widely rejected. The naive attitude of Wikipedians who never read anything about the subject but simply argue 'it's Oxford University Press, so it must be reliable' is highly damaging and will ensure that fringe scholarship is promoted on Wikipedia.

The publisher is only an indication: reliability must ultimately be judged based on reception by other scholars, and whenever that real mark of reliability is clear, the publisher is in fact of no consequence at all. A self-published paper which is cited approvingly by everyone in the field is reliable; a Cambridge University Press monograph which is universally ignored or rejected is not. Wikipedians should know this, and it should be a guideline. ☿ Apaugasma (talk ☉) 18:06, 27 June 2024 (UTC)