Saturday, October 8, 2011
Peer-Reviewed Only, Please
This question kind of irritated me. Not because the works weren't reviewed by the researcher's peers but because the question was beside the point. It seems to me that this shouldn't matter. I've been thinking about why someone may ask this question and what might be a response that keeps the conversation moving (my irritation certainly won't).
As I have mentioned in a previous post (Do You Rely On Authorities), we all rely on someone else to give us the majority of knowledge that we have. Since we must rely on other people, it is imperative that we rely on trustworthy people. If someone requests that material to be taken seriously be "peer-reviewed", they may simply be asking if the material is coming from a credible researcher. This is not a bad thing. I have heard of instances when researchers did falsify data and the peer-review process caught it before the information made it to the public. It is a way to check and balance research to ensure that good and fruitful research is made known and bad or deceptive research is not.
Another reason is that someone who does not specialize in the area being discussed may not have the ability to critically analyze the claims. This comes back to our reliance upon authorities. This is not a problem. I do get annoyed when people have the ability, but they are unwilling to. They believe that if a paper (or book) has not be peer-reviewed, they may disregard any evidence contrary to their view that is in it.
There are two concerns with the requirement of peer-reviewed material only: First the accuracy of the data and the soundness of the arguments contained within the material is independent of whether they are reviewed by a peer or not. Second, just because something is peer-reviewed does not mean that the material has accurate data or sound arguments. Of course, peer-review does indicate that it has been submitted to a certain level of critique, so some of the critical evaluation may already be done, but the peers may also not be as critical as one would hope and not look at it as critically as they should. Ironically, it is possible that one may be more suspicious of certain material that is peer-reviewed than not.
Now, I have to admit that I'm one of those who would ask if something is peer-reviewed. However, I normally ask for the purpose of getting the majority consensus of a basic idea. I don't project this result onto the truth of the claims. They must be evaluated independently of what the majority thinks. I don't have a problem with material that is not peer-reviewed (most blogs, including mine aren't). I don't have a problem with peer-reviewed materials. I just want people to be aware that both require the same level of evaluation; neither is above nor beneath the other in this regard.
So, what do I do if someone asks me? Well, first, I would NOT go into this whole spill. That could easily be interpreted as a personal attack and a diversion from the real issue (not that the request itself was not a diversion). If someone demands that what I'm presenting be from a peer-reviewed source, then I give it to them. I don't know if they will seriously consider something peer-reviewed versus not, or if they are being nefarious with the request. Sometimes, a request for peer-review material is an opportunity for me to find something more "official" on a particular subject, and be more prepared next time.I expect this of people I talk with; there is no reason they should expect less of me.