Saturday, October 8, 2011

Peer-Reviewed Only, Please

Not too long ago I was talking with a college student about scientific theories and whether they accurately reflected reality. As is to be expected on a topic like this, we discussed evolution. Specifically the natualistic kind. He provided me some reasons why he believed that evolution explained the diversity of life that we have today. I disagreed and proposed some counter-evidence from some researchers. He became immediately defensive and asked if the works were peer-reviewed.

This question kind of irritated me. Not because the works weren't reviewed by the researcher's peers but because the question was beside the point. It seems to me that this shouldn't matter. I've been thinking about why someone may ask this question and what might be a response that keeps the conversation moving (my irritation certainly won't).


As I have mentioned in a previous post (Do You Rely On Authorities), we all rely on someone else to give us the majority of knowledge that we have. Since we must rely on other people, it is imperative that we rely on trustworthy people. If someone requests that material to be taken seriously be "peer-reviewed", they may simply be asking if the material is coming from a credible researcher. This is not a bad thing. I have heard of instances when researchers did falsify data and the peer-review process caught it before the information made it to the public. It is a way to check and balance research to ensure that good and fruitful research is made known and bad or deceptive research is not.

Another reason is that someone who does not specialize in the area being discussed may not have the ability to critically analyze the claims. This comes back to our reliance upon authorities. This is not a problem. I do get annoyed when people have the ability, but they are unwilling to. They believe that if a paper (or book) has not be peer-reviewed, they may disregard any evidence contrary to their view that is in it.

There are two concerns with the requirement of peer-reviewed material only: First the accuracy of the data and the soundness of the arguments contained within the material is independent of whether they are reviewed by a peer or not. Second, just because something is peer-reviewed does not mean that the material has accurate data or sound arguments. Of course, peer-review does indicate that it has been submitted to a certain level of critique, so some of the critical evaluation may already be done, but the peers may also not be as critical as one would hope and not look at it as critically as they should. Ironically, it is possible that one may be more suspicious of certain material that is peer-reviewed than not.

Now, I have to admit that I'm one of those who would ask if something is peer-reviewed. However, I normally ask for the purpose of getting the majority consensus of a basic idea. I don't project this result onto the truth of the claims. They must be evaluated independently of what the majority thinks. I don't have a problem with material that is not peer-reviewed (most blogs, including mine aren't). I don't have a problem with peer-reviewed materials. I just want people to be aware that both require the same level of evaluation; neither is above nor beneath the other in this regard.

So, what do I do if someone asks me? Well, first, I would NOT go into this whole spill. That could easily be interpreted as a personal attack and a diversion from the real issue (not that the request itself was not a diversion). If someone demands that what I'm presenting be from a peer-reviewed source, then I give it to them. I don't know if they will seriously consider something peer-reviewed versus not, or if they are being nefarious with the request. Sometimes, a request for peer-review material is an opportunity for me to find something more "official" on a particular subject, and be more prepared next time.I expect this of people I talk with; there is no reason they should expect less of me. 

2 comments:

  1. IMO, they ask that because they themselves are rather uninformed on the issue and are most likely simply regurgitating things that they have heard but haven't done the necessary intellectual leg work to make the knowledge their own.

    So you have presented them with research that they can't handle and as such are trying to worm they way out from under it by asking for an even smaller slice of the evidential pie in hopes that they won't have to answer to the evidence that you've presented.

    Someone knowledgeable enough and confident enough in their understanding of the issue wouldn't care if it was peer reviewed or not. To them it would either be damming or irrelevant and they should be able to handle either.

    As for peer reviewed itself, its not the bastion of intellectual propriety that it may have once been and many claim that it is.

    http://www.economist.com/node/21528593

    ReplyDelete
  2. Here's an interesting article concerning the credibility of peer reviewed "scientific" findings:

    ReplyDelete

****Please read my UPDATED post Comments Now Open before posting a comment.****