Karl Johanson wrote:

Everyone loves quoting "peer review" as if it's a guarantee of veracity.
It's not. Peer review is basically an editorial process that attempts to
ensure that a paper is written properly, suitable for publication, and that
no obvious mistakes have been made (e.g. a wrong equation, or a poor
experimental method). That's all. The lifeblood of science is the
publication of all views, which means papers are published which may be
correct, or may be incorrect- e.g. cosmologists publishing papers supporting
different models of cosmological evolution.

"Peer reviewed" has become a slogan.

Peer reviewers don't decide whether papers are right or wrong. Neither do
they attempt to verify the study. An example; the infamous "hockey stick"
graph that purported to show runaway global warming. Nobody had checked the
data, nobody had checked which proxy studies were used- in fact the
scientists, lead by the infamous Michael Mann, hadn't even made the data
available, and still haven't released some of it. When asked for it, they
initially said they didn't have it to hand! Neither was the methodology
checked by *anyone*. The paper mentioned "a new statistical method" but that
wasn't explained fully in the paper (something a peer reviewer *should* have
flagged) and when checked by McIntyre and McKitrick, the statistical method
used was shown to be just plain wrong. The paper was worthless. Peer review
failed to flag this up.

Also, a central problem with peer review is its nature; review by peers.
While this ensures that the reviewers are knowledgable in their field, it
also means that if a particular received wisdom permeates that field,
opposing views can easily be censored. In climatology for instance, it's now
well established that a small "cabal" of researchers are writing GW papers
and peer reviewing each others' work. As such, it's no more use than
homeopathists peer reviewing homeopathy research. Since they all believe the
same thing, the system is no use for spotting bunk.

(Having mentioned the Hockey Stick, I'd better defend my mentioning of it.
Greenies like to say "oh that old thing, it's only one little study, even if
it's wrong there's lots of other studies. No biggie." Two points reply to
that-

1) The debunking of the hockey stick casts severe doubts on other
reconstructions. "Independent" reconstructions with similar results are
being churned out by scientists directly associated with the Hockey Team,
using the same or similar proxy studies as data and the same or similar
methodologies. Since they're all peer reviewing each others work, this is
cause for severe concern.

2) Michael Mann, lead author of the Hockey Stick has a powerful position in
climatology. His paper was riddled with bias and errors. This calls him into
severe doubt and calls into severe doubt whether he should be peer reviewing
and editing journals, let alone popping up on TV all over the place. Worst,
he was lead author of the IPCC's last report's section on climate change
reconstructions. This means that a scientist who doesn't know what he's
doing is effectively writing world policy on Global Warming. He included his
own graph in the report 6 times. The Canadian government copied it to every
household in their country. Last I looked it was still up on Wikipedia. It
pops up everywhere. It's a very important graph indeed.)

It could be a lot better. For instance, peer review could be extended such
that, for instance, non-climatologist statisticians review the statistical
methods used. In order to do that of course the scientists would actually
have to make all their data and methodology available and be prepared for
their work to be genuinely scrutinised; something about which they're
strangely reticent.

The bigger problem that most people overlook is this: The scientific method,
and the conventions used within science, are fairly adequate for Science.
But they're inadequate for use in the wider world. For instance a scientific
study which appears to show a weak association between eating carrots and
bowel cancer, in scientific terms, is a morsel of information medical
researchers into bowel cancer may use in their studies. Nowadays, however,
it will be breathlessly announced in a press release that carrots cause
bowel cancer. Governments will be under pressure to reduce carrot
consumption. And so on.

What is needed is a re-evaluation of the relationship between academia and
greater society, because academics have over the past decades manouevered
themselves into a position as "advisors" to government, making them
effectively the bishop at the king's ear. This has prompted them to declare
absolute knowledge; consensus and certainty, where science isn't supposed to
be about that. We need to recognise that science, when it enters the wider
public arena, must be far more heavily scrutinised than it is by peer review
systems developed when scientists were just pottering about in their
laboratories divorced from the political process. A 95%% CI study may well be
perfectly adequate in science, because it's just another factoid to add to
the general pool of knowledge, for scrutiny by other knowledgable
researchers. If it's to be taken as "proof" by greater society, it's
hopelessly inadequate. When politically motivated climatologists are
churning out paper after paper in support of their views, then writing them
into IPCC reports to force government action, far greater scrutiny is
required than mere peer review. The utmost skepticism should be applied, the
data ripped apart by skeptics; and acceptance gained only when the science
has passed through that trial by fire. Only then can we have any confidence
that we're getting a balanced picture.

McIntyre and McKitrick's demolition of the hockey stick (and by association
the rest of the Reconstructionists' models) was published in Geophysical
Research Letters.

True, but not much use. Allusions to well known iconic tales are routinely
used in conversation. Clearly, the point of this allusion is to the theme of
the story; a character who is terrified of something that isn't really
happening because of a lack of understanding of the bigger picture. As I
understand it, anyway, the above quote is wrong. At no point is Chicken
Little hit by a piece of sky. A more mundane object (an acorn) hits him on
the head, which he *presumes* to be the sky. He's jumped to an unsupported
conclusion without considering the alternatives. As such, it's a very good
allusion to the current state of climate "science".

http://www.geocities.com/mjloundy/

Ian