‘Tipping Points’: Does the Opinion of Experts Reflect Reality?

By Chip Knappenberger -- July 6, 2010

Last week, an advance copy of a paper to appear in the Proceedings of the National Academy of Sciences (PNAS) was released which reported that a collection of “experts” suggests that climate tipping points (codename for something bad but we don’t know exactly what) would be knocked over by 2200 if we stay on our current greenhouse gas emissions pathway (for about the next 200 years). Underlying these views is the experts’ opinions as to what the earth’s equilibrium climate sensitivity—the rise in global temperatures resulting from a doubling of the earth carbon dioxide concentration—likely is.

But do the experts opinions actually reflect the scientific knowledge on these subjects?

The answer is no.

In fact, the experts’ opinions tended towards the extreme, despite recent science which should have reeled them in. Which is a lesson in and of itself.

How much confidence would you place in expert assessments such as the IPCC reports, when the experts themselves disregard some science in favor of others (namely the science that they themselves have been involved in), despite any good reasons to do so?

Not much would be my answer.

A prime example of experts disregarding scientific knowledge is found in the PNAS survey conducted by Kirsten Zickfeld and colleagues.

Zickfeld et al. interviewed 14 experts about what they thought the climate future holds in store under various scenarios of atmospheric carbon dioxide concentrations. The experts’ responses to the scenario in which atmospheric CO2 levels reach about 1000 parts per million (ppm) by the year 2200 is the one which got the most attention—predictably so, why pay any attention to outcomes which aren’t dramatic.

In fact, as evidence that the experts were themselves prone to the dramatic, Dr. Myles Allen (one of the experts interviewed), was quoted in the UK’s The Independent, that “We are certainly capable of committing ourselves to an emissions trajectory that make 1,000 ppm in 2200 almost inevitable if we make the wrong decisions over the next 20 years.”

This seems a little short-sighted and moralistic, but hey, these guys are experts.

And 13 out of 14 stated that there was a greater than 50% likihood that some sort of climate tipping point would occur by 2200 under the 1000ppm CO2 scenario while 9 of them put the probability as over 80%.

Undoubtedly, a leading factor in these beliefs comes down to the experts’ opinions as to what is the true value of the earth’s climate sensitivity. It turns out, that this has been a hard value to pin down—largely because uncertainties abound in both our understanding of the observed climate as well as how it got that way.

Zickfeld et al. report that the experts consider the leading cause of uncertainty about the behavior of the climate system to lie with cloud processes. Others causes consistently listed near the top were feedbacks involving ice and snow, water vapor feedback, and large-scale ocean circulation.

It is interesting to note that Zickfeld found that the experts had little confidence that scientific research would reduce the impacts of these uncertainties by half even if research budgets were increased by 3 times current levels.

Considering all that we don’t know, the Intergovernmental Panel on Climate Change (IPCC) can only state that the “likely” range (a confidence of between 66 and 90%) of the true value of the climate sensitivity lies between 2°C and 4.5°C, which, as Zickfeld explains, means that the IPCC thinks that there is just a 5% to 17% chance that the climate sensitivity is greater than 4.5°C.

But what to the experts think about the range of climate sensitivity? Figure 1 shows their expert opinion.

Figure 1. Range of values which are thought to contain the true value of the earth’s climate sensitivity, as given by 14 experts. The tick marks on the whiskers bound the 90% range, the boxes bound the 50% range and the small, thick horizonal mark reprents the median value of the distribution. The values in parentheses under the number of each reviewer is the probability that the climate sensitivity is greater than 4.5°C. (the grayed out boxes associated with some experts are how those experts responded to the question in a similar survey more than 10 years ago).

Now, before I go much further, let me discuss briefly who Zickfeld’s experts are. Probably a good half of them have been directly involved in research projects which have determined that very high climate sensitivity values were a distinct possibility, and exactly none of them were involved in research projects (such as those being carried out by Richard Lindzen or Roy Spencer) which suggest that the climate sensitivity is at (or below) the low end of the IPCC range.

So, the following results will hardly be surprising.

Zickfeld et al. report:

The IPCC report assesses that the “equilibrium climate sensitivity is likely to lie in the range 2– 4.5°C, with a most likely value of about 3°C.” IPCC defines likely as a 0.66–0.90 probability, which in Chapter 19, Working Group II (WGII) is interpreted as a 0.05–0.17 probability that climate sensitivity is >4.5°C. Examining the elicited distributions obtained from our experts, we find that 10 of the 14 experts placed >0.17 of their probability above 4.5 °C. [emphasis added]

So 10 of the 14 experts think the chance of a high sensitivity value is greater than that laid out in the IPCC. As a note, the other 4 experts listed the probability at 7%, 10%, 12% and 15%.

Even more enlightening is that 9 of the 14 experts consider a climate sensitivity of greater than 6°C as having a not insignificant 5% chance of being the true value.

What is so incredible about these numbers is that there is solidly argued scientific research (appearing in the peer-reviewed literature) that constrains the climate sensitivity range to a much greater degree than these experts do.

Most notable is a string of papers published by James Annan, a climate researcher at the Research Institute for Global Change of the Japan Agency for Marine-Earth Science and Technology and his colleagues. James argues that his research pretty strongly suggests that the upper end of the range of potential climate sensitivity is much more constrained than the experts realize (or admit to).

In his most recent paper on the subject, James concludes:

Based on some conservative assumptions regarding the value of independent estimates, we conclude that climate sensitivity is very unlikely (<5% probability) to exceed 4.5°C. We cannot assign a significant probability to climate sensitivity exceeding 6°C without making what appear to be wholly unrealistic exaggerations about the uncertainties involved. This represents a significant lowering of the previously-estimated bound.

Apparently, most of Zickfeld’s experts weren’t above “making wholly unrealistic exaggerations about the uncertainties involved” as 9 of them included a probability of 5% or more that the true climate sensitivity was above 6°C (and not a one had a probability as low as 5% for the climate sensitivity being above 4.5°C).

Clearly, the experts did not think too highly of James’ analysis.

All of this has left James shaking his head (from his review of the Zickfeld article):

So even though as far as I can tell everyone accepts that the fundamental points we make in our two papers are valid, they still stick to these old discredited results with long tails to high values. – in fact the answers are more alarmist than 15 years ago. Makes us wonder why we bother…

Read more of why James is disheartened at his blog post “Oh noes we’re all going die…by 2200.”

All of this shines interesting light on the latest entry over at the blog RealClimate—a prayer of sorts, that Gavin Schmidt picked up at a recent climate meeting which he offers for discussion:

Grant us…
The ability to reduce the uncertainties we can;
The willingness to work with the uncertainties we cannot;
And the scientific knowledge to know the difference.

The way I see it, there is real work being done to reduce the uncertainties on the estimate of the climate sensitivity, but the experts would rather not recognize these results and instead, they prefer to treat climate sensitivity as a type of uncertainty that can’t be known, because this allows them to better entertain the high end leading to future catastrophe and tipping points.

Which gets to the bottom of the bigger issue—that is, what is wrong with Assessment reports such as those from the IPCC.

Simply put, they reflect the opinion of experts rather than a comprehensive, or even unbiased, review of the scientific literature. The assumption that expert assessments draw upon the best scientific knowledge is a false one—what experts do drawn upon are their own ideas, some borne by their own work, and even if subsequent work seriously calls their own work into question, they stick to their old ideas. This is on display here, and also throughout the Climategate emails.

References:

Annan, J. D., and J. C. Hargreaves, 2006. Using multiple observationally-based constraints to estimate climate sensitivity. Geophysical Research Letters, 33, L06704.

Zickfeld, K., et al., 2010. Expert judgments about transient climate response to alternative future trajectories of radiative forcing. Proceedings of the National Academy of Sciences, doi:10.1073/pnas.0908906107.