Bishop Hill What every politician should know about climate models
Sep 8, 2014 Climate: Models Climate: Parliament Last week Richard Betts of the Met Office got a bit grumpy with me over my comments about Keith Shine FRS. In discussing remarks Shine had made to parliamentarians about climate models I had said that I felt it was grossly misleading of him to restrict his remarks about their reliability to a reference to "...state-of-the-art climate models, which are our embodiment of the laws of physics as applied to the atmosphere..." Richard felt this was unfair, noting Shine's high integrity.
Shine has certainly never come to my attention before as one of the "bad guys" so I am happy to accept Richard's assurances on this point. Nevertheless, I stand by my comments. What Shine told his audience about GCMs gave a thoroughly misleading impression of how reliable GCMs are. This is perhaps understandable as Shine speaking as part of a panel of prominent scientific peers, all of whom were keen to get the message across to the parliamentarians, all of whom were keen to hear a message of alarm (this was the All Party Parliamentary Climate Change Group, after all). In such an atmosphere it's easy to overstep the mark.
What, then, should Shine have said about the climate models? There has been some interesting discussion of this point in the comments underneath the posts about what climate policy and the extent to which it relies on GCMs. Jonathan Jones made a valiant attempt to paraphrase Richard's position and in doing so put together this paragraph on the climate models:
An alternative approach [to looking at temperature records] is to try to work out how much warming might occur from first principles. We try to do this with our big computer models called GCMs. Unfortunately the problem is very difficult, and while we think we might be getting close to cracking it we're not there yet. Our best guess is that the warming will be larger than simple calculations suggest, but frankly that's just a best guess.
I think a few readers will be surprised to learn that Richard seemed to have no great objections to this summary, although he preferred the use of "best estimate" to "best guess". However, I would say that if it is recognised that the problem is not yet "cracked", then the word "guesses" seems to convey roughly the correct impression. I don't think you are in the realms of estimates until you know that every relevant process is correctly captured in the model. Given that the failure of the GCMs to predict the pause has been blamed on deep-ocean heat transport, I think we can safely conclude that this process is not correctly captured. No doubt there are others.
What else should politicians know about climate models? It would be fair to say that they reproduce the main features of the global climate - tropical storms, monsoons and the like. It would also be fair to say that they tend to reproduce the temperature anomalies of the past, but that is only achieved with a measure of "fudge". It is also important to communicate that they fail to reproduce the absolute temperatures of the past, and to note that to the extent that recent warming has been natural, the climate models may be being fudged in line with the wrong thing. The failure to predict the hiatus and the deep-ocean heat transport that is alleged to be behind it should be mentioned too.
This is quite a lot to convey, so one can forgive Keith Shine for not going into this level of detail in a five-minute talk. But as a shorthand, I would suggest that "best guess" is the way forward.
GCMs may well be state of the art. They may well be amazing scientific achievements too, in the way that Leonardo da Vinci's design for a hang glider was. But in the same way they are not ready for prime time yet. Politicians need to know that these are works in progress, mere guesses about what the future might hold. They do not give the answers that the politicians want.
Reader Comments (48)
Very well said Bish. That interaction between Betts and JJ on Sunday 31st and Monday 1st is well worth returning to. Two tweet streams where Richard 'got a bit grumpy' with you on Wednesday (that I saw, anyway) are here and here. Not untypically Richard finishes the last one awarding bad marks to Bob Ward as well. We've had worse enemies.
[BH adds: I wouldn't describe Richard B as an enemy]
When predictions of future events were made by examining the entrails of a goat, the practitioners were always able to explain the basic validity of the method and give reasons why particular predictions did not match emerging reality
I find it a bit rich that climate "scientists" talk about integrity whilst they stood by silently:
As Al Gore misled people
As Mann created the Hocky stick
As Climategate showed people behaving disgracefully
As prediction (or projection) after prediction hit the dust
As many said the science was clear and "settled"
As billions of public money flushed down the toilet under the title of "research"
As a suicidal energy strategy was formed using poor modelling
Watching other people in climate "science" do bad things and say nothing (or wishy washy things) is not my idea of integrity!
Remember people like Julia Slingo and Vicky Pope have made many alarmist statements which were later proven to be wrong or unsubstantiated.
This applies even more to regional climate models, that everybody in the science community agrees have no known skills and yet are being financed and pushed hard no doubt in order to provide politicians with more tools to push on the recalcitrant public.
This study is still highly relevant. It deals, amongst other things, with the question what modellers themselves think about the reliability of their models.
http://www.kepplinger.de/files/Die_Klimaforscher_sind_sich_laengst_nicht_sicher_0.pdf
Unfortunately only in German. But with Google Translator the English reader may get the gist of it.
Hans Labohm
Robert Brown of Duke has the best take on the worth(lessness) of GCMs.
Integrity and empiricism versus, computer modellers?
No contest..............um - except in climate science.
It seems that modelling clouds is currently beyond the capabilities of GCM's. We are told that the level of scientific understanding of clouds is low. There is evidence that they respond to heat on a timescale of minutes in the tropics, and there is evidence that they are the true regulator of surface temperatures
http://s1114.photobucket.com/user/Chief_Hydrologist/media/cloud_palleandLaken2013_zps73c516f9.png.html?sort=3&o=16
This is a known unknown. The unknown unknowns will be more of a challenge.
[BH adds: rainfall is pretty much beyond the ability of GCMs too]
Charming: For me calling their critics deniers is worse than all of these put together. But we mustn't use any of that dreck as an excuse to neglect our own responsibilities. For instance, on Twitter on Wednesday:
I respected JJ for that. Whiter than white isn't just a nice choice of garb for those of us who choose to be public critics of the 'consensus without an object' as Ben Pile calls it. Much more pragmatically it's the only way we'll overcome when so much political power is arrayed against us.
I'd 'prefer wishful' thinking to 'best guess'. As Matt Ridley recently pointed out in the WSJ, the hiatus suggests that natural variations have been underestimated and the contribution of CO2 likely overestimated. The honest best guess based on recent trends would be that warming will be lower than simple calculations suggest.
The mere existence of a phenomenon such as "Climate Policy" should be enough to tell us there is something fundamentally rotten in the state of Denmark. When elephants start tripping over ants in the road it will perhaps be time for governments to start worrying about controlling the climate of the planet. Such is mankind's contribution to the temperature of the planet. Until then it would be wise to assume climate cannot possibly come within the remit of politicians. It could certainly be argued that if we accept we must adjust to changes in climate (rather than seek to control them) there is a case for knowing what is to come. The problem with that is the predictive powers of establishment funded scientists demonstrably boil down merely to telling politicians en masse what they wish to hear, i.e. the very opposite of what Mother Nature insists on doing, in order unsurprisingly to put bread on the table. Maybe it will always be thus.
It is a very easy problem, but they don't seem capable to understand it. You may have "the best knowledge available", but "best" doesn't necessarily mean "good enough" for the purpose.
Change the words with "state of the art", or whatever, it doesn't change the problem.
I'd go with "best guess" and that's the view of someone who has made a living using the English language!
I can see Richard's point about "best estimate" — as a scientist he hates the idea of guessing in his profession, and rightly so. But remember that this particular communication we are talking about is with laymen who will understand the phrase "best guess" as a little different from "best estimate". The latter phrase would still be giving a stronger message to a layman than is justified by the state of the science and the ability of the models.
I am also extremely unhappy with the phrase "state of the art" in the same context. To a layman "state of the art" is the TV remote that changes channels when you think rather than when you press the button; it implies something really, really up to the minute. The models may well be the best that they can be at the moment; to call them state of the art sends out a false signal into the mind of the lay listener.
Of course, words will always be used in a way that gets a message across and we know that there are plenty of people, scientists among them, who are more concerned with the message regardless of how strong the facts are to justify it. All the more reason why the honest ones should make sure that the impression they are conveying is justified by the facts.
Can anybody list three outputs of GCMs which are useful and reliable for policy work?
If not three, then how about one? Along with an explanation and justification of why it it is useful and reliable?
Input from Keith Shine or Richard Betts (or even Julia Slingo?) would be particularly welcome.
@ Martin A. (or anyone who knows)...
Is there a link to that 2008 publication available..?
"Our best guess is that the warming will be larger than simple calculations suggest, but frankly that's just a best guess."
An oxymoron.
A guess, by definition, admits to unknowns.
If they're unknowns, who knows whether the guesses are 'optimistic' or 'pessimistic'?
I'd go with 'worst guess'.
I think the correct word would be guesstimate, "an estimate based on a mixture of guesswork and calculation"
We have a new Lysenkoism, to assume 'back radiation' is a real energy flux instead of the atmospheric IR emittance, allied to a new Phlogiston, the idea that surface IR emittance, near black body level, is also real. A third mistake is to assume the Earth emits IR energy to Space from a single upper atmosphere region which does not exist. The fourth mistake is hindcasting with ~40% extra low level cloud albedo to get temperature right. Otherwise, the GCMs are fine.
@ Martin A. (or anyone who knows)...
Is there a link to that 2008 publication available..?
Sep 8, 2014 at 10:28 AM | Salamano
www.metoffice.gov.uk/media/pdf/b
...and what about the somewhat interesting point that 97% of these "best guesses/estimates" have grossly exaggerated the amount of warming that we have actually seen over the last 20 years??
Rising Discomfort
• Predictions can’t be rigorously evaluated for order of a century
• Insufficient exploration of model & simulation uncertainty
• Impenetrability of the model and formulation process; extremely large number of modeler degrees of freedom
• Lack of formal model verification & validation, which is the norm for engineering and regulatory science
• Circularity in arguments validating climate models against observations, owing to tuning & prescribed boundary conditions
• Concerns about fundamental lack of predictability in a complex nonlinear system characterized by spatio-temporal chaos with changing boundary conditions
• Concerns about the epistemology of models of open, complex systems
Judith Curry (slide from presentation 2012)
"...state-of-the-art climate models, which are our embodiment of the laws of physics as applied to the atmosphere..."
In my opinion, this statement is designed specifically to speak to the ininformed, It sounds very impressive, which
of course is the intention and that is why it is misleading.
I am not even sure what "OUR embodiment of the laws of physics as applied to the atmosphere" means.
We all have our own primitive GCM software running on homo sapiens v1 cerebral hardware
The most recent output of my own GCM is 'no warming for the next 5 years'. You may call it a guess or an estimate.
Mir ist es scheißegal.
I wonder what sort of artist design the "state-of-the-art climate models"?
Bish
I read the discussion between Jonathan and Richard. I thought it was great. They spoke in a simple but yet precise language. It'd be good to have a more formal post; perhaps you could have a post which presents their entire discussion. Then such a post could be used as a reference.
I think Richard Betts should be commended. He took timeout in order to communicate with skeptics and then discuss, without any condescension, the merits of climate science.
"They do not give the answers that the politicians want."
The cynic might say that they give exactly the answers the politicians want.
Shine has certainly never come to my attention before as one of the "bad guys" so I am happy to accept Richard's assurances on this point.
Why acceptanca ? When you have enough proof that your acceptance is flawed.
You must know that Betts' job is predicated on GCMs. If they are proven to be fatally flawed Betts' job will be at risk. That's why he did not like your analysis of Shine's deliberate obvuscation.
"...state-of-the-art climate models, which are our embodiment of the laws of physics as applied to the atmosphere..."
1. 'cept for those pesky clouds
2. 'cept for that pesky rainfall
3. 'cept for that ocean heat content thingy
4. with our magic CO2 triple forcing positive feedback assumptions thrown in (though as far as we know the sign could be reversed)
5. with our other brilliant positive feedback assumptions thrown in
6. with little or no negative feed backs assumed
Ok, let's just call it our self-serving Climate SWAG . . .
Oh, and don't forget, we're all gonna die . . .
In science papers unrelated to global warming, I stop reading or at least start to lose interest when the authors use the phrase "state of the art". It's generally a give-away as a marker for BS.
Stephen Richards:
I don't think his job will be at risk if the GCMs are no longer considered fit to inform policy as they have in the past, as Andrew and Richard are both it seems now arguing. But it's a point of view. You see inside the human heart much better than I do.
michael hart - thank for another to add to the list
robust
rigorous
transparent
open
peer reviewed
tipping point
consistent with
state of the art
The exchanges between Jones and Betts were interesting and I want to give them a second look soon. But whereas it is likely that simple methods for computing sensitivity will either under or over estimate it, I would like to see more discussions of the case for either. My impression is that 'over' is more plausible than 'under' but I'd appreciate learning more of contrary views. First, perhaps, we need to agree what those simple methods are.
Thanks Martin.
Regarding the term "consistent with", I recall that I have used the phrase myself in a published paper. It was to indicate that a molecular modelling program gave results that were in line with my pre-determined goals and expectations..
I chose the phrase quite carefully and deliberately. It was not "evidence" as I understand the term. The evidence came later, after a lot of lab work synthesizing the molecules. But the modelling results were presented (more like “mentioned”, actually), at the same time as the evidence.
In an industrial setting, I have also been advised to use the term "consistent with" when an analytical method (NMR spectroscopy) gave some evidence that was expected, but did not provide all the evidence that was expected. At that point it starts to become a judgement call and one needs to ask “what is the most important evidence” and how much further should one go before stopping and cutting your losses.
IMO the old expression "absence of evidence is not evidence of absence" probably sums up the best defence of cAGW proponents at the moment, and some of them now wish to redefine what the most important evidence is.
Richard's talented enough to survive GCMs. And young enough to see better ones.
===============================
Under the conditions stated, the word 'guess' is also incorrect.
Their 'estimates or guesses' are perhaps better described as 'preferred assumptions without error bars'.
To paraphrase Donald Rumsfeld....
When there are known knowns you can estimate.
When there are known unknowns, you might be able to estimate, it might be a guess.
When there are unknown unknowns you are guessing.
I have no idea of precisely when the phrase 'state of the art' was coined, but to me it smacks of something that was invented by a very imaginative advertising copywriter who understood that the public loves technology without much real understanding of what technology is.
The phrase always appears when dodgy goods are being placed on offer, such as the very real instances of Head Teachers promoting their schools by using the number of computers their school has spread about its classrooms as an indicator of the quality of the education on offer.
All these years, all that money, all those reports full of charts and graphics but none of it useful and reliable for policy purposes?
Really?
You only have to look at the UKMO North Atlantic tropical cyclone seasonal forecast record to see how badly things can go wrong in a very short time. In 2009 they were congratulating themselves on how well their forecasts had turned out for the previous 3 years. It's all gone badly awry since despite giving themselves massive ranges.
http://www.metoffice.gov.uk/weather/tropicalcyclone/seasonal/northatlantic2013
The range for ACE in the above quote goes from more or less a record low to well-above average and this is classed as 'considerable skill'. The number of storms turned out to be 70% above the forecast; again 'considerable skill'.
These models may be of academic interest but exposing them to the public makes fools of their authors.
The term 'state of the art' is quite commonly used by people who want to write a paper for a conference they fancy going to but have nothing new to offer from their own work. To get around this inconvenience, they review other peoples work and its application and then write a paper along the lines of 'State of the Art in Metagastrologicanomics'. As many conferences are desperate for both papers and delegates, especially those organised by professional conference promoters, the chance of acceptance is high (as long as every other abstract received does not include 'state of the art' in the title), so it's "exotic location here I come".
The WMO out in force for the Sep 23 NYC climate meeting, with a series of "weather reports from 2050". This is complete rubbish of course, due to the Pooh Sticks phenomenon ((c)Richard Betts) but I doubt there will be as much as a tweet from The Man explaining it clearly to the world.
I still think of Keith as more of a ozone hole person then a global warming modeller. There must be quite a few 25 old model predictions from working on that. How accurate have they been?
Our politicians cannot hope to understand GCM's as thier PPE degrees do not equip them with anything like the mathematical background required.
As anyone with a hint of non-linear fluid dynamics will know - solving the thermodynamics across fluids in motion is currently IMPOSSIBLE - the best numerical methods can offer is an estimate of the Navier-Stokes equations - the quality of which is generally very dependent upon 2 criteria:
Firstly spacial resolution - i.e. very fine meshing particularly in areas of turbulent flow and across boundary layers
Secondly temporal resolution (integration time steps) - required for robust numerical convergence without significant error
In everyday CFD usage we use spacial resolution down to milimeters and time steps of mili-seconds to give a hope of good correlation with measurements of reality. In practice simulations are kept short (< 1min) to avoid numerical 'drift' in the integration process causing the accumulation of error's - which cannot be avoided...
Perhaps our politicians should be asking how GCM's provide reasonable estimates of the N-S equations without using small spacial and temporal resolutions - and why this magic has not been adopted by industry?
Keith Shine was at the Royal Society meeting reviewing the AR5 report.
He is very smart and knows radiative physics of the atmosphere backwards . So yes - unfair to critisise him. Howeve, in the future he does assume carbon sinks will saturate thereby rising CO2 forcing above logarithmic forcing. Yet there is as yet no evidence this is happening.
Ok - Last bump on this before I conclude that, out of the Bishop's wide and informed readership, nobody can name a single output from GCMs which is useful and reliable for policy making.
Sep 10, 2014 at 10:24 AM | Unregistered CommenterASH
One point related to the numerical methods to solve the Navier Stokes equations is are they proven to conserve energy. It would be irrelevant for a 5 day weather forecast but could be a significant factor in a 100 year run, hence things like use of a 'Flux Correction' in the earlier days when the tropics got too hot and the poles too cold, so literally they were fudged with a large hard coded energy movement polewards. Other issues from a modelling point of view is that the solution techniques are inherently unstable so damping has to be added too to deal with this. None of this even touches any of the science.