This is a guest post by Doug Keenan.
It has been widely claimed that the increase in global temperatures since the late 1800s is too large to be reasonably attributed to natural random variation. Moreover, that claim is arguably the biggest reason for concern about global warming. The basis for the claim has recently been discussed in the UK Parliament. It turns out that the claim has no basis, and scientists at the Met Office have been trying to cover that up.
The Parliamentary Question that started this was put by Lord Donoughue on 8 November 2012. The Question is as follows.
To ask Her Majesty’s Government … whether they consider a rise in global temperature of 0.8 degrees Celsius since 1880 to be significant. [HL3050]
The Answer claimed that “the temperature rise since about 1880 is statistically significant”. This means that the temperature rise could not be reasonably attributed to natural random variation — i.e. global warming is real.
In statistics, significance can only be determined via a statistical model. As a simple example, suppose that we toss a coin 10 times and get heads each time. Here are two possible explanations.
(Other explanations are possible, of course.)
Intuitively, getting heads 10 out of 10 times is very implausible. If we have only those two explanations to consider, and have no other information, then we would conclude that Explanation 1 is far more likely than Explanation 2.
A statistician would call each explanation a “statistical model” (roughly). Using statistics, it could then be shown that Explanation 1 is about a thousand times more likely than Explanation 2; that is, statistical analysis allows us to quantify how much more likely one explanation (model) is than the other. In strict statistical terminology, the conclusion would be stated like this: “the relative likelihood of Model 2 with respect to Model 1 is 0.001”.
A proper Answer to the above Parliamentary Question must not only state Yes or No, it must also specify what statistical model was used to determine significance. The Answer does indeed specify a statistical model, at least to some extent. It states that they used a “linear trend” and that the “statistical model used allows for persistence in departures using an autoregressive process”.
If you are unfamiliar with trending autoregressive processes, that does not matter here. What is important is that HM Government recognized, in its Answer, that some statistical model must be specified. There is, however, still something missing: is their choice of statistical model reasonable? Might there be other, more likely, statistical models?
(There is also a minor ambiguity in the Answer, because there many types of autoregressive processes. The ambiguity is effectively resolved in a related Question, from 3 December 2012, which discussed “autoregressive (AR1) processes” [HL3706]; other Answers, discussed below, confirmed that the process was of the first order.)
I found out about the Question (HL3050) put by Lord Donoughue via the Bishop Hill post “Parliamentarians do statistical significance”. I then discussed the choice of statistical model with Lord Donoughue. I pointed out that there were other models that had a far greater likelihood than the trending autoregressive model used by the Answer. In other words, the basis for the Answer to the Question was untenable.
Moreover, I had published an op-ed piece discussing this, and related issues, in the Wall Street Journal, on 5 April 2011. The op-ed piece includes a technical supplement, which describes one other statistical model in particular: a driftless ARIMA(3,1,0) model (again, unfamiliarity with the model does not matter here). The supplement demonstrates that the likelihood of the driftless model is about 1000 times that of the trending autoregressive model. Thus the model used by HM Government should be rejected, in favor of the driftless model. With the driftless model, however, the rise in temperatures since 1880 is not significant. In other words, the correct Answer to the Question (HL3050) might be No.
Lord Donoughue then tabled a Parliamentary Question asking HM Government for their assessment of the likelihood of the trending autoregressive model relative to the driftless model. HM Government did not answer. Lord Donoughue asked a second time. They did not answer. He asked a third time. Again they did not answer. He then asked a fourth time.
A Parliamentary Question that has been tabled in the House of Lords is formally answered by HM Government as a whole. In practice, HM Government assigns the Question to a relevant ministry or department. In our case, the Questions have been assigned to the Department of Energy and Climate Change; the designated minister is the Parliamentary Under Secretary of State, Baroness Verma. Verma obtains answers from the Met Office. The person at the Met Office with final authority is the Chief Executive Officer, John Hirst. In practice, Hirst delegates authority to the Chief Scientist at the Met Office, Julia Slingo. Thus, it is actually Slingo who was refusing to answer the Parliamentary Questions, with Hirst and Verma backing her (perhaps without thinking).
I have had a few e-mail exchanges with Slingo in the past. Slingo has never really addressed the issues that I raised. Instead, she has replied largely with rhetoric and a display of gross ignorance about undergraduate-level statistics; for an example, see the Bishop Hill post “Climate correspondents”. Thus, I decided that trying to talk directly with Slingo about the Parliamentary Questions would be a waste of time. Hence, I tried talking with Hirst. My message to Hirst included the following.
Last week, Lord Donoughue tabled Parliamentary Question HL6132, about statistical models of global temperature data. HL6132 is essentially the same as HL5359, which the Met Office refused to answer. The Met Office Chief Scientist does not have the statistical skills required to answer the Question; there is, however, at least one scientist at the Met Office who does have the skills—Doug McNeall. I ask you to ensure that the Question is answered.
Doug McNeall is a statistician. He and I have had cordial e-mail discussions in the past. In particular, after my op-ed piece in WSJ appeared, on 12 August 2011, McNeall sent me an e-mail stating that the trending autoregressive model is “simply inadequate”. Indeed, that would be obvious to anyone who has studied statistical time series at the undergraduate level. Note that this implies that a statistician at the Met Office has stated that the Answer given to the original Parliamentary Question (HL3050) is unfounded.
Lord Donoughue’s fourth Question was, as before, refused an answer. Afterwards, I received the following message from Hirst.
I would like to assure you that the Met Office has not refused to answer any questions. The questions you refer to were answered by Baroness Verma, Parliamentary Under-Secretary of State at the Department of Energy and Climate Change.
I note that in her response to HL5359 and HL6132, and a number of other questions from Lord Donoughue, Baroness Verma has offered for him to meet officials to discuss this and related matters in more detail.
Afterwards, Lord Donoughue asked the question a fifth time. And I sent the following message to Hirst.
I do not know whether your message is serious or just your way of telling me to get lost. In case of the former, some elaboration follows.
The question that Lord Donoughue has been asking requires the calculation of a single number. The calculation is purely arithmetical: there is no opinion or judgment involved (nor is background in climate needed). Furthermore, the calculation is easy enough that it could be done in minutes, by someone with the appropriate statistical skills. You could think of it as being similar to finding the total of a column of integers.
The number that Lord Donoughue is asking for is 0.001, according to my calculation. (Yes, it is that simple.) Lord Donoughue, though, would like the number calculated by an official body. He therefore tabled Parliamentary Questions asking HM Government for the number.
Lord Donoughue has now received Written Answers to four such Parliamentary Questions: HL4414, HL5031, HL5359, HL6132. None of those Answers give the number. Instead, the Answers make excuses as to why the number is not given. The main excuse seems to be that the number is not important. The importance of the number, however, is a separate issue: even if the number has no importance at all, the arithmetical calculation can still be done, and the number can still be given.
HM Government has been relying upon the Met Office, to supply them with the number; the Met Office has refused to do this. In other words, the Met Office has refused to answer the question—contrary to the claim in your message. What reason does the Met Office have for refusing to supply the number? The required time would be less than the amount of time that the Met Office has spent in refusing.
Parliamentary Questions have a history going back centuries. I do not have expertise in this area, but it is my understanding that HM Government is obliged to either provide an Answer to a Question or else give a valid reason for not providing an Answer. The refusal of the Met Office to supply the number would thus seem to be leading to a violation of a centuries-old parliamentary convention. Indeed, I have now talked with other members of the House of Lords and the Commons about this: there is real concern, and apparently also by parliamentary officials.
Lord Donoughue has now asked for the number a fifth time. The tabled Question is as follows (HL6620).
To ask Her Majesty’s Government … whether they will ensure that their assessment of [the number] is published in the Official Report; and, if not, why not.
The Answer is due by April 12th. My hope is that if the Met Office continues to refuse to supply the number, HM Government will get the number from elsewhere.
There was no immediate response to that. I did, however, receive an invitation from Doug McNeall to visit the Met Office and discuss the statistics of trends in global temperatures. I replied as follows.
Kind thanks for this. In principle, such a meeting would surely be valuable. The Met Office, however, is refusing to answer a simple arithmetical question, and moreover, is presenting dishonest reasons for doing so. Given that, I do not have confidence that discussion could be in good faith.
Hence, I respectfully decline. If the Met Office supplies the number, I would be happy to discuss this further.
A week later, the fifth Question (HL6620) was answered as follows.
As indicated in a previous Written Answer given … to the noble Lord on 14 January 2013 (Official Report, col. WA110), it is the role of the scientific community to assess and decide between various methods for studying global temperature time series. It is also for the scientific community to publish the findings of such work, in the peer-reviewed scientific literature.
Thus, in the opinion of the Met Office, Parliament has no right to ask scientific questions of government scientists.
A few days later, I received the following message from Hirst.
I’m sorry for the delay in replying; I have been away from the office.
I’m sorry if my previous e-mail gave you the impression I did not wish to discuss this matter further. That was not my intention. Indeed, if you are not satisfied with the answers that have been given to Lord Donoughue’s Parliamentary Questions, I would be more than happy for us to debate your concerns, as part of a detailed scientific discussion about the statistical modelling of global mean temperatures.
I understand Doug McNeall has offered to arrange a meeting with you and other Met Office scientists who work in this area. I feel this would be a sensible way forward and, although our views may differ in some respects, can assure you we would approach this meeting in good faith.
I look forward to hearing from you.
Hirst is clearly supporting the obstructionism. I decided that there was no point in replying.
Under the rules of Parliament, the person with responsibility for a Parliamentary Question is the government minister who delivers the Answer. In our case, that minister is Baroness Verma. According to the Companion to the Standing Orders and Guide to the Proceedings of the House of Lords, §4.68 Ministerial Responsibility, “Ministers should be as open as possible with Parliament, refusing to provide information only when disclosure would not be in the public interest” and “Ministers who knowingly mislead Parliament will be expected to offer their resignation to the Prime Minister”.
Lord Donoughue then sent a strongly-worded letter to Under Secretary Verma, citing the section on Ministerial Responsibility, and adding “I trust we will not reach that point since you are clearly not behind the wilful refusal to answer the Question”. Indeed, Verma seems to have been trusting that the Answers supplied to her by the Met Office were written in good faith.
Then Lord Donoughue asked the question a sixth time (HL62). The Answer, this time, included the relative likelihood. The full Answer (excluding footnotes) was as follows.
There are many ways to analyse time series, including the use of physical and statistical models. The relevance of any technique depends on the question asked about the data. The Met Office has compared the likelihood of the two specified models for fitting the three main independent global near-surface temperature time series (originating from UK Met Office and NASA and NOAA in the US), using a standard approach.
The statistical comparison of the model fits shows the likelihood of a linear trend model with first-order autoregressive noise in representing the evolution of global annual average surface temperature anomalies since 1900, ranges from 0.08 (Met Office data) to 0.32 (NOAA data), relative to the fit for a driftless third-order autoregressive integrated model. The likelihood is 0.001 if the start date is extended back for example to 1850 (Met Office data). These findings demonstrate that this parameter is very sensitive to the data period chosen and to the dataset chosen for a given time period, for such a statistical model.
A high value of relative likelihood does not necessarily mean that a model is useful or relevant. The climate is a highly complex physical system; to model it requires an understanding of physical and chemical processes in the atmosphere and oceans, natural variability and external forcings, i.e. with physically-based models. Work undertaken at the Met Office on the detection of climate change from temperature observations is based on formal detection and attribution methods, using physical climate models and not purely statistical models, as discussed in Chapter 9 of the Contribution of Working Group I to the IPCC’s Fourth Assessment Report, 2007.
The second paragraph gives the relative likelihood of the trending autoregressive model with respect to the driftless model. The relative likelihood is 0.08, if we analyze years 1900–2012 , and it is 0.001, if we analyze years 1850–2012 (using Met Office data). In either case, then, the trending autoregressive model is much less likely than the driftless model to be the better model of the data. Hence, the statistical model that was relied upon in the Answer to the original Question (HL3050) is untenable.
Most of the third paragraph is verbiage. In particular, the cited “physical climate models”, which the Met Office runs on its supercomputer, do indeed provide some evidence for global warming. Physical climate models and statistical models are both known as “models”, but they are different things. It is only the statistical models that are relevant to the Question. The physical climate models, though impressive in many ways, do not provide observational evidence for global warming.
The issue here is the claim that “the temperature rise since about 1880 is statistically significant”, which was made by the Met Office in response to the original Question (HL3050). The basis for that claim has now been effectively acknowledged to be untenable. Possibly there is some other basis for the claim, but that seems extremely implausible: the claim does not seem to have any valid basis.
Plainly, then, the Met Office should now publicly withdraw the claim. That is, the Met Office should admit that the warming shown by the global-temperature record since 1880 (or indeed 1850) might be reasonably attributed to natural random variation. Additionally, the Met Office needs to reassess other claims that it has made about statistically significant climatic changes.
Lastly, it is not only the Met Office that has claimed that the increase in global temperatures is statistically significant: the IPCC has as well. Moreover, the IPCC used the same statistical model as the Met Office, in its most-recent Assessment Report (2007). The Assessment Report discusses the choice of model in Volume I, Appendix 3.A. The Appendix correctly acknowledges that, concerning statistical significance, “the results depend on the statistical model used”.
What justification does the Appendix give for choosing the trending autoregressive model? None. In other words, the model used by the IPCC is just adopted by proclamation. Science is supposed to be based on evidence and logic. The failure of the IPCC to present any evidence or logic to support its choice of model is a serious violation of basic scientific principles — indeed, it means that what the IPCC has done is not science.
To conclude, the primary basis for global-warming alarmism is unfounded. The Met Office has been making false claims about the significance of climatic changes to Parliament—as well as to the government, the media, and others — claims which have seriously affected both policies and opinions. When questioned about those claims in Parliament, the Met Office did everything feasible to avoid telling the truth.