Many dimensions to life and science

This post is timed to coincide with a meeting tomorrow, the Royal Meteorological Society’s “Communicating Climate Science”. If you are going, do come and say hello. If you aren’t, look out for me tweeting about it from 2-5.30pm BST.

On not blogging

I haven’t forgotten about you. I’ve still been churning over ideas and wanting to share them with you. I’ve thought of all of you that comment here, and those that silently lurk, whether friends, family, scientists, sceptics, passers-by, or a combination of these. But two big things this year have had to take priority over blogging (and the even more time-consuming process of moderating and replying to comments).

The first was a deadline. As some of you know well, the Intergovernmental Panel on Climate Change (IPCC) produces a report summarising the state-of-the-art in climate science research, and related topics, about every six years. They do this so policymakers have a handy (in practice, enormous and not very handy) reference to the evidence base and latest predictions. The IPCC set cut-off dates for including new research: one date for submission to journals, and another for acceptance after the peer-review process. The first of these dates was the 31st July this year. Translation: “try to finish and write up every piece of work you’ve ever started by this date”. Not every climate scientist chose to do this. But the project I work for, ice2sea, actually had it written into a contract with its funders, the European Union. We had no choice but to submit whatever was our current state-of-the-art in sea level predictions. I was a co-author of six papers* finished and submitted during June and July, and had several other studies on the go that didn’t make the deadline. So it was a rather intense time, and science had to take priority over talking about science.

The second was personal. I hesitated about whether to say this here. But part of my motivation for being a climate scientist in the public eye was to show the human side. And I also wanted to let you know that this blog is so important to me, has been so transformative, that it took something very big to keep me away. My husband and I separated two months ago.

I’m back, and I’m preparing for a big move. The US-based publisher and organisation PLoS (Public Library of Science) has invited me to be their climate blogger. It’s a fantastic opportunity to gain a big audience (more than 200,000 visitors per month, and a feed to Google News). I’m very happy to support PLoS because they publish open access journals, and because one of these (PLoS ONE) goes even further in its commitment to transparency in science. It will publish anything scientifically valid, whether or not it is novel. This might not sound important, or even a good idea, but it is an essential counter to the modern problem that plagues journals: that of only publishing new results, and not repeat studies. For the scientific method to work, we need studies that repeat and reproduce (or contradict) previous research. Otherwise we risk errors, chance findings, and very occasionally fraud, remaining unnoticed for years, or forever. I’m hosted at PLoS from the second week in December and will be posting twice a month.

The first post at PLoS will be a (long overdue) introduction to predicting climate change. It will probably be based around a talk I gave at the St Paul’s Way summer science school, at which I was the final speaker, which made Prof Brian Cox my warm-up act.

In other news, I talked about the jet stream and climate change live on BBC Wiltshire (9 mins), which was well received at the climate sceptic site Bishop Hill, and did a live Bristol radio show, Love and Science (1 hour). I also returned to my particle physics roots, with a Radio 4 interview about the discovery of the Higgs Boson (3 mins).

Our new(-ish) paper

Now the science bit. This is an advertisement for a paper we published in August:

Stephens E.M., Edwards T.L. and Demeritt D. (2012). Communicating probabilistic information from climate model ensembles—lessons from numerical weather prediction. WIREs Clim Change 2012, 3: 409-426.

It’s paywalled, but I can send a copy to individuals if they request it. Liz Stephens is a colleague and friend from my department at Bristol that did a great study with the UK Met Office and David Spiegelhalter on the interpretation of probability-based weather forecasts, using an online game about an ice cream man. I’ve never met David Demeritt, except in one or two Skype video calls. He’s interested in, amongst other things, how people interpret flood forecasts. I haven’t passed this post by them, but hopefully they will comment below if they have things to add or correct.

We noticed there was quite a bit of research on how well people understand and make decisions using weather forecasts, such as the probability of rainfall, and uncertainty in hurricane location, but not much on the equivalents in climate change. There have been quite a few papers, particularly in the run-up to the new IPCC report, that talk in general terms about how people typically interpret probability, uncertainty and risk, and about some of the pitfalls to avoid when presenting this information. But very few actual studies on how people interpret and make decisions from climate change predictions specifically. We thought we’d point this out, and draw some comparisons with other research areas, including forecasting of hurricanes, rain, and flooding.

Ensembles

The ‘ensembles’ in the title are a key part of predicting climate and weather. An ensemble is a group, a sample of different possibilities. Weather forecasts have been made with ensembles for many years, to help deal with the problem of our chaotic atmosphere. The most well-known explanation of chaos is the ‘butterfly effect’. If a butterfly stamps its foot in Brazil, could it cause a tornado in Illinois? Chaos means: small changes can have a big effect. A tiny change in today’s weather could lead to completely different weather next week. And in the same way, a tiny error in our measurements of today’s weather could lead to a completely different forecast of the weather next week. But errors and missing measurements are inevitable. So we try to account for chaotic uncertainty by making forecasts based on several slightly different variations on today’s weather. This is one type of ‘ensemble forecast’. It’s simply a way of dealing with uncertainty. Instead of one prediction, we make many. We hope that the ensemble covers the range of possibilities. Even better, we hope that the most common prediction in the ensemble (say, 70% of them predict a storm) is actually the most likely thing to happen. This gives us an estimate of the probability of different types of weather in the future.

Ensembles are at the heart of our attempts to describe how sure we are about our predictions. They are used to explore an uncertain future: what are the bounds of possibility? What is plausible, and what is implausible? Some climate prediction ensembles, like the weather forecast ensemble above, relate to the information we feed into the model. Others relate to imperfections in the models themselves. Some specific examples are in the footnotes below.**

The question we ask in our paper is: how should we express these big, complex ensemble predictions? There are too many dimensions to this problem to fit on a page or screen. Our world is three dimensional. Add in time, and it becomes four. There are very many aspects of climate to consider, such as air temperature, rainfall, air pressure, wind speed, cloud cover, and ocean temperature. We might have a prediction for each plausible input value, and a prediction for each plausible variation of the model itself. And one of these ensembles is produced for each of the different climate models around the world. Frankly, ensembles are TMI***.

To simplify or not to simplify

Scientists often think that the more information they can give, the better. So they dump all the raw ensemble predictions on the page. It’s a natural instinct: it feels transparent, honest, allows people to draw their own conclusions. The problem is, people are a diverse bunch. Even within climate science, they have different knowledge and experience, which affects their interpretation of the raw data. When you broaden the audience to other scientists, to policymakers, businesses, the general public, you run the risk of generating as many conclusions as there are people. Worse still, some can be overwhelmed by a multitude of predictions and ask “Which one should I believe?”

To avoid these problems, then, it seems the expert should interpret the ensemble of predictions and give them in a simplified form. This is the case in weather forecasting, where a meteorologist looks at an ensemble forecast and translates it based on their past experience. It works well because their interpretations are constantly tested against reality. If a weather forecaster keeps getting it wrong, they’ll be told about it every few hours.

This doesn’t work in climate science. Climate is long-term, a trend over many years, so we can’t keep testing the predictions. If we simplify climate ensembles too much, we risk hiding the extent of our uncertainty.

Our conclusions can be summed up by two sentences:

a) It is difficult to represent the vast quantities of information from climate ensembles in ways that are both useful and accurate.

b) Hardly anyone has done research into what works.

We came up with a diagram to show the different directions in which we’re pulled when putting multi-dimensional ensemble predictions down on paper. These directions are:

  1. “richness”: how much information we give from the predictions, i.e. whether we simplify or summarise them. For example, we could show a histogram of all results from the ensemble, or we could show just the maximum and minimum.
  2. “saliency”****: how easy it is to interpret and use the predictions, for a particular target audience. Obviously we always want this to be high, but it doesn’t necessarily happen.
  3. “robustness”: how much information we give about the limitations of the ensemble. For example, we can list all the uncertainties that aren’t accounted for. We can show maps in their original pixellated (low resolution) form, like the two maps shown below, rather than a more ‘realistic-looking’ smoothed version, like these examples.

Here’s the diagram:

The three ‘dimensions’ are connected with each other, and often in conflict. Where you end up in the diagram depends on the target audience, and the nature of the ensemble itself. Some users might want, or think they want, more information (richness and robustness) but this might overwhelm or confuse them (saliency). On the other hand, climate modellers might reduce the amount of information to give a simpler representation, hoping to improve understanding, but this might not accurately reflect the limitations of the prediction.

In some cases it is clear how to strike a balance. I think it’s important to show the true nature of climate model output (blocky rather than smoothed maps), even if they are slightly harder to interpret (you have to squint to see the overall patterns). Otherwise we run the risk of forgetting that – cough – all models are wrong.

But in other cases it’s more difficult. Giving a map for every individual prediction in the ensemble, like this IPCC multi-model example, shows the extent of the uncertainty. But if this is hundreds or thousands of maps, is this still useful? Here we have to make a compromise: show the average map, and show the uncertainty in other ways. The IPCC deals with this by “stippling” maps in areas where the ensemble predictions are most similar; perhaps the unstippled areas still look quite certain to the hasty or untrained eye. I like the suggestion of Neil Kaye, fading out the areas where the ensemble predictions disagree (examples of both below).


This brings us to the second point of our conclusions. The challenge is to find the right balance between these three dimensions: to understand how the amount of information given, including the limitations of the ensemble, affects the usefulness for various audiences. Do people interpret raw ensemble predictions differently to simplified versions of the same data? Do full ensemble predictions confuse people? Do simplifications lead to overconfidence?

There is very little research on what works. In forecasting rainfall probabilities and hurricanes, there have been specific studies to gather evidence, like workshops to find out how different audiences make decisions when given different representations of uncertainty. People have published recommendations for how to represent climate predictions, but these are based on general findings from social and decision sciences. We need new studies that focus specifically on climate. These might need to be different to those in weather-related areas for two reasons. First, people are given weather forecasts every day and interpret them based on their past experiences. But they are rarely given climate predictions, and have no experience of their successes and failures because climate is so long-term. Second, people’s interpretation of uncertain predictions may be affected by the politicisation of the science.

To sum up: we can learn useful lessons from weather forecasting about the possible options for showing multi-dimensional ensembles on the page, and about ways to measure what works. But the long-term nature of climate creates extra difficulties in representing predictions, just as it does in making them.

 

* Papers submitted for the IPCC Fifth Assessment Report deadline:

  • Ritz, C., Durand, G., Edwards, T.L., Payne, A.J., Peyaud, V. and Hindmarsh, R.C.A. Bimodal probability of the dynamic contribution of Antarctica to future sea level. Submitted to Nature.
  • Shannon, S.R., A.J. Payne, I.D. Bartholomew, M.R. van den Broeke, T.L. Edwards, X. Fettweis, O. Gagliardini, F. Gillet-Chaulet, H. Goelzer, M. Hoffman, P. Huybrechts, D. Mair, P. Nienow, M. Perego, S.F. Price, C.J.P.P Smeets, A.J. Sole, R.S.W. van de Wal and T. Zwinger. Enhanced basal lubrication and the contribution of the Greenland ice sheet to future sea level rise. Submitted to PNAS.
  • Goelzer, H., P. Huybrechts, J.J. Fürst, M.L. Andersen, T.L. Edwards, X. Fettweis, F.M. Nick, A.J. Payne and S. Shannon. Sensitivity of Greenland ice sheet projections to model formulations. Submitted to Journal of Glaciology.
  • Nick, F.M., Vieli, A., Andersen, M.L., Joughin, I., Payne, A.J., Edwards, T.L., Pattyn, F. and Roderik van de Wal. Future sea-level rise from Greenland’s major outlet glaciers in a warming climate. Submitted to Nature.
  • Payne, A.J., S.L. Cornford, D.F. Martin, C. Agosta, M.R. van den Broeke, T.L. Edwards, R.M. Gladstone, H.H. Hellmer, G. Krinner, A.M. Le Brocq, S.M. Ligtenberg, W.H. Lipscomb, E.G. Ng, S.R. Shannon , R. Timmerman and D.G. Vaughan. Impact of uncertainty in climate forcing on projections of the West Antarctic ice sheet over the 21st and 22nd centuries. Submitted to Earth and Planetary Science Letters.
  • Barrand, N.E., R.C.A. Hindmarsh, R.J. Arthern, C.R. Williams, J. Mouginot, B. Scheuchl, E. Rignot, S. R.M. Ligtenberg, M, R. van den Broeke, T. L. Edwards, A.J. Cook, and S. B. Simonsen. Computing the volume response of the Antarctic Peninsula ice sheet to warming scenarios to 2200. Submitted to Journal of Glaciology.

** Some types of ensemble are:

  1. ‘initial conditions': slightly different versions of today’s weather, as in the weather forecasting example above
  2. ‘scenarios': different possible future storylines, e.g. of greenhouse gas emissions
  3. ‘parameters': different values for the control dials of the climate model, which affect the behaviour of things we can’t include as specific physical laws
  4. ‘multi-model': different climate models from the different universities and meteorological institutes around the world

*** Too Much Information

**** Yes, we did reinvent a word, a bit. 

25 comments

  1. thingsbreak

    Oh, I didn’t see the personal news you shared. Sorry to hear that, I hope you make it through everything okay. Didn’t mean to be insensitive to that.

  2. Paul Young

    Hi Tamsin,

    Very pleased to have found this blog – lots of issues that I’ve been thinking about too, after having drifted around atmospheric chemistry modelling, temperature obs and multi-model stuff for AR5/ACCMIP.

    On graphical representations, I’ve become a fan of contour plots that are only colour-filled where the signal is significant (e.g. Fig 2 of Young et al. (submitted) [obs and models] and Figs 9-11 of “Young et al. (submitted; ACPD) [models]). For the multi-model context, I also like Tebaldi et al.’s (2012, GRL; paywalled!) idea of showing both where the signal is significant and where models agree that it is small/insignificant. Couldn’t do that for my tropospheric ozone study unfortunately….

    Keep up the good work!

  3. Tamsin Edwards

    I should have said 2-5.30pm GMT, not BST!

    Roddy – great, see you later

    Michel – thank you very much

    thingsbreak – no apologies necessary, but thank you and I’ll bookmark your post

    Paul – absolutely, yes – Jonty and I usually blank out areas of high uncertainty (e.g. outside 3sigma) or low significance (e.g. rank histogram test for differences between two equilibrium simulations) – when we publish you’ll see them!

  4. Lindsay Lee

    Hi Tamsin,
    Thanks for another great post. I’m pleased you’re back after what has surely been a difficult time and wish you all the best for the future. Congratulations on your PLoS role!
    On the post, this is something i’ve been thinking about a lot lately – in fact, i’ve been thinking a lot lately about all sorts of uncertainty and probability issues since attending the Uncertainty in Climate Change Workshop http://www2.image.ucar.edu/event/uncertainty-climate-change-research-integrated-approach at NCAR which brought together climate modellers and climate data users and a few statisticians. It’s also been on my mind as we try to find the best ways to present our own work where we have an immense amount of data. It seems to me that there is a lot of data ‘wasted’ in climate science because of the issues you talk about. My obvious view is that more statisticians should be involved – and from the very start! They can be used to help collect the right data and help get as much information as possible with massive data.
    I have 2 particular points. 1) talks about how we are communicating the results of a perturbed physics ensemble for model uncertainty and 2) in general about the use of probabilities.
    1. In our work we create ensembles of a single model based on perturbed physics and we have produced a lot of data which has taught us a lot about our model that we want to present. The work is based on statistical design, emulation and sensitivity analysis and because the application of the methods are pretty new in the climate world we did a smaller study to illustrate what we were trying to do. http://www.atmos-chem-phys.net/12/9739/2012/acp-12-9739-2012.html. We learnt more than i expected from this in how to present the work. Forgive me for filling your blog with our work but here goes…
    Obviously we present the mean output and its uncertainty that results from the model ensemble (as a global map) but alone they don’t really tell you much. In fact, during the workshop above there was a bit of negativity towards such studies because of that fact, and so i found myself taking a defensive stance and speaking up a lot more than i ever have before! The problem is that these specific numbers depend on the model used and the parameters perturbed and their ranges – all of which won’t apply to other studies. The relative uncertainty in different regions of the globe is more informative but still doesn’t really tell you much more than you probably already know. For me, the value of our work is carrying out the next step of a variance-based sensitivity analysis, possible thanks to emulation. This decomposes the uncertainty into its various parametric sources and you can start to see what processes in the model contribute to the uncertainty in the predictions they make and the science that is happening in the model can be seen by mapping the variance contributions. Producing the maps like those in the paper above has completely changed how people see the work. Initially, when i had no results, banging on about the behind-the-scenes statistical methods wasn’t really getting me very far! We’ve found the model sensitivities have scientific meaning which should be true no matter what model you used, parameters or the ranges. I don’t think the particular parameters matter as long as they are representing some process that is in all similar models, though this is normally a point of debate. Whilst it is model dependent in terms of its structure will be different to similar models you can see which processes are happening where and when, which is science – and if the processes aren’t happening but they should be then that is important knowledge too. So in perturbed physics experiments, I agree that the mean and variance are not very informative to understanding uncertainty but there is so much data not exploited and this data can really help understand what is going on. We’ve used what we’ve learnt to write the next paper which has more parameters and is done for a whole year . Hopefully, when it’s published you’ll be able to make up your own minds about whether it’s a good way of communicating what is happening in our model and what leads to uncertainty.
    2. In a more general sense, also as a result of this workshop I’ve been thinking about how people view probability. There was a lot of talk that probabilities should be dispensed with altogether, particularly in the sense of producing scenarios, so again i got all defensive about my field! My main point during the workshop was that completely removing any mention of probability from discussions about scenarios is that people will think probabilistically about them regardless and you rely on the fact they all think the same and also that unweighted doesn’t mean ‘no probability’ but actually refers to ‘equally weighted’. In fact, someone said ‘i haven’t attached probability but treated them 50/50′, or something close to that. We all think probabilistically every day but doing it formally strikes fear! If you give people 10 options they will weight them somehow to summarise, most likely either treating them equally or choosing one they’ve heard more about to dominate. If we can be more clear about precisely what probabilities are assigned or should/shouldn’t be applied depending on what you want to learn is the way forward not ignoring them altogether. If you state what probabilities you have attached it’s more likely to spark debate than claiming none. Perhaps we should be better at communicating the fact that the probabilities are conditional and what they are conditional on (or just the fact that they are conditional stimulating the user to wonder what they are conditional on) – rather than just talk about probability and uncertainty in too general a fashion. I realise that this sounds like it would complicate matters but I think clarity is most important given that these words are used to represent a hundred different meanings.
    Basically, I think there is a huge gap between scientists who need to use probability and statisticians who understand it that needs to be filled and this is through closer collaboration rather than stand offs. Blogs like this and papers like the one you link to are hopefully going to help decrease the gap too but it needs to be put into action. It’s great being a statistician working in a science department working on such interesting problems with access to the relevant tools, and we’ve just employed two more stattos – yay to Leeds!

    • Jonty Rougier

      hi Lindsay,

      Regarding your point (2): I completely agree that “we’re not attaching probabilities” often ends up implicitly at equal probabilitites. But I want to mention that there is a large literature in Decision Theory about how far one can go without probabilities. It comes back to the divide between Frequentist and Bayesian statistics, because the shibboleth of Frequentist statisticians is not attaching a probability distribution to the scenario; something that Bayesians feel much more comfortable about.

      Decision theory is all about choosing actions under uncertainty, where each action has a consequence under each scenario. There are two probability-free approaches to choosing actions. Minimax regret (originating with A. Wald but tuned by L.J. Savage) is one theoretical possibility, but unworkable in practice, since there is usually a very low probability catastrophe scenario which ends up dominating the calculation. These would have to be screened out in order to produce something workable, and then we are back with probabilities again, albeit at a lower intensity.

      The second approach is based on admissibility. This is the idea that one would not want to choose an action that was unambiguously dominated by another, no matter what actually happens. Such actions are termed ‘inadmissible’. Admissible actions are not necessarily good, but selecting an inadmissible one is embarrassing. What is interesting is that, putting aside complications that arise from uncountable sets, an action is admissible if and only if it is Bayesian. That is to say, if you insist on using only admissible actions, then your behaviour is consistent with the existence of probabilities on the scenarios, even if you vehemently deny that they exist.

      I cannot figure out if this theoretical result has any operational implications. It seems to be saying that you might as well attach probabilities to the scenarios, because if your chosen action is not consistent with some set of probabilities, then you will look stupid, having chosen an inadmissible action.

      Enough of all this theory — I ought to finish with something a bit more earthy! First, there are those who deny that it is meaningful to attach probabilities to scenarios. They may be talking about their own personal aversion, or they may have some view about science being ‘objective’. If it is their personal aversion, then perhaps they can be counselled. If they think that science is objective, then they cannot be doing any!

      Then there are those who are not averse to attaching probabilities to scenarios, but feel they cannot determine what those probabilities should be. There is a large literature on eliciting probabilities (which must be done carefully and sympathetically), designed to help people to quantify their judgements, at least approximately. And often it turns out that this is all that is needed; the optimal choice of action is typically robust to small perturbations in probabilities, as discussed in ch1 of Jim Smith’s recent book Bayesian Decision Analysis.

      I think we statisticians ought to be able to persuade scientists to do what to most people would come naturally: to attach probabilities to scenarios. But the scenarios need to be constructed with this in mind: ideally they should be a partition of the future, so that the probabilities can sum to one. I think one of the reasons that people struggle to attach probabilities to SRES-type scenarios is that they were not constructed like this.

      Jonty.

      • Alexander Harvey

        Hi Lindsay and Jonty,

        I have a point to make but my usage of terms is likely idiosyncratic and hence much of the long introduction may bring no joy. I may not be right on all points but I am passed carrying for sake of pride and must relax into just relating my thinking such as it is. It being wonderfully subjective and pleased to be so.

        I did appreciate:

        “If it is their personal aversion, then perhaps they can be counselled. If they think that science is objective, then they cannot be doing any!”

        even though I do not in fact do science.

        I commence by defining something I will call my rational prejudice. Rational in the sense of a goal oriented application of logic, prejudice in the sense of a prior judgement, prior in the sense of before all and any evidence, my purest distilled prejudice, that which I alone bring to the party, ideally before any evidence exists or more likely before I have seen any.

        I will need to know what type or class the statistic belongs to, but little else, and I will try and find a density, or mass function, that under certain coordinate transformations, that I deem sensible, has some invariant properties, that I deem desirable. I may additionally wish the density function to be optimal, in some sense, or at least not sub-optimal. This last point is I think related to your admissability criterion, yet different.

        I draw a sharp distinction between a density function and a probability density function, in that I am relaxed if the function is not integrable. I need it to be the fount of density, to make the final integrals after evidence meaningful, I don’t need it to be integrable by itself. This I think is important but it might be a minority view but I find it to be a practical necessity.

        I will illustrate by way of some examples.

        I might be informed that the statistic will have simple units and is the result of a simple measurement, e.g. not a difference or a ratio etc. nor that there be some uniquely privileged value equivalent to a zero or origin. In that case I would gravitate to a uniform density on the whole of the real line. I haven’t enquired as to the actual units, I have no scale information so I must use the whole of the real line. This is a density function but it is not integrable, it is not a pdf. The goal behind my rational prejudice is the preservation of the integral of the density function between any two bounds under a translation along the coordinate, something I deem to be desirable.

        However if I am informed that the statistic is confined to non-negative values, I would gravitate to a power law function, the goal being the invarince of relative point density at two points under a coordinate transformation by a constant multiple, a stretching or compression of the coordinate or more prosaically under a change of units.

        If I am further informed that the statistic is the result of a ratio of two measurements, I would graviate to a one specific power law, the reciprocal e.g. 1/x, or 1/abs(x) . The goal being to additionally preserve the form of the density function under an exchange of the naming of the measurments, under reciprocation

        I am being tedious for I labour the point that I am discussing something rather different to a prior pdf, which is only prior to new evidence and is itself integrable to unity. I am discussing a purer form of prejudice, as complete a lack of empirical objectivity as I can muster and still be rational, my essential subjectivity, my rational prejudice based on the class of the statistic alone.

        I will give an example to demonstrate a further narrowing based on some desrable optimum.

        If I am told that I may safely assume a normally distributed error function and that the mean, the true value, could be anywhere on the real line and that the statistics of interest are the unknown parameters (mu,var) or (mu,sigma) if one prefers, I have additional room for rational prejudice.

        Having set the prior density fuction, i.e. df, for mu, (the mean or true value) to a constant everywhere on the real line, and the prior df for var to be some power function according to the reasoning above. I can further narrow this second df to optimise some further property. I will be using the likelihood function as expressed on the two space (mu,var) for a normal distribution. Specific choices for the exponent of the power function on var may give me optimal, in the sense of unbiased, estimates for either the median, mean or mode of the var parameter, but not I fear all three. Given sufficient, e.g. overwhelming evidence, this choice is immaterial for I will get convergence, but I would probably choose the exponent that produces an unbiased estimator for the mean or expectation of the variance once there be suffucient evidence to perform the necessary integral.

        I will try to make this a bit more relevant to you.

        My distinction between a df, or pdf on one hand, and a likelihood function on the other is a class distinction based on how they transform under a transformation of coordinates. I cannot see that it be necessary to ellicit an expert’s prior pdf if a df will suffice. To put it crudely, I think an expert squiggle with no vertical scale may suffice. And If the expert is non-commital concerning the tails beyond that they are bounded above by some straight line value, evidence may come to our rescue by sufficiently constraining the a posterior df to one that is integrable, and at the very least you may have a more relaxed and honourable expert.

        The expert squiggle and my rational prejudice are different things or the expert isn’t one. Yet my treatment of density does I think carry over. If the evidence is sufficient to allow integration you have a result, if it is not, you have an honest, or honourable problem. Either you lack sufficient evidence and require more experimental runs, or the evidence you are gathering would never suffice, i.e. no matter how much you gathered the result would never be integrable unless subjective forcing were applied, more about which later. I think this is saying that there be no frequentist solution given that type of evidence. Given such a parlous state of affairs one could engage in expert arm twisting which is probably an unethical treatment of an expert, or admit that one is barking up the wrong tree and redesign the experiment.

        I will conclude with an observation.

        Whenever I see a prior chosen with a deliberate goal of ensuring integrability I dare say that the author sees such constraint to be uncontroversial. Well it isn’t to me. If the evidence is not up to the task, e.g. would not eventual converge to an integrable function give any superfluity then I think such practice be something more than dishonourable if not quite dishonest. By this I mean something other than a choice of a function less constrained than desired but having the advantage of being integrable or more simple to integrate, I refer to a chopping, cropping, or reprofiling to force a pdf to emerge. I would rather an honourable a posteriori df than a torturer’s pdf. I would still have the all important density, I would still be able to combine further evidence of the same type or perhaps put it forward as an a priori df to some future experiment. To summarise I think being dense is essential and being probable less so.

        P.S.

        A small technical note lest my treatment for finding the parameters given normally distriubted errors seems impenetrably bizarre. My update is performed as a singular action on the two space (mu,var) not on mu and var separately. I suspect that the separate application of updates in fact relies on an unspecified but necessary specific assumption as to the a priori density function on var, or sigma, and that be the one that gives an unbiased estimate for the expectation of the variance, or standard deviation as appropriate.

        Alex

  5. Dolphinlegs

    Tamsin

    Sorry to hear about your domestic situation but pleased to see you are back on blogging duty and that the future looks bright for you.

    A slightly difficult question if I may – do you accept that it is entirely possible that, by its very nature, climate cannot be predicted?

    Kind regards

    • Tamsin Edwards

      Thank you.

      We had quite a big discussion about that under the last post: “initial condition” versus “boundary condition” problems. Perhaps you would be interested to take a look? (and do come back with more questions that I or others will try to answer).

  6. David Young

    Interesting post. Ensembles are clearly a step forward. What I’m interested in is:

    1. Trying to estimate numerical error and control it using “adaptive” methods. There is a host of literature on this, none of which I’ve ever seen used in climate models or weather models.

    2. Trying to look for nonlinear effects such as bifurcations, instabilities, etc. Once again there are methods to do this, they are expensive but perhaps could be used with simplified models.

    3. I suspect there is a huge missing piece here in the subgrid models. There are obviously almost an unlimited number of mathematical forms these could take. It’s not just about varying parameters, but about the form of the nonlinear terms themselves In turbulence models, e.g., there is a strong effect of nonlinear terms in the models with macroscopic behaviour such as flow separation. This has gone totally unnoticed until recently when people realized that separated flow was an ill-posed problem and that it was OK to publish negative results. In short, as long as we “discard” variants we regard as “not looking good” we will never get to the effects of these nonlinear terms.

  7. Alexander Harvey

    Tamsin,

    Thanks, your posts are most welcome. I trust things are looking more positive for you.

    Hoping to stay on the topic if rather obliquely. I will make an appeal to the inner artist and try to unravel something about communication by image, by graphical representation. The advantage is in the having of an historic perspective. Representation has a rich history both in the fine arts and in science. I think it is clear that realism and usefulness are quite different things when it comes to the communication of facts, ideas, and beliefs.

    An example is the botanical illustration. Highly stylised, abstracted, representations of plants, produced by the skilled and knowledgeable illustrator may be much more useful a guide to the differentiation of species than that given by a visually realistic image, e.g. photographic representation. Both are truthful or faithful in their way but not in the same way. One embodies an idea by way of an ideal that may never occur, never be obtainable from nature, the other a representation of an instance or actualisation that did at least once occur in nature if not totally unaided.

    There is a question here on the relationship between climate and weather. The ideal botanical representation is never the plant but may be seen to engender representations of the species. Nor is it an average plant. Is climate averaged weather, a statistical representation of weather, or that which engenders weather? I happen to favour the latter, climate as process. In my case different types of representation would be appropriate, e.g. the caricature, cartoon, analogue or ideal of process.

    Since the rise of desk-top publishing, the relationship between author and reader has changed. An intermediatory, the diagrammatic illustrator, the graphic artist, has fallen to the wayside. The author may be expected to produce a product with close to the final imagery, under certain pressures and guidelines. This transition has perhaps been more true of the textbook than the journal, the latter sometimes having had an horrid scrawl as illustration. The ability of an author to produce something neat and tidy has changed the process. In many cases the image is a direct projection of the data and its mechanisation has increased fidelity to data but may lack clarity, may at once gain in precision yet lessen communication.

    That is a simplification, but will suffice as metaphor. Two regimes being distinguished, the need for factual precision, and for clarity of ideas, once mediated by two very different types of worker. Although commonly somewhat ignorant of the subject matter the illustrator traded in the communication of ideas, by trying to purvey the authors intentions.

    The rise in precision may have had a somewhat bizarre effect; the creation of a less subtle readership. If an image be regarded as a composite of projection and code it requires the reader to be less literal and more interpretive. The image is not simply true in a literal sense, purveying not just data but ideas it cannot be so. The reader could reasonably assume that the image was idealised, and in need of interpretation at the level of the idea. This is image as metaphor. A folly of the image as fact lies in perception of false precision.

    A need to encode ideas begets a need to decode them; a visual language, a consistent style. Style is traditionally in the domain of the publishing house, its editors and illustrators. With whomsoever it is now borne, this burden for the sake clear communication persists. I fear that a rise of a more literal interpretation will prevail unless there is a consitency to the signalling of ideas. One might query as to what signal is being sent when a representation of global temperatures is projected over a geographical image that includes South Sea Islands as opposed to a cartoon of the outline of the continents. Each is faithful but to different concepts, each giving a different signal to the reader.

    In closing, I will restate that we have a rich history of representation in both the graphic and fine arts. The fine art movements have used realism, abstraction, impression, etc., in the communication of ideas and ideals, in various media. I think that we still have some way to go in the dumping of “too much information” in computer readable form, for the readers to project as they will. Yet as long as the journal and other presentations of fixed iconography persist, we may continue to communicate both facts and ideas visually. To purvey both data and metaphor in a rich way. The author may decide as to whether an image is better spent to purvey facts or ideas and how to signal which is which. All of this may be of little practical assistance but I do think your ideas are well worth pursuing.

    Alex

  8. Max™

    First time posting here, glad to hear things are looking up for you, and I was compelled to post because I noticed an even more subtle version of the data presentation issue you mention above regarding the same graph!

    I remember noticing this years ago, http://www.ipcc.ch/publications_and_data/ar4/wg1/en/spmsspm-projections-of.html in the AR4 WG1 summary, the projections of precipitation map has the stippling you pointed out, but you can’t quite tell.

    I thought perhaps it was just unavoidable because the larger version it was resized from couldn’t be saved to that size without taking up extra space.

    Original 226 kb version: http://www.ipcc.ch/publications_and_data/ar4/wg1/en/fig/figure-spm-7-l.png

    Resized 103 kb version in the SPM: http://www.ipcc.ch/publications_and_data/ar4/wg1/en/fig/figure-spm-7.jpeg

    My 162 kb version the same size, resized in GIMP without reducing the quality: http://i341.photobucket.com/albums/o396/maxarutaru/figure-spm-7-l.png

    I have a sub 90 kb .jpg version somewhere which is just as clear as the .png, but I could never figure out why they resized it with such low quality.

  9. Doug Cotton

    Physics tells us that the adiabatic lapse rate represents that change in temperature that is required to keep the entropy of a parcel of air or water constant when its pressure is changed in an adiabatic and isohaline manner.

    Gravity alone determines the change in pressure for a given atmospheric mass. So gravity alone determines the adiabatic lapse rate.

    It seems that most climatologists have never learnt this basic fact of physics, so they were bluffed into believing a false conjecture that an imaginary greenhouse effect caused the observed temperature gradient responsible for the surface temperature being higher than the planet’s radiating temperature.

    Consideration of what happens on Venus (whose surface receives only about 10% of the insolation received by Earth’s surface) demonstrates that the adiabatic lapse rate can be the only reason for the surface temperature being hundreds of degrees hotter. Thus it also demonstrates the fiction of the GHE conjecture.

    Refer Section 8 of this paper for more detail on Venus.

    [This was off-topic but am approving for completeness — Tamsin]

  10. ikh

    Hi Tamsin,

    I am not a climate scientist but I am a computer scientist. I was a warmist
    and now I am a sceptic. This is simply a declaration of bias.

    I fully understand that all models are wrong and that the only way forward
    is via modelling. That is science.

    iiuc ( If I understand correctly ), Weather models are validated against
    reality. Because of chaos, weather models are only good for forecasting about
    5 days ahead before the uncertainties caused by chaos overwhelm their
    predictive capabilities.

    Climate models are not validated at all. iiuc GCM’s can not reproduce
    the MWP LIA CWP. They can not predict the climate for next year or for the
    next 5 or 10 or 30 years. So the only predictions we can make for weather
    are limited to a few days ahead before chaos overwhelms the model
    ( ensembles ). We can not predict the climate even a few years ahead. And yet, you assume that you can do something useful predicting the climate 100 years ahead. I’m sorry, but that somewhat defies credibility.

    Everyone seems to agree that the climate is chaotic and that intrinsically means
    that there will be a limit on how far forward we can realistically predict the
    climate of the future.

    The climate models seem to have a built in assumption of a feedback loop
    that a warming from CO2 will lead to a much larger warming in total.
    Mainly due to water vapour. And yet the empirical observational data tends to
    show a linear approx one degree C warming for a doubling of CO2. The satellite
    data tends to reduce this.

    Research in 2011 showed that particulates from volcanoes reach the
    stratosphere even for small to medium sized eruptions. The assumption was that only large eruptions like Pinatubo did this. This is a good example of unknown unknowns.

    Whilst climate GCM’s have these flaws, they are not suitable for making policy
    decisions.

    I would really appreciate your thoughts on this.

    /ikh

  11. Alexander Harvey

    Hi ikh,

    What is climate? I am never quite sure what people imply by the term. I can but express a view and illustrate it as best I can.

    Climate is at least an abstraction, some concept within which we embed meaning. For some it is the statistics of weather, a set of accumulations of observables that may converge to statistical moments (mean, variance, skew, etc.). In that sense it is forever a work in progress; all finite accumulations being only sufficient to produce some approximation of the underlying limits to which the statistics may converge.

    In such terms, the notion of climate as something predictable is troublesome. In common understanding a prediction refers to the future but it can also refer to both present and past. When not acted upon by some change in those agencies defined to be external to the climate system would not the climate be unchanging? Would not any statistical limit be fixed, and hence for past, present and future climate to be the same thing?

    Even so climate may be predictable in the sense that such limit values could be predicted from first principles. This has the flavour of what some assert as the concept of climate as a boundary condition problem. That given a sufficient understanding of climate as process, the limit values of its statistics can be determined irrespective of the system state at any particular moment. Sadly we may never have such an understanding, it may not be possible to determine the statistics, at best we may only predict the statistics in some statistical sense. Climate simulation is an attempt to predict what we cannot determine, we attempt various simulations of climate as process, in order to form some prediction of the real climate as process.

    When considering climate prediction under changing boundary conditions, when the climate and its statistics are not in any sense stationary, we may be refering to the statistics of the transitional path from one climate regime to another. I think that this notion may give rise to problems. Whereas it may be true that such statistics exist, as determined by the limits found through the accumulation of the inter-regime trajectories commencing from all starting points, one instance of that trajectory is paramount importance, the one we are on. Even if the transition of the climate can be predicted from first principles in the way I have indicated as a set of limits, the actual trajectory may only be quantifiable in a probabilistic sense.

    I have defined climate in a way that is predictable, about which we may make predictions. I have answered the question as to how a climate can be predictable by abstracting only notions that are capable of prediction, and I have drawn a distinction between a determination and a prediction which I will expand on.

    If I say it will rain at some place at some time, say here and tomorrow, I would regard that as a determination. If I say that there be a 50% chance of such rain, I would regard that as a prediction, its accuracy, or reliability is not revealed in the instance of rain here tomorrow.

    Any one such prediction, unlike a determination, is not capable of being accurate as it is stated as a range of probabilites, only I the predictor, the process of prediction, can be assessed for accuracy either from first principles or through repetition. If in the long run, occasions of my issuing a prediction of a 50% chance of rain are followed by an instance of such rain 50% of the time, the process of prediction would be accurate, well calibrated. If however it rained 60% of such times it would be nether accurate, nor well calibrated. That said, the process of prediction could still be reliable and simply in need of calibration. There is some of the flavour of that sort of recalibration in the way that simulator output can be recalibrated to enhance predictive accuracy.

    We are capable of at least one more level of abstraction, in that we may be able to refine predictions and carry out that process by a method capable of some accuracy. Given some sufficiently random process about which we have parametric uncertainty, say the flip of a coin of unknown bias, I could produce predictions accompanied by a measure of their uncertainty. For the first flip I could suggest that the chances of an instance of a head be 50% +/- 50%, an expression of complete ignorance. I could use the actual outcome to refine my second prediction, and so on. In truth I would be updating my judgement of the probability distribution of the bias parameter and be expressing a new range of probabilities based on this. In order to do so in a methodical way I need some prior notion of the relative likelihood of various values of the bias. Such a combination of prior judgement or prejudice, plus evidence is a basis for a Bayesian update. Provided I have covered all bases, e.g. not ruled out, or given a zero probability to, the actual bias; a sufficiency of evidence will eventually overwhelm my prejudice and the refining process will converge to the correct statistics, just as my assesment of the bias parameter converges to the true bias. I cannot see that there be either a correct or objective basis for such prior judgements, I cannot see that they can be anything better than a personal choice with some clearly stated rationale about which others may legitimately disagree.

    When it comes to the future of climate we are faced with projections. Personally I find this usage unsatisfying. I have no clear idea what people imply by the term projection other than their wish to avoid making predictions. That said, within the realm of a single scenario, a single non-stationary set of boundary conditions with or without some relevant starting climate state, it ought to be possible to express some form of prediction at least in the last and most abstract sense above. However that is not always what people seem to do. There are some who strive towards that end, who try and express the likelihood of various outcomes in a way I would consider to be combatible with my thinking. Unfortunately to my mind the current projections are not accurate in the sense that they accurately represent our knowledge of climate as process under changing boundary conditions. In particular I expect that they tend to underestimate our ignorance, that they favour precision of outcome over accuracy in our uncertainty of outcome.

    When I say our uncertainty, or my uncertainty, or perhaps your uncertainty, I mean our personal or collective ignorance. I do not think that the outcome is indeterminate, merely that we are ignorant of climate as process, the changing boundary conditions, and the system state of the universe. We can however make worthwhile predictions, in the sense of a best current judgement, as to our assesment of the likelihood of various outcomes, which may be refined given more evidence or more insight.

    To summarise: I would judge a prediction of future climate not on whether a particular outcome does or does not occur but on the criterion of whether I consider the expressed uncertainty of outcome is a fair representation of our collective ignorance of the process. It is a fortunate view in that I can form that judgement now rather than later.

    Alex