Probabilistic climate change projections using neural networks

Anticipated future warming of the climate system increases the need for accurate climate projections. A central problem are the large uncertainties associated with these model projections, and that uncertainty estimates are often based on expert judgment rather than objective quantitative methods. Further, important climate model parameters are still given as poorly constrained ranges that are partly inconsistent with the observed warming during the industrial period. Here we present a neural network based climate model substitute that increases the efficiency of large climate model ensembles by at least an order of magnitude. Using the observed surface warming over the industrial period and estimates of global ocean heat uptake as constraints for the ensemble, this method estimates ranges for climate sensitivity and radiative forcing that are consistent with observations. In particular, negative values for the uncertain indirect aerosol forcing exceeding –1.2 Wm^–2 can be excluded with high confidence. A parameterization to account for the uncertainty in the future carbon cycle is introduced, derived separately from a carbon cycle model. This allows us to quantify the effect of the feedback between oceanic and terrestrial carbon uptake and global warming on global temperature projections. Finally, probability density functions for the surface warming until year 2100 for two illustrative emission scenarios are calculated, taking into account uncertainties in the carbon cycle, radiative forcing, climate sensitivity, model parameters and the observed temperature records. We find that warming exceeds the surface warming range projected by IPCC for almost half of the ensemble members. Projection uncertainties are only consistent with IPCC if a model-derived upper limit of about 5 K is assumed for climate sensitivity.

We thank T. Crowley for providing the volcanic and solar radiative forcing reconstructions and S. Levitus for compiling the time series of ocean heat uptake. We enjoyed discussions with S. Gerber, N. Edwards and J. Flückiger. This work was supported by the Swiss National Foundation.

Authors and Affiliations

1. Climate and Environmental Physics, Physics Institute, University of Bern, Sidlerstrasse 5, 3012, Bern, Switzerland

   R. Knutti, T. F. Stocker, F. Joos & G.-K. Plattner

Corresponding author

Correspondence to R. Knutti.

Appendix 1: estimating the carbon cycle-climate feedback factor

To account for the uncertainty in the carbon cycle in the individual model simulations, a feedback factor γ was introduced, describing the feedback of a warming climate on the oceanic and terrestrial carbon cycle (see Sect. 2.2).

The value of the feedback factor γ and the baseline for CO₂ radiative forcing were derived separately from simulations with the Bern Carbon Cycle-Climate (Bern CC) model (Joos et al. 2001; Gerber et al. 2003). Details of the model and of the simulations are described elsewhere (Joos et al. 2001). In the Bern CC model, the physical climate system is represented by an impulse response-empirical orthogonal function substitute of the ECHAM3/LSG atmosphere/ocean general circulation model (Hooss et al. 2001) which is driven by radiative forcing. The impulse response function and the spatial patterns (EOFs) of the simulated changes in temperature, precipitation and cloud cover were derived from an 800 year long ECHAM3/LSG AOGCM simulation wherein atmospheric CO₂ was quadrupled in the first 140 years and held constant thereafter (Voss and Mikolajewicz 2001). The climate sensitivity of the substitute is prescribed. The carbon cycle module includes a well-mixed atmosphere, the HILDA ocean model (Siegenthaler and Joos 1992; Joos et al. 1996), and the Lund-Potsdam-Jena Dynamic Global Vegetation Model (LPJ-DGVM) (Sitch et al., Submitted 2002). The effect of sea surface warming on carbon chemistry is included (Joos et al. 1999), but ocean circulation changes are not considered. The LPJ-DGVM is run on a resolution of 3.75° × 2.5�� and is driven by local temperatures, precipitation, incoming solar radiation, cloud cover, and atmospheric CO₂. It simulates the distribution of nine plant functional types based on bioclimatic limits for plant growth and regeneration. Photosynthesis is a function of absorbed photosynthetically active radiation, temperature, atmospheric CO₂ concentration and an empirical convective boundary layer parameterization to couple the carbon and water cycles. After spinup, the model was driven by prescribed anthropogenic radiative forcing and observed atmospheric CO₂ for the period from 1765 to 2000. Afterwards, the atmospheric concentration of CO₂ and other agents and radiative forcing were calculated from emissions of greenhouse gases and aerosol precursors. The baseline trajectory for CO₂ forcing was calculated from simulations where the climate sensitivity was set to zero. The feedback parameter γ is calculated from the difference in CO₂ radiative forcing between simulations with and without global warming divided by the simulated temperature increase since year 2000 of the warming simulation. It varies around 0.25 W m^–2/K for a range of emission scenarios and for a range of climate sensitivities (see Fig. 10). It is noted that the feedback factor γ as derived here implicitly accounts for the climate change up to year 2000; γ would be somewhat lower when derived from simulations where emissions are prescribed over the historical period and expressed relative to the warming since preindustrial time. The uncertainty of γ has been estimated based on the simulations of different scenarios with the model described and based on other published modelling studies (Friedlingstein et al. 2000; Cox et al. 2000). The feedback strength in the IPSL model (Friedlingstein et al. 2000) is slightly lower than in the Bern CC model, whereas the Hadley centre model has a much stronger feedback (Cox et al. 2000).

Global warming-carbon cycle feedback factor γ versus global mean surface warming since year 2000 AD. The feedback factors have been evaluated for the six illustrative SRES scenarios and applying climate sensitivities of 1.5, 2.5, and 4.5 K in the Bern Carbon Cycle-Climate Model (Joos et al. 2001). Atmospheric CO₂, its forcing and radiative forcing by other agents were prescribed up to year 2000 according to observations. Afterwards, concentrations and radiative forcing were simulated from emissions of greenhouse gases and aerosol precursors. Simulations with global warming and baseline simulations without global warming (climate sensitivity set to 0 K) were performed. The feedback factors were computed from the model output for the period 2000 to 2100. First, the difference in CO₂ radiative forcing between a global warming simulation and its baseline simulation was determined; then this difference was divided by the increase in global mean surface temperature realized since year 2000 in the warming simulation. Only results for temperature changes larger than 1 K are shown, i.e. when the factors have converged to a relatively stable value

Full size image

Appendix 2: neural network design

1.1 Principles of neural networks

The research field of neural networks is relatively young and has experienced three periods of extensive activity. The first wave of interest emerged after the pioneering work of McCulloch and Pitts in 1943. The second occurred in the 1960s with the Perceptron Convergence Theorem of Rosenblatt (1962), and was damped by the work of Minsky and Papert (1969) showing the limitations of a simple perceptron. In the early eighties, new theoretical results (e.g. the Hopfield energy approach and the discovery of error back-propagation), and increased computational capacities renewed the interest in neural networks. They are now used to solve a variety of problems in pattern recognition and classification, tracking, control systems, fault detection, data compression, feature extraction, signal processing, optimization problems, associative memory and more.

There are many different types of neural networks, each suitable for specific applications. The L-layer feed-forward network consists of one input layer, (L – 2) hidden layers, and one output layer of units successively connected in a feed-forward fashion with no connections between units in the same layer and no feedback connections between layers. A sufficiently large three-layer feed-forward network can approximate any functional relationship between inputs and outputs. We use such a three-layer feed-forward network, with N = 10 units in the input and hidden layer. The number of neurons in the output layer is equal to the number of output values desired. The choice of the network size N is critical, and discussed in Appendix 1.2 for our application.

Before the neural network can perform anything reasonable, it has to go through a training or learning phase, where the connection weights w _j (and possibly also the network architecture) are continuously updated so that the network can efficiently perform a specific task. The ability of neural networks to learn automatically from examples makes them attractive and exiting. Instead of following a set of rules specified by human experts, the neural network appears to learn the underlying input–output relationship from the examples presented to it in a training set. There are numerous learning algorithms, which can be classified into supervised and unsupervised learning. In our case, a limited training set of parameter vectors \({\vec R}\) and the corresponding simulations \(C(\vec R,t)\) from the climate model are available. In this case, the supervised learning phase is the problem of minimizing the sum of the squared differences between the climate model simulations \(C(\vec R,t)\) and the corresponding substitute response \(F(\vec R,t)\) approximated by the neural network by adjusting the weights w _j in each of the neurons. For efficiency, only the desired subset of the model output \(C(\vec R,t)\) is approximated. The Levenberg-Marquardt (Hagan and Menhaj 1994) algorithm is the most efficient learning algorithm for our application due to its extremely fast convergence. A detailed introduction to neural network architectures, learning rules, training methods and applications was written by Hagan et al. (1996).

1.2 Sensitivity to neural network size

The size of the neural network, i.e. the number of neurons in the input and hidden layer, is critical for performance as well as for efficiency reasons. If the network is too small, it has too few degrees of freedom to learn the desired behaviour and will do a poor approximation of the input–output relationship. If it is too large, it will be computationally expensive to run. Further, too many neurons can contribute to "overfitting", the case in which all training samples are very well fit, but the fitting curve takes wild oscillations between these points, causing the network to lose its ability to generalize and correctly predict the output from an input it has not been trained with. One way to accomplish this task is to start with a small network and to subsequently increase the number of neurons in the layers. The mean error will drop asymptotically to zero or to an approximately constant value if the function cannot be approximated perfectly (which is the case here). This is shown in Fig. 11a, b for the mean error (difference between climate model and neural network) of the surface warming and ocean heat uptake over the observational period. Further, correlation coefficients between the approximated function and the climate model-predicted time evolution are shown (Fig. 11c, d). The choice of N = 10 neurons in the input and hidden layer is reasonable, since a larger network does not perform better, but takes significantly longer to train. The number of neurons in the input and hidden layers must not necessarily be identical. Sigmoid and linear activation functions were used in the input and hidden layer, respectively. Other activation functions yield similar or less accurate results (not shown).

Mean error and correlation coefficient between climate model output and neural network predictions for surface warming and ocean heat uptake as a function of the network size. The network consists of three layers, and there are N neurons in layer 1 and 2. The number of neurons in the output layer 3 is fixed and given by the output size

Full size image

1.3 Sensitivity to size of training set

Choosing an appropriate set of training patterns for the neural network is the second important point. Too few or poorly selected training samples will prevent the network from learning the whole input–output relationship, and only part of the presented input will be recognized correctly. Too many training samples, on the other hand, will not do any harm, but increase the computational cost dramatically. Again, the performance of the network was evaluated here for different sizes of the training sample set (Fig. 12). A training set of 500 simulations was found to be still efficient enough to handle and was chosen to ensure proper training over the whole parameter space. An interesting feature is seen in Fig. 12b, d for small numbers of training samples. The mean error does not necessarily decrease when adding more training samples, if the samples are unevenly distributed in parameter space. Adding a few samples lying close together in parameter space by coincidence causes the network to behave better in this specific region, but poorly at other locations. If the samples are randomly chosen, then only a sufficiently large sample will ensure an approximately even distribution and good performance in the whole parameter range of interest.

Mean error and correlation coefficient between climate model output and neural network predictions for surface warming and ocean heat uptake as a function of the size of the training set

Full size image

The problem of overfitting mentioned above can be tackled in several ways. Here we adopt a method called "early stopping", which ensures the generalization ability by stopping the training process before overfitting starts. This is achieved by using a separate validation set instead of the training set to test the performance of the network. Typically, the validation error will decrease during the initial phase of the training, as does the training set error. However, when the network begins to overfit the data, the error on the validation set will usually begin to rise, while the error on the training set is still decreasing. When the validation error increases for a specified number of iterations, the training is stopped, and the network weights at the minimum of the validation error are returned.

Reprints and permissions