Grading System for evaluating transport (i.e. residual circulation + mixing) in the UT/LS using aircraft N2O and CO2. S.E. Strahan I. N2O Grading System The purpose of an N2O-based grading system is to evaluate a model's residual circulation and mixing in the LS. This grading system compares model N2O profiles with an N2O climatology created from aircraft measurements. The details of the climatology are given at the end of this section. The climatology contains 11 mean N2O profiles which cover 3 latitude ranges and 4 seasons (=max of 12 profiles). One mean profile is missing because there are no measurements of N2O in the SH midlatitudes during summer (DJF); hence there are 11 profiles in the climatology. Models are scored by their ability to reproduce these mean profiles. Model means are calculated for the same seasons, latitudes, and potential temperatures as are the observations. Because of the coarseness of model vertical resolution compared to the aircraft measurements, model results will be binned using 20K wide bins. The N2O climatology covers the range 380-500 K, therefore there are 6 vertical bins in the model profiles. In order NOT to judge the models by their tropospheric transport in this test, model results will be scaled so that their mixing ratio entering the tropical lower stratosphere (380 K) is equal to the observed 380 K mixing ratio in the tropics. There are 11 mean profiles and up to 6 bins per profile (only 5 bins in the tropical profiles because the max theta measured is 480 K) This gives a total of 62 locations for model/measurements comparisons (one location= one latitude range, one season, one 20 K wide vertical range). At each location, a model can earn 0, 1/2, or 1 point for its ability to reproduce the climatology. The proposed scoring system is as follows: 1 point: model mean mixing ratio is within 1 climatological std dev of the climatological mean. 1/2 point: the model mean +/- 1 model std dev is within 1 climatological std dev of the climatological mean. 0 points: the model mean +/- 1 model std dev is not within 1 climatological std dev of the climatological mean. Here is an example: N2O (ppb) model mean model std dev climat. mean climat std dev GRADE ----------------------------------------------------------------------- 255 15 245 15 1 pt 200 15 180 15 0.5 pt 175 15 140 15 0 pt A maximum of 62 points is possible. Of course, these results must be normalized to the desired scale. The details of the climatology are as follows. It is created from 126 ER-2 flights measuring ATLAS N2O since Dec. 1988. The goal of the climatology is to determine a mean N2O profile in the LS for each of 4 seasons and 3 latitudes ranges. The three latitude ranges are 35-50 S (SH midlatitudes), 10S-10N (deep tropics), and 34-54 N (NH midlatitudes). The most poleward latitudes were chosen so that they do not include vortex air. The four seasons are DJF, MAM, JJA, and SON. Therefore, this climatology has a maximum of 12 mean profiles (3 lats x 4 seasons) from 380-500 K. DJF in the SH is not sampled by the ER2. Almost all Flights from AAOE and AASE were excluded because they did almost no sampling equatorward of 55 degrees. Equatorial sampling during AAOE had calibration problems. First, each flight's data is multiplied by a scale factor to bring the mixing ratios to 1997 values (e.g. 315 ppb in the troposphere). This is necessary for the purpose of binning together flight data from different years because of the 0.2% increase per year in N2O. (In practice this has almost no effect on the mixing ratios!) Using 10 second averages of ATLAS N2O and MMS potential temperature and latitude, the N2O from each flight is binned in 5 K wide bins between 380 and 500 K and an average profile is calculated for each latitude region for each flight. All flights are averaged together to get the mean and standard deviation for each season, latitude region, and theta. This results in 11 mean N2O profiles. Plots of these mean profiles and their standard deviations can be found on our anonymous ftp site: hyperion.gsfc.nasa.gov ; cd to pub/gmi ; file is n2o_all_seasons.ps. On these figures, the average number of flights that went into each bin between 400 and 480 K is given in each panel. A subtropical mean profile is included in these figures but is not used in the grading system. **************************** II. CO2 Grading System The purpose of a CO2-based grading system is to evaluate 1) the pathway of model transport between the UT and LS, 2) the ability of the model to keep the UT and LS separate, and 3) the rate of mixing out from the tropics to the NH midlatitudes. Models will be compared to an analysis of CO2 measurements from the ER-2 Models will be compared to an analysis of ER-2 CO2 measurements. (This observational analysis is presented in Strahan et al.: "The CO2 Seasonal Cycle as a Tracer of Transport", submitted to JGR and available by anon ftp upon request.) There are 3 CO2 tests. Each is worth 1 point. (The CO2 score and the N2O score will need to be normalized to the same scale.) Test 1. The separation of the troposphere and stratosphere based on the (fairly) abrupt phase shift of the CO2 cycle going from the UT to the LS. (addresses points 1 and 2 above) Test 2. The maximum altitude of the CO2 extratropical seasonal cycle. (addresses point 3 above) Test 3. The phase (time) lag between the tropical CO2 cycle and the extra- tropical cycle on roughly the same isentropic surface. (addresses points 1 and 3 above) Details of Test 1 ----------------- The model tropopause will be identified from model temperature fields. The standard definitions of the temperature minimum and/or the decrease in lapse rate to less than 2K/km will be applied to determine the tropopause. For models with fairly high (~1 km) vertical resolution, the two model levels nearest (i.e. straddling) the temperature minimum will be considered the 'tropopause' levels. For the case of coarser (>=2 km) vertical resolution, the level nearest the temp minimum will be used as the sole tropopause level. The model level just above the tropopause will be used as the 'purely stratospheric' level and the level just below will be the 'purely tropospheric' level. The aircraft observations of Nakazawa et al. (1991) showed that near 60N, the UT had a May maximum in the CO2 seasonal cycle and the LS (10-12 km) had a September maximum. Therefore, we will look for a MINIMUM difference of 2 months between the maxima of the lowest stratospheric level and the highest purely tropospheric level. This is a pass/fail test. The model gets either 0 or 1 point. Details of Test 2 ----------------- This will test the rate of mixing out from the tropics by comparison with the observed mixing out of the CO2 seasonal cycle. Boering et al [1996] and Strahan et al [1997, submitted] have both found an extratropical seasonal cycle between 380 and 440 K that appears to be transported from the tropics. Above ~440 K, no clear CO2 seasonal cycle is observed in the NH midlatitudes. As in the above referenced papers, the CO2 seasonal cycle is determined on N2O surfaces. This is necessary to eliminate the component of the cycle due to the residual circulation (i.e., temperature changes in the lower stratosphere that cause theta levels to shift up and down). Taking into account the uncertainties of the observational analysis, to pass the test, a model must not have an extratropical seasonal cycle at >=460 K that is greater than 20% of the tropical seasonal cycle just above the tropopause (380 K). A model would fail this test by not having a seasonal cycle at 420 K in the extratropics that is at least 20% of the tropical seasonal cycle at 380 K. For clarity of results, the determination of a seasonal cycle will use detrended model output (annual trend is ~1.4 ppm/yr). For the purposes of test 2 and 3, the tropics are defined at 10S-10N and the NH midlatitudes are 34-54N (same definitions used in the N2O grading system). This too is a pass/fail test. A model gets either 0 or 1 point. Details of Test 3 ----------------- This tests the pathway of model transport by comparing the timing of the arrival of the tropical CO2 maximum with the arrival of the extratropical CO2 maximum on roughly the same isentropic surface. (This test uses the same CO2-binned-by- N2O data set used above.) The observational analyses of Boering et al [1996] and Strahan et al. [1997, submitted] show at least a 2 week delay between the tropical and extratropical maximum, which suggests transport of the cycle FROM the tropics. Obviously, an extratropical CO2 maximum arriving before the tropical CO2 maximum suggests an unrealistic pathway of transport in the model. In the observational analysis of Strahan et al., 3 altitude (N2O) ranges in the tropics exported a CO2 seasonal cycle to the extratropics. These altitude ranges spanned approx. 380-460 K. The model tropical CO2 will also be binned into 3 ranges covering approximately the 380-460 K range (using whichever model N2O values that are appropriate to this range). 3 points will be possible, one for each N2O range compared. A model will get a full point for an extratropical seasonal maximum that arrives 0.5-2 months after the tropical seasonal maximum. Simultaneous arrival gets 0.5 point. An extratropical maximum preceding the tropical maximum gets 0 points. Summary of the CO2 Scoring -------------------------- Each CO2 test will be normalized so as to be weighted equally.