In this next section on air concentration statistics, we will look at a very special version of one of the concentration utility programs.  As with the other sections, it is important that you have completed the simulation for CAPTEX release number two in section 8.1.  If you have not created those output files you should do that now, before you continue on with the air concentration statistics.  

The statistical programs that are packaged together here are essentially an extension of the utility program Convert to Station.  But instead of just doing one station, we will convert all the stations that have measured data for the CAPTEX experiment.  In this utility program, or this menu tab is Convert to DATEM, and this essentially uses the Convert to Station, but going through at least a version of the program, and then it goes through each of the measured data sites and computes the value from the model output.  So let’s take a look at what we are doing here.  We need to identify the measured data file by using the browse button and we already know that the measured data for CAPTEX release number two are in this file, so select that, and we're going to use the same conversion factors before, 1×1012, to give us units of picograms.  If I were to take a look at that measured data set, the values in here are already in picograms, you would have to check the reports to know this for sure.  And this particular format and the reason why this menu tab is called Convert to DATEM is that the measured data have been converted to a common format called the Data Archive of Tracer Experiments and Meteorology. And many of the tracer experiments that are available for model verification and testing have been converted to this format by NOAA ARL. 

This format is relatively simple; it has two identification records, and then the data records.  Each one are independent of the others, it's the starting time of the sample in a year, month, day, hour, minutes, and the duration of the sample and hour minutes, and the location of the sampler, then the value of the concentration, the measured concentration, in this case picograms per cubic meter, and then the station identification number.  So this is the DATEM format and on the ARL website you can find many other experiments that have been converted to this format.  

So as I said this was a, this is a special version of the Convert to Station program, where we're going to create this DATEM file and it will be called hysplit.txt and we're going to use the bilinear interpolation method.  You can either select nearest neighbor or bilinear.  And when we click on create DATEM file, is what we have done, is in the working directory, we created this file called hysplit.txt,  which is identical in format to the measured data, except this is the predictions, the HYSPLIT simulation using the North American Regional Reanalysis data, to correspond with each sample. So the program essentially looks at the gridded binary output file and interpolates a concentration from that file and writes out this output file, so that there is one record that corresponds with each record in the measured data file.  And this program also properly manages the three hour and six hour samples.  As you recall, the predicted concentrations, the model computed concentrations, were all three our averages.  So in this case, for this particular sample, it would be a one to one correspondence between measurement and model prediction.  However for the six hour samples, the program, the Convert to DATEM program, will take 2 three-hour samples and merge them together to create a six hour average duration sample.  

Now once you've done this conversion, now it is possible to compute statistics.  So this particular conversion program is tied together with the statistical program and we will do the verification statistics on all values.  But we're not to do any averaging, spatial and temporal, so it'll just be a paired in space and time comparison of each measured value with each model predicted value.  And the contingency level is such that concentrations below 10 pg/m³ are treated as zero.  We will compute statistics and this shows the performance of the model.  There were 395 pairs of predicted and measured concentrations that were analyzed and there are various parameters here, for instance 0.91 is the ratio of the calculated to measured concentration, the figure of merit in space shows us how much overlap there was between the measured and the calculated predictions, spatial overlap that is.  We had almost 30% within a factor of 10 of each other and there're also unpaired statistics, where we compare the cumulative distributions between the measured and the calculated. So for instance the 95th percentile concentration that was measured was 1450 picograms, whereas the 95th percentile calculated concentration was 1170. And the Kolmogorov-Smirnov parameter is just a comparison, or the maximum distance between the calculated and measured cumulative probability distributions, the farther, the larger this number, the farther these distributions are from each other.  And the final rank is a screening number that combines the correlation coefficient, the fractional bias, the figure of merit in space, and the Kolmogorov-Smirnov parameter, into one number.  That's discussed; you can look at the help file for more information on how this is computed.

And then we can also generate a scatter plot from these results. Now having this together as a single package is quite useful and it permits us to do various sensitivity tests.  So we can see here in the working directory, we left some files as a result of this, the text file here was the DATEM formatted model predictions, the statA file were the statistical results, and the dataA file was the measured and calculated values in DATEM format paired together which was then used by the plotting program. 

This particular procedure is outlined as well, when you run this through the graphical user interface, is actually running a TCL script, and you can see the steps that are involved by looking at either the batch file or the LINUX script, where we show how the model is run, the HYSPLIT model, and then we convert the output to the DATEM format from the measured data to an output file called hysplit.txt using this conversion factor, and then we run the statistics on the model predictions and measured data, and then we generate a scatter diagram.  So if you have any questions at any point in how these calculations are done, remember you can always go to the scripts. 

Now the second step of this, one of the advantages of having the package of statistics and model together here, is we can do some sensitivity tests.  So a simple experiment would be, let's run this CAPTEX tracer release number two again, but this time let's include some additional high resolution meteorology.  Now if you still have the release information in the graphical user interface, that's great, if not, go ahead and retrieve the file,  which was called captex_control.txt and also on the advanced menu, retrieve captex_setup.txt.  Let’s just do something very simple, let's just include an additional meteorological file, a high resolution meteorological file.  Now do not press the clear button, but we’re just going to add meteorology file here.  I'm looking to add the 3 km resolution WRF file which is available for at least the first part of the period.  And by clicking on open we've added this to the list.  Now when HYSPLIT has more than one meteorological file available at a particular computational point, that is as we are following a particle, at each advection step the model looks to see which meteorological file will be best to use, if there are multiple be meteorological files defined.  And if there are, then it will use the one with the highest spatial resolution, or if there are comparable files, it might use the one with the least forecast hour, it would use analysis data before it would use forecast data for instance.  So at this point save, and run model. Now this could take quite some time, so I'm going to pause while this runs and you can go get a cup of coffee perhaps.

The calculation has finally completed, so we can go ahead and do the conversion and statistics just like we did for the original calculation.  But before I do that, let's just rename the previous scatter diagram and the previous statistical results file using the NARR abbreviation, just so they're not overwritten.  And if you recall, the next step would be utilities Convert to DATEM and we can leave everything in the menu the way it is; we create a new DATEM file, we compute the statistics, and we create a scatter plot. 

And comparing this plot with the original, you can clearly see there are many more concentrations along the one-to-one line when we used two additional high resolution meteorological data.  Now this meteorological data did not apply for the entire calculation, and I will show you that in just a moment. Also the statistics, if we're to look at that, the new statistics compared to the old statistical file, so the new one, it was .65 correlation, up from .57, the ratio of mean, calculated to measured means, was not as good, it went to .74.  The bias is a little higher as well.  There was some improvement in the figure of merit in space and there was an overall improvement in the final rank, somewhat higher.  So some of the statistical metrics, some went up, and some went down.  Overall it was a slight improvement.

The way you know what happened is if you go to the MESSAGE file, you can see now that there are two major meteorological models defined, the WRF and the North American Regional Reanalysis and the way you know what happens is that you can see at least an initial phase of the calculation the model is reading the WRF data,  and somewhere, perhaps at the 12 hour point, it is now reading both the WRF and the NARR data, because the WRF domain does not cover, the WRF models does not cover the full domain of the calculation.  And so at this point there are particles that are on the finer WRF domain but there are other particles that are only on part of the domain that’s only covered by the NARR data.  

Now the WRF data is more frequent, so there are more reads of the WRF data. In fact the WRF output file, I believe is every 15 minutes, so there are four WRF reads per hour.  But in any event, at the end of the simulation, notice we closed unit number one, so at 21 hours there are no more WRF data available.  The rest of the calculation is done with the NARR data. 

And that concludes our discussion of the Convert to DATEM option and the automatic statistics can be generated.