$CTSMROOT/tools/README					Jun/08/2018

CTSM tools for analysis of CTSM history files -- or for creation or
modification of CTSM input files.
	
I.  General directory structure:

    $CTSMROOT/tools
	mksurfdata_map --- Create surface datasets.

        mkmapgrids ------- Create regular lat/lon SCRIP grid files needed by mkmapdata 
        mkmapdata -------- Create SCRIP mapping data from SCRIP grid files (uses ESMF)
        mkprocdata_map --- Convert output unstructured grids into a 2D format that
                           can be plotted easily
        ncl_scripts ------ NCL post or pre processing scripts.

        site_and_regional  Scripts for handling input datasets for site and regional cases.
                           These scripts both help with creation of datasets using the 
                           standard process as well as subsetting existing datasets and overwriting
                           some aspects for a specific case.

        contrib ---------- Miscellaneous tools for pre or post processing of CTSM.
                           Typically these are contributed by anyone who has something
                           they think might be helpful to the community. They may not
                           be as well tested or supported as other tools.

    cime-tools ($CIMEROOT/tools/) (CIMEROOT is ../cime for a CTSM checkout and ../../../cime for a CESM checkout)
        $CIMEROOT/mapping/gen_domain_files
            gen_domain ------- Create data model domain datasets from SCRIP mapping datasets.

II. Notes on building/running for each of the above tools:

    Each tool that has FORTRAN source code (mksurfdata_map and mkprocdata_map) has the following files:

	README ------- Specific help for using the specific tool and help on specific 
                       files in that directory.
	src/Filepath ----- List of directories needed to build the tool 
                           (some files in ../src directories are required).
	src/Makefile ----- GNU Makefile to build the tool 
                           (these are identical between tools.
	src/Macros.custom  Customization of make macros for the particular tool in question
	src/Srcfiles ----- List of source files that are needed.
        src/Mkdepends ---- Dependency generator program

    mkmapdata, mkmapgrids and ncl_scripts only contain scripts so don't have the above build files.

    Most tools have copies of files from other directories -- see the README.filecopies
    file for more information on this.

    Tools may also have files with the directory name followed by namelist to provide sample namelists.

	<directory>.namelist ------ Namelist to create a global file.

    These files are also used by the test scripts to test the tools (see the
    README.testing) file.

    NOTE: Be sure to change the path of the datasets references by these namelists to 
    point to where you have exported your CESM inputdata datasets.

    To build:

	cd <directory>
        setenv INC_NETCDF <path-to-NetCDF-include-files>
        setenv LIB_NETCDF <path-to-NetCDF-library-files>
	gmake

    The process will create a file called "Depends" which has the dependencies
    for the build of each file on other files.

      By default some codes may be compiled non-optimized 
      so that you can use the debugger, and with bounds-checking, and float trapping on. 
      To speed up do the following...

   gmake OPT=TRUE  (by default already on for mksurfdata_map)

      Also some of the tools allow for OpenMP shared memory parallelism 
      (such as mksurfdata) with

   gmake SMP=TRUE

    To run a program with a namelist:

        ./program < namelist

    To run a program built with SMP=TRUE:

        setenv OMP_NUM_THREADS=<number_of_threads_to_use>

        run normally as above

III. Process sequence to create input datasets needed to run CTSM

    1.) Create SCRIP grid files (if needed)

       a.) For standard resolutions these files will already be created. (done)

       b.) To create regular lat-lon regional/single-point grids run site_and_regional/mknoocnmap.pl

        This will create both SCRIP grid files and a mapping file that will
        be valid if the region includes NO ocean whatsoever (so you can skip step 2).
        You can also use this script to create SCRIP grid files for a region
        (or even a global grid) that DOES include ocean if you use step 2 to
        create mapping files for it (simply discard the non-ocean map created by
        this script).

        Example, for single-point over Boulder Colorado.

           cd site_and_regional
           ./mknoocnmap.pl -p 40,255 -n 1x1_boulderCO

       c.) General case

        You'll need to convert or create SCRIP grid files on your own (using scripts
        or other tools) for the general case where you have an unstructured grid, or 
        a grid that is not regular in latitude and longitude.

       example format
         ==================	
          netcdf fv1.9x2.5_090205 {				   
          dimensions:						   
               grid_size = 13824 ;				   
               grid_corners = 4 ;				   
               grid_rank = 2 ;					   
          variables:						   
               double grid_center_lat(grid_size) ;		   
                       grid_center_lat:units = "degrees" ;	   
               double grid_center_lon(grid_size) ;		   
                       grid_center_lon:units = "degrees" ;	   
               double grid_corner_lat(grid_size, grid_corners) ;  
                       grid_corner_lat:units = "degrees" ;	   
               double grid_corner_lon(grid_size, grid_corners) ;  
                       grid_corner_lon:units = "degrees" ;	   
               int grid_dims(grid_rank) ;			   
               int grid_imask(grid_size) ;			   
                       grid_imask:units = "unitless" ;            

    2.) Create ocean to atmosphere mapping file (if needed)

        a.) Standard resolutions (done)

        If this is a standard resolution with a standard ocean resolution -- this
        step is already done, the files already exist.

        b.) Region without Ocean (done in step 1.b)

        IF YOU RAN mknoocnmap.pl FOR A REGION WITHOUT OCEAN THIS STEP IS ALREADY DONE.

        c.) New atmosphere or ocean resolution

        If the region DOES include ocean, use $CIMEROOT/tools/mapping/gen_domain_files/gen_maps.sh to create a 
        mapping file for it.

    Example:

    cd $CIMEROOT/tools/mapping/gen_domain_files
    ./gen_maps.sh -focn <ocngrid> -fatm <atmgrid> -nocn <ocnname> -natm <atmname>


    3.) Add SCRIP grid file(s) created in (1) into XML database in CTSM (optional)

        See the "Adding New Resolutions or New Files to the build-namelist Database" 
        Chapter in the CTSM User's Guide

  http://www.cesm.ucar.edu/models/cesm1.0/clm/models/lnd/clm/doc/UsersGuide/book1.html

         If you don't do this step, you'll need to specify the file to mkmapdata 
         in step (3) using the "-f" option.

    4.) Create mapping files for use by mksurfdata_map with mkmapdata
        (See mkmapdata/README for more help on doing this)

       - this step uses the results of (1) that were entered into the XML database
         by step (3). If you don't enter datasets in, you need to specify the 
         SCRIP grid file using the "-f" option to mkmapdata.sh.

       Example: to generate all necessary mapping files for the ne30np4 grid

           cd mkmapdata
           ./mkmapdata.sh -r ne30np4

    5.) Add mapping file(s) created in step (4) into XML database in CTSM (optional)

       See notes on doing this in step (3) above. 
       Edit ../bld/namelist_files/namelist_defaults_clm.xml to incorporate new 
       mapping files.

       If you don't do this step, you'll need to specify the grid resolution name
       and file creation dates to mksurfdata_map in step (5) below.

    6.) Convert map of ocean to atm for use by DATM and CTSM with gen_domain
        (See $CIMEROOT/tools/mapping/README for more help on doing this)

       - gen_domain uses the map from step (2) (or previously created CESM maps)

       Example:

        cd $CIMEROOT/tools/mapping/gen_domain_files/src
        gmake
        cd ..
        setenv CDATE       090206
        setenv OCNGRIDNAME gx1v6
        setenv ATMGRIDNAME fv1.9x2.5
        setenv MAPFILE $CSMDATA/cpl/cpl6/map_${OCNGRIDNAME}_to_${ATMGRIDNAME}_aave_da_${CDATE}.nc
        ./gen_domain -m $MAPFILE -o $OCNGRIDNAME -l $ATMGRIDNAME

        Normally for I compsets running CTSM only you will discard the ocean domain 
        file, and only use the atmosphere domain file for datm and as the fatmlndfrc 
        file for CTSM. Output domain files will be named according to the input OCN/LND
        gridnames.

    7.) Create surface datasets with mksurfdata_map
        (See mksurfdata_map/README for more help on doing this)

       - Run mksurfdata_map/mksurfdata.pl
       - This step uses the results of step (4) entered into the XML database
         in step (5).
       - If datasets were NOT entered into the XML database, set the resolution
         to "usrspec" and use the "-usr_gname", and "-usr_gdate" options.

       Example: for 0.9x1.25 resolution

       cd mksurfdata_map/src
       gmake
       cd ..
       ./mksurfdata.pl -r 0.9x1.25
   
       NOTE that surface dataset will be used by default for fatmgrid - and it will 
       contain the lat,lon,edges and area values for the atm grid - ASSUMING that 
       the atm and land grid are the same 

    8.) Add new files to XML data or using user_nl_clm (optional)

       See notes on doing this in step (3) above. 

IV. Example of creating single-point datasets without entering into XML database.

    Here we apply the process described in III. for a single-point dataset
    where we don't enter the datasets into the XML database (thus skipping
    steps 3, 5 and 8), but use the needed command line options to specify where the
    files are. This also skips step (2) since step 1 creates the needed mapping file.

    0.) Set name of grid to use and the creation date to be used later...
       setenv GRIDNAME 1x1_boulderCO
       setenv CDATE    `date +%y%m%d`
    1.) SCRIP grid and atm to ocn mapping file
       cd site_and_regional
       ./mknoocnmap.pl -p 40,255 -n $GRIDNAME
       # Set pointer to MAPFILE that will be used in step (6)
       setenv MAPFILE `pwd`/map_${GRIDNAME}_noocean_to_${GRIDNAME}_nomask_aave_da_${CDATE}.nc
       cd ../..
    2.) skip
    3.) skip
    4.) Mapping files needed for mksurfdata_map
       cd mkmapdata
       setenv GRIDFILE ../mkmapgrids/SCRIPgrid_${GRIDNAME}_nomask_${CDATE}.nc
       ./mkmapdata.sh -r $GRIDNAME -f $GRIDFILE -t regional
       cd ../..
    5.) skip
    6.) Generate domain file for datm and CTSM
        cd $CIMEROOT/tools/mapping/gen_domain_files/src
        gmake
        cd ..
        setenv OCNDOM domain.ocn_noocean.nc
        setenv ATMDOM domain.lnd.{$GRIDNAME}_noocean.nc
        ./gen_domain -m $MAPFILE -o $OCNDOM -l $ATMDOM
    7.) Create surface dataset for CTSM
       cd mksurfdata_map/src
       gmake
       cd ..
       ./mksurfdata.pl -r usrspec -usr_gname $GRIDNAME -usr_gdate $CDATE
    8.) skip

V.  Notes on which input datasets are needed for CTSM

       global or regional/single-point grids
         - need fsurdata and fatmlndfrc

      fsurdata ---- from mksurfdata_map in step (III.7)
      fatmlndfrc -- use the domain.lnd file from gen_domain in step (III.6)