fre.cmor.cmor_mixer module

python module housing the metadata processing routines utilizing the cmor module, in addition to click API entry points see README.md for additional information on fre cmor run (cmor_mixer.py) usage

fre.cmor.cmor_mixer.cmor_run_subtool(indir=None, json_var_list=None, json_table_config=None, json_exp_config=None, outdir=None, run_one_mode=False, opt_var_name=None)

Primary steering function for the other routines in this file, i.e essentially main.

Parameters:

indir: string, directory containing netCDF files. keys specified in json_var_list are local

variable names used for targeting specific files.

json_var_list: string, points to a json file containing directory of key/value pairs. keys are “local” names

used in the filename. values are strings representing the name of the data stored within the file. the key and value are often the same. making the key and value different is helpful for focusing on a specific file.

json_table_config: string, points to a json file containing per-variable-metadata for specific MIP table. json_exp_config: string, points to a json file containing metadata dictionary for CMORization. this metadata

is effectively appended to the final output file’s header.

outdir: string, directory root that will contain the full output and directory structure generated by

the cmor module.

run_one_mode: bool, optional, for every variable in the list, if files are found, process one of them, then

move on. largely of interest when debugging.

opt_var_name: string, optional, specify a variable name to specifically process only files with that variable.

Note that this is checked against the variable name that’s usually embedded in the nc filename.

Returns:

int: 0 if successful.

fre.cmor.cmor_mixer.cmorize_all_variables_in_dir(vars_to_run, indir, iso_datetime_arr, name_of_set, json_exp_config, outdir, mip_var_cfgs, json_table_config, run_one_mode)

Processes all variables in the given directory.

Parameters:

vars_to_run: dictionary, local-variable:target-variable pairs to process. indir: string, path to directory containing netCDF files. iso_datetime_arr: list of strings, ISO datetime strings found in targeted netCDF filenames. name_of_set: string, representing the post-processing component (GFDL convention) of the targeted files. json_exp_config: string, path to experiment configuration JSON file. outdir: string, path to output directory root. mip_var_cfgs: json dictionary object, variable table read from json_table_config. json_table_config: string, path to table configuration JSON file. run_one_mode: bool, if True, process only one file per variable.

Returns:

int: 0 if successful.

fre.cmor.cmor_mixer.cmorize_target_var_files(indir=None, target_var=None, local_var=None, iso_datetime_arr=None, name_of_set=None, json_exp_config=None, outdir=None, mip_var_cfgs=None, json_table_config=None, run_one_mode=False)

processes a target directory/file this routine is almost entirely exposed data movement before/after calling rewrite_netcdf_file_var it is also the most hopelessly opaque routine in this entire dang macro. this badboy right here accepts… lord help us… !!!NINE!!! arguments, NINE.

indir: string, path to target directories containing netcdf files to cmorize target_var: string, name of variable inside the netcdf file to cmorize local_var: string, value of the variable name in the filename, right before the .nc

extension. often identical to target_var but not always.

iso_datetime_arr: list of strings, each one a unique ISO datetime string found in targeted

netcdf filenames

name_of_set: string, representing the post-processing component (GFDL convention) of the

targeted files.

json_exp_config: see cmor_run_subtool arg desc outdir: string, path to output directory root to move the cmor module output to, including

the whole directory structure

mip_var_cfgs: an opened json file object, read from json_table_config json_table_config: see cmor_run_subtool arg desc

fre.cmor.cmor_mixer.rewrite_netcdf_file_var(mip_var_cfgs=None, local_var=None, netcdf_file=None, target_var=None, json_exp_config=None, json_table_config=None, prev_path=None)

Rewrite the input netcdf file containing target_var in a CMIP-compliant manner.

Parameters:

mip_var_cfgs: json dictionary object, variable table read from json_table_config. local_var: string, variable name used for finding files locally containing target_var. netcdf_file: string, representing path to input netcdf file. target_var: string, representing the variable name attached to the data object in the netcdf file. json_exp_config: string, representing path to configuration file holding metadata for appending to output. json_table_config: string, representing path to configuration file holding variable names for a given table.

Returns:

filename: The name of the file returned by cmor.close.