fre.cmor.cmor_mixer module

python module housing the metadata processing routines utilizing the cmor module, in addition to click API entry points see README.md for additional information on fre cmor run (cmor_mixer.py) usage

fre.cmor.cmor_mixer.check_dataset_for_ocean_grid(ds)

checks netCDF4.Dataset ds for ocean grid origin, and throws an error if it finds one. accepts one argument. this function has no return.

ds: netCDF4.Dataset object containing variables with associated dimensional information.

fre.cmor.cmor_mixer.cmor_run_subtool(indir=None, json_var_list=None, json_table_config=None, json_exp_config=None, outdir=None, opt_var_name=None)

primary steering function for the cmor_mixer tool, i.e essentially main. Accepts six args:

indir: string, directory containing netCDF files. keys specified in json_var_list are local

variable names used for targeting specific files

json_var_list: string, path pointing to a json file containing directory of key/value

pairs. the keys are the “local” names used in the filename, and the values pointed to by those keys are strings representing the name of the variable contained in targeted files. the key and value are often the same, but it is not required.

json_table_config: json file containing CMIP-compliant per-variable/metadata for specific

MIP table. The MIP table can generally be identified by the specific filename (e.g. “Omon”)

json_exp_config: json file containing metadata dictionary for CMORization. this metadata is effectively

appended to the final output file’s header

outdir: string, directory root that will contain the full output and output directory

structure generated by the cmor module upon request.

opt_var_name: string, optional, specify a variable name to specifically process only filenames matching

that variable name. I.e., this string help target local_vars, not target_vars.

fre.cmor.cmor_mixer.cmorize_target_var_files(indir=None, target_var=None, local_var=None, iso_datetime_arr=None, name_of_set=None, json_exp_config=None, outdir=None, proj_table_vars=None, json_table_config=None)

processes a target directory/file this routine is almost entirely exposed data movement before/after calling rewrite_netcdf_file_var it is also the most hopelessly opaque routine in this entire dang macro. this badboy right here accepts… lord help us… !!!NINE!!! arguments, NINE.

indir: string, path to target directories containing netcdf files to cmorize target_var: string, name of variable inside the netcdf file to cmorize local_var: string, value of the variable name in the filename, right before the .nc

extension. often identical to target_var but not always.

iso_datetime_arr: list of strings, each one a unique ISO datetime string found in targeted

netcdf filenames

name_of_set: string, representing the post-processing component (GFDL convention) of the

targeted files.

json_exp_config: see cmor_run_subtool arg desc outdir: string, path to output directory root to move the cmor module output to, including

the whole directory structure

proj_table_vars: an opened json file object, read from json_table_config json_table_config: see cmor_run_subtool arg desc

fre.cmor.cmor_mixer.copy_nc(in_nc, out_nc)

copy target input netcdf file in_nc to target out_nc. I have to think this is not a trivial copy operation, as if it were, using shutil’s copy would be sufficient. accepts two arguments

in_nc: string, path to an input netcdf file we wish to copy out_nc: string, an output path to copy the targeted input netcdf file to

fre.cmor.cmor_mixer.create_tmp_dir(outdir)

creates a tmp_dir based on targeted output directory root. returns the name of the tmp dir. accepts one argument:

outdir: string, representing the final output directory root for the cmor modules netcdf

file output. tmp_dir will be slightly different depending on the output directory targeted

fre.cmor.cmor_mixer.get_iso_datetimes(var_filenames, iso_datetime_arr=None)

appends iso datetime strings found amongst filenames to iso_datetime_arr.

var_filenames: non-empty list of strings representing filenames. some of which presumably

contain datetime strings

iso_datetime_arr: list of strings, empty or non-empty, representing datetimes found in

var_filenames entries. the objet pointed to by the reference iso_datetime_arr is manipulated, and so need-not be returned

fre.cmor.cmor_mixer.get_var_filenames(indir, var_filenames=None, local_var=None)

appends files ending in .nc located within indir to list var_filenames accepts three arguments

indir: string, representing a path to a directory containing files ending in .nc extension var_filenames: list of strings, empty or non-empty, to append discovered filenames to. the

object pointed to by the reference var_filenames is manipulated, and so need not be returned.

local_var: string, optional, if not None, will be used for ruling out filename targets

fre.cmor.cmor_mixer.get_vertical_dimension(ds, target_var)

determines the vertical dimensionality of target_var within netCDF4 Dataset ds. accepts two arguments and returns an object represnting the vertical dimensions assoc with the target_var.

ds: netCDF4.Dataset object containing variables with associated dimensional information. target_var: string, representating a variable contained within the netCDF4.Dataset ds

fre.cmor.cmor_mixer.rewrite_netcdf_file_var(proj_table_vars=None, local_var=None, netcdf_file=None, target_var=None, json_exp_config=None, json_table_config=None)

rewrite the input netcdf file nc_fl containing target_var in a CMIP-compliant manner. accepts six arguments, all required:

proj_table_vars: json dictionary object, variable table read from json_table_config. local_var: string, variable name used for finding files locally containing target_var,

this argument is often equal to target_var.

netcdf_file: string, representing path to intput netcdf file. target_var: string, representing the variable name attached to the data object in the netcdf file. json_exp_config: string, representing path to json configuration file holding metadata for appending to output

this argument is most used for making sure the right grid label is getting attached to the right output

json_table_config: string, representing path to json configuration file holding variable names for a given table.

proj_table_vars is read from this file, but both are passed anyways.