For developers
Developers should consult this section for detailed and specific information relevant to development/maintenance efforts, AFTER familiarizing themselves with the rest of the user-targeted documentation.
Contributing to fre-cli
Get a Copy of the Code
Get your own copy of the code with git clone --recursive git@github.com:NOAA-GFDL/fre-cli.git
for the NOAA-GFDL fork,
or replace with your fork’s link (recommended).
Local/Editable Installation
Developers can test local changes by running a pip install [-e] .
inside of the root directory after activating a
virtual environment with python>=3.11.*
and all requirements. This installs the fre-cli
package locally with
any local changes.
Development work on fre-cli
should occur within a conda environment housing fre-cli
’s requirements, and
a local copy of the repository to install with pip
using the -e/--editable
flag on. This specific approch is
described here
Testing Your Local Changes and Installation
There are a myriad of different ways of testing your efforts locally during your development cycle. A few examples and
fre-cli
specific recommendations are described here, but contributors are welcome to be creative so-long as they
provide documentation of what they have contributed.
All contributed code should come with a corresponding unit-test. This section is not about writing unit-tests, but does contain some advice towards it. This section is mostly for streamlining a new developer’s approach to working with the code.
Running CLI-calls
Most development cycles will involve focused efforts isolated to a specific fre TOOL COMMAND *ARGV
, where *ARGV
stands
in for a shell-style argument vector (e.g. -e FOO -p BAR -t BAZ
, a common pattern in fre-cli
). Likely, the code one
is working on here is housed somewhere approximately like fre/TOOL/COMMAND.py
(generally, this is not a law), with
the click
CLI entry-point under fre/TOOL/freTOOL.py
.
Here, the developer usually uses the fre TOOL COMMAND *ARGV
call as a test, focused on seeing the changes they are
introducing, and develop the code until they see the result they are looking for. The specific fre TOOL COMMAND *ARGV
should/can often become a unit-test in one of the corresponding files in fre/tests
. The sought-after changes the
devleloper wished to introduce should become assert
conditions encoded within the unit-test. Both success and failure
conditions should ideally be tested.
Running the above, with no click
Every fre TOOL COMMAND *ARGV
approximately maps to a single function call shown in fre/TOOL/freTOOL.py
at this time.
Then, to accomplish the same thing as the previous section, but removing click
and the CLI-aspect from it, and assuming
the code being executed is in fre/TOOL/COMMAND.py
, in a function called named like FUNCTION
,
python -i -c 'from fre.TOOL.COMMAND import FUNCTION; FUNCTION(**args);
.
Writing a pytest
unit-test for fre-cli
If the functionality one desires to test is that of CLI call, the tests should use the CliRunner
approach shown in
fre/tests
. click
based CLI calls should NOT be tested with subprocess.run
ever within pytest
. See
click
’s documentation for more
information.
If the functionality one desires to test is removed from that of a CLI call, the test should likely be housed in the directory
structure corresponding to the TOOL
under-which the functionality lives. In that case, the usual pythonic-testing approaches,
guidelines, documentation etc. applies.
Adding a New Requirement to fre-cli
Currently, all required packages are conda
packages listed in environment.yml
, and also, equivalently in meta.yaml
.
conda
packages that have a corresponding pip
package should list the pip
package as a python requirement in setup.py
.
Pure pip
packages cannot be listed currently as a requirement for fre-cli
. This is because only environment.yml
can list
pip
packages as requirements. But, only meta.yaml
can be used as a the conda build
target. New dependencies for fre-cli
MUST have a conda
package available through a non-proprietary conda
channel, but preferable the open-source conda-forge
channel, which requires stronger quality control.
In general, the requirement being added is created by a third-party. As such, before adding a new requirement, the developer is
responsible for verifying that the desired package is safe, well-documented, and actively-maintained as necessary. The developer should
also consider the cost-benefit-problem of taking the extra time to introduce new functionality via standard-library approaches first,
and be prepared to defend the proposition of adding the new third-party package as a fre-cli
requirement.
How fre-cli
is updated
fre-cli
is published and hosted as a Conda package on the NOAA-GFDL conda channel. On pushes to
the main
branch, the package located at https://anaconda.org/NOAA-GFDL/fre-cli will automatically be updated using the workflow
defined in .github/workflows/publish_conda.yml
, which is equivalent to``.github/workflows/build_conda.yml`` with an extra conda publish
step.
Get desired logging
verbosity
The logging
module’s configuration initially occurs in fre/__init__.py
, and gets inherited everywhere else logging
creates a logger
object under the fre.
namespace. If your development is being tested with a fre TOOL COMMAND *ARGV
style CLI call, it’s recommended you add verbosity flags, i.e. like fre -vv TOOL COMMAND *ARGV
.
If your development does not fit nicely into that category, the next easiest thing to do is to adjust the base logger
object
in fre/__init__.py
to have the verbosity level you’d like. It’s important you adjust it back to the default verbosity level of
fre-cli
before requesting a merge of your branch/fork to the repository’s trunk.
logging
practice to avoid
The pitfall to avoid during development is calling logging.basicConfig
to re-configure the logging
behavior OUTSIDE of
fre/__init__.py
. What this does is it creates another logging.handler
to manage the output, but does not resolve
the ambiguity to previously defined loggers
of which handler
should be getting used. If this secondary logging.basicConfig
call is left in the PR or fork at merge-time, it can cause oddly silent logging behavior. This can be VERY tricky to debug!
avoid os.chdir
if you can
Directory changing in python
is not transient by-default, i.e., if when running fre
the interpreter changes directories,
then the result of a os.cwd()
later in the program may be changed to an unexpected value, leading to difficult bugs.
This being said, sometimes an os.chdir
is hard to not want to use. If one has to use directory changing instead of managing
directory targets explicitly as pathlib.Path
instances, then one can use the following logic to safely chdir
where needed
and chdir
back:
go_back_here = os.cwd()
try:
os.chdir(target_dir)
# DO STUFF AFTER CHDIR HERE
except:
raise Exception('some error explaining what went wrong')
finally:
os.chdir(go_back_here)
MANIFEST.in
In the case where non-python files like templates, examples, and outputs are to be included in the fre-cli
package,
MANIFEST.in
can provide the solution. Ensure that the file exists within the correct folder, and add a line to the
MANIFEST.in
file saying something like include fre/fre<tool>/fileName.fileExtension
For more efficiency, if there are multiple files of the same type needed, the
MANIFEST.in
addition can be something likerecursive-include fre/fre<tool> *.fileExtension
which would recursively include every file matching thatfileExtension
within the specified directory and its respective subdirectories.