Fix pip module tests

[simmsa]

Status: Ready

This fixes test failures related to module loading using pip install "mhkit[<module>]" syntax.

There are some modules that have interdependencies with other modules, i.e., loads module needs wave module code that are causing test failures.

The root cause of test failures is specific packages that are not specified in pyproject.toml for a specific module, but are called when the tests are run.

The fix is to add lazy loading for the wave.io module and reconfigure the modules in pyproject.toml to capture the a more correct dependency list derived from running the tests.

Also the command that runs these tests was changed from pip install -e "mhkit[<module>]" . to pip install -e ".[<module>]" because the tests fail with -e "mhkit" but succeed with -e ".". I suspect that this changes the way mhkit is installed, but I don't know what the specific differences are.

[simmsa]

@akeeste, this is good to go after the test-optional-pip-depencencies tests pass.

These failures are expected and are fixed in #422

It may make sense to merge the 3 open PR's into develop and then merge that into main, but I will leave that up to you. If you go this route, LMK if there are merge conflicts, and I can fix them in each PR individually.

[akeeste]

@simmsa the previous install command is likely failing because it was using the version on pypi which is currently broken. Your change updates it to install the local github clone which contains the entire source. I think we should be using the git clone as you updated to since we do that everywhere else.

I went forward with changes into main. Let's stick there until we get all of our bugfixes in and then we'll fast forward develop

[akeeste]

Also can you clarify the forced install of netcdf and hdf5 except for river, power, utils, loads? If those modules don't depend on netcdf and hdf5, which were causing issues that you solved in #422, then we can simplify their tests by treating them identically to the other tests right?

[simmsa]

@simmsa the previous install command is likely failing because it was using the version on pypi which is currently broken. Your change updates it to use the local github clone which contains the entire source.

To my understanding, the changes to pyproject.toml will fix the pip-optional-dependency tests by themselves. The lazy loading of mhkit.wave.io is for a different convenience correct?

I went forward with changes into main. Let's stick there until we get all of our bugfixes in and then we'll fast forward develop

@akeeste thanks for looking into this.

Did the test-optional-pip-dependencies tests every fully pass before? The failures here https://github.com/MHKiT-Software/MHKiT-Python/actions/runs/18510669330 in this test https://github.com/MHKiT-Software/MHKiT-Python/actions/runs/18510669330/job/52750802829:

==================================== ERRORS ====================================
_______________________ ERROR collecting test_extreme.py _______________________
ImportError while importing test module '/home/runner/work/MHKiT-Python/MHKiT-Python/mhkit/tests/loads/test_extreme.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mhkit/tests/loads/test_extreme.py:3: in <module>
    import mhkit.loads as loads
mhkit/loads/__init__.py:12: in <module>
    from mhkit.loads import extreme
mhkit/loads/extreme/__init__.py:20: in <module>
    from mhkit.loads.extreme.mler import (
mhkit/loads/extreme/mler.py:25: in <module>
    from mhkit.wave.resource import frequency_moment
mhkit/wave/__init__.py:2: in <module>
    from mhkit.wave import io
mhkit/wave/io/__init__.py:1: in <module>
    from mhkit.wave.io import ndbc
mhkit/wave/io/ndbc.py:6: in <module>
    import requests
E   ModuleNotFoundError: No module named 'requests'
________________________ ERROR collecting test_loads.py ________________________
ImportError while importing test module '/home/runner/work/MHKiT-Python/MHKiT-Python/mhkit/tests/loads/test_loads.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mhkit/tests/loads/test_loads.py:5: in <module>
    from mhkit.wave import resource
mhkit/wave/__init__.py:2: in <module>
    from mhkit.wave import io
mhkit/wave/io/__init__.py:1: in <module>
    from mhkit.wave.io import ndbc
mhkit/wave/io/ndbc.py:6: in <module>
    import requests
E   ModuleNotFoundError: No module named 'requests'
=========================== short test summary info ============================
ERROR .github/workflows/test_extreme.py
ERROR .github/workflows/test_loads.py
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
============================== 2 errors in 4.31s ===============================

Are what this attempts to fix. During those tests it looks like there are interdependencies, i.e., the loads module imports frequency_moment from the wave module, which imports all wave modules, which imports the wave.io module, which has dependencies not specified in the loads module deps in pyproject.toml, and specifically in the above case, requests.

In the above error the critical detail is that frequency_moment is not in the wave.io module. This means that the wave module imports are "greedy". This was fine before we allowed module specific installs, but now this design choice in the wave module init.py and downstream init.py files import everything within the wave module and which cause undesired side effects with other modules.

The changes here fix the above interdependency errors in two ways:

1. Add lazy loading to wave.io. This is the "heaviest" module in terms of dependencies. This fixes other module code that requires the wave module, but does not need wave.io

2. Updates pyproject.toml dependencies to account for modules that call other modules. There seems to be a few cases like this.

[simmsa]

@simmsa the previous install command is likely failing because it was using the version on pypi which is currently broken. Your change updates it to install the local github clone which contains the entire source. I think we should be using the git clone as you updated to since we do that everywhere else.

Agreed. I think that pip install . is the right choice here. I didn't catch that until well into this PR.

[simmsa]

Also can you clarify the forced install of netcdf and hdf5 except for river, power, utils, loads? If those modules don't depend on netcdf and hdf5, which were causing issues that you solved in #422, then we can simplify their tests by treating them identically to the other tests right?

Because the module code uses pip to install it will have the same "limitations" as other pip installs and we need to perform the same steps to add the h5 and nc binaries and then rebuild against them. Each of the tests are "independent" pip installs so we need individually install the necessary dependencies. We exclude the modules where nc/h5 files are not used which you listed above.

[akeeste]

Thanks for clarifying @simmsa. The only tests that are failing are tcl errors. These will rerun on commits to main anyways. Approved and merging!