Skip to content

Commit 265956d

Browse files
authored
Fix pip module package building (#421)
Status: Ready The v1.0.0 release on PyPI is missing all submodules (wave, tidal, dolfyn, etc.) when installed via pip and downstream via conda. ## Reproduction ```bash pip install "mhkit[all]" ``` ```bash python Python 3.11.12 (main, Apr 8 2025, 14:15:29) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import mhkit >>> mhkit.__version__ 'v1.0.0' >>> from mhkit import river Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: cannot import name 'river' from 'mhkit' (/opt/homebrew/lib/python3.11/site-packages/mhkit/__init__.py) ``` ### Installed Package Contents Inspecting the installed package shows two files (run `python -m site` to find `site-packages` path): ```bash ls /opt/homebrew/lib/python3.11/site-packages/mhkit __init__.py __pycache__ warnings.py ``` All submodules (wave, river, tidal, dolfyn, power, loads, mooring, acoustics, qc, utils) are missing. ### Reproduction Running the same build command used by GitHub Actions: ```bash python -m build --wheel --outdir dist/ . ``` - `python -m build`: Runs Python's `build` module (PEP 517 build frontend) to build the package - `--wheel`: Build only a wheel distribution (skips source distribution) - `--outdir dist/`: Output directory for built files - `.`: Build the package in the current directory (where `pyproject.toml` is) Build output shows two files being copied: ``` running build_py creating build/lib/mhkit copying mhkit/warnings.py -> build/lib/mhkit copying mhkit/__init__.py -> build/lib/mhkit ``` Wheel contents: ``` adding 'mhkit/__init__.py' adding 'mhkit/warnings.py' ``` ## Probable Root Cause In `pyproject.toml`, this configuration tells setuptools to only include the top-level `mhkit` package, not any subpackages per: https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#setuptools-specific-configuration ```toml [tool.setuptools] packages = ["mhkit"] ``` ## Proposed Fix Replace the explicit package list with automatic package discovery using setuptools' `find` directive. [Setuptools Package Discovery Documentation](https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#finding-simple-packages) ```toml [tool.setuptools.packages.find] where = ["."] include = ["mhkit*"] [tool.setuptools] zip-safe = false include-package-data = true ``` What this does (from setuptools docs): - `where = ["."]`: Search for packages starting from the root directory - `include = ["mhkit*"]`: Only include packages that match the pattern `mhkit*` (this includes `mhkit`, `mhkit.wave`, `mhkit.river`, etc.) - Setuptools will automatically discover all directories containing `__init__.py` files This will automatically discover and include: - `mhkit` - `mhkit.wave` - `mhkit.river` - `mhkit.tidal` - `mhkit.dolfyn` - `mhkit.dolfyn.adp` - `mhkit.dolfyn.adv` - `mhkit.dolfyn.io` - `mhkit.dolfyn.rotate` - `mhkit.dolfyn.tools` - `mhkit.power` - `mhkit.loads` - `mhkit.loads.extreme` - `mhkit.mooring` - `mhkit.acoustics` - `mhkit.qc` - `mhkit.utils` - `mhkit.wave.io` - `mhkit.wave.io.hindcast` - `mhkit.tidal.io` - `mhkit.river.io` - `mhkit.tests.*` ### Verification of Fix ```bash python -m build --sdist --wheel --outdir dist/ . unzip -l dist/mhkit-*.whl | grep "mhkit/" | grep "__init__.py" ``` - `python -m build`: Runs Python's `build` module to build the package - `--sdist`: Build a source distribution (`.tar.gz` file) - a compressed archive of source code - `--wheel`: Build a wheel (`.whl` file) - a pre-built distribution that pip installs directly - Can be unzipped and inspected: `unzip -l dist/mhkit-1.0.0-py3-none-any.whl | grep "mhkit/" | grep "__init__.py"` Why build both formats? - Wheel: This is what `pip install mhkit` downloads and installs. The bug affects this format. - Source distribution: Backup format for systems that can't use wheels. Always includes all source files. - Current GitHub Actions builds both, so this matches the actual release process. References: - [Python Packaging: Source Distribution](https://packaging.python.org/en/latest/specifications/source-distribution-format/) - [Python Packaging: Wheel](https://packaging.python.org/en/latest/specifications/binary-distribution-format/) Contents of newly build wheel: ```bash unzip -l dist/mhkit-1.0.0-py3-none-any.whl | grep "mhkit/" | grep "__init__.py" 1556 10-02-2025 16:10 mhkit/__init__.py 1156 10-02-2025 16:10 mhkit/acoustics/__init__.py 493 05-08-2024 16:38 mhkit/dolfyn/__init__.py 18 05-08-2024 16:38 mhkit/dolfyn/adp/__init__.py 18 05-08-2024 16:38 mhkit/dolfyn/adv/__init__.py 362 04-24-2024 16:42 mhkit/dolfyn/io/__init__.py 227 04-24-2024 16:42 mhkit/dolfyn/rotate/__init__.py 71 04-24-2024 16:42 mhkit/dolfyn/tools/__init__.py 493 10-02-2025 16:10 mhkit/loads/__init__.py 974 10-02-2025 16:10 mhkit/loads/extreme/__init__.py 84 04-24-2024 16:42 mhkit/mooring/__init__.py 94 05-08-2024 16:38 mhkit/power/__init__.py 145 05-08-2024 16:38 mhkit/qc/__init__.py 223 10-02-2025 16:10 mhkit/river/__init__.py 161 10-02-2025 16:10 mhkit/river/io/__init__.py 284 10-02-2025 16:10 mhkit/tidal/__init__.py 157 05-08-2024 16:38 mhkit/wave/__init__.py ``` 5. Installed and tested the fixed wheel: ```bash pip uninstall mhkit pip install dist/mhkit-1.0.0-py3-none-any.whl ``` 6. Verified imports work: ```bash In [1]: import mhkit In [2]: mhkit.__version__ Out[2]: 'v1.0.0' In [3]: from mhkit import river In [4]: river.performance.circular(30) Out[4]: (30, 706.8583470577034) ``` ## Preventing Future Issues To prevent this issue from happening again, this adds two GH actions changes in `.github/workflows/main.yml` and `.github/workflows/pypi.yml, `to verify the built wheel includes all submodules and run the tests against the installed wheel.
1 parent f4a1e8d commit 265956d

File tree

4 files changed

+77
-2
lines changed

4 files changed

+77
-2
lines changed

.github/workflows/main.yml

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -390,6 +390,62 @@ jobs:
390390
parallel: true
391391
path-to-lcov: ./coverage.lcov
392392

393+
test-wheel-packaging:
394+
name: Test Built Wheel
395+
runs-on: ubuntu-latest
396+
steps:
397+
- uses: actions/checkout@v4
398+
399+
- uses: actions/setup-python@v5
400+
with:
401+
python-version: '3.11'
402+
403+
- name: Install build dependencies
404+
run: python -m pip install build --user
405+
406+
- name: Build wheel
407+
run: python -m build --wheel --outdir dist/ .
408+
409+
- name: Install MHKiT from built wheel
410+
run: |
411+
# Install from build wheel in dist/ directory, not from PyPI
412+
# -f, --find-links <url> If a URL or path to an html file, then parse for links to archives such as sdist (.tar.gz) or wheel (.whl) files.
413+
# If a local path or file:// URL that's a directory, then look for archives in the directory listing.
414+
pip install 'mhkit[all]' --find-links dist/
415+
416+
- name: Test mhkit submodule imports
417+
run: |
418+
python -c "
419+
import mhkit
420+
print(f'Version: {mhkit.__version__}')
421+
422+
# Test all submodules can be imported
423+
modules = ['wave', 'river', 'tidal', 'dolfyn', 'power', 'loads', 'mooring', 'acoustics', 'qc', 'utils']
424+
for mod in modules:
425+
exec(f'from mhkit import {mod}')
426+
print(f'Successfully imported mhkit.{mod}')
427+
428+
print('All submodules imported successfully!')
429+
"
430+
431+
- name: Verify mhkit package structure
432+
run: |
433+
python -c "
434+
import mhkit
435+
import os
436+
437+
mhkit_path = os.path.dirname(mhkit.__file__)
438+
expected_dirs = ['wave', 'river', 'tidal', 'dolfyn', 'power', 'loads', 'mooring', 'acoustics', 'qc', 'utils']
439+
440+
for d in expected_dirs:
441+
dir_path = os.path.join(mhkit_path, d)
442+
assert os.path.isdir(dir_path), f'Missing submodule directory: {d}'
443+
init_file = os.path.join(dir_path, '__init__.py')
444+
assert os.path.isfile(init_file), f'Missing __init__.py in {d}'
445+
446+
print('Package structure verified!')
447+
"
448+
393449
test-optional-pip-dependencies:
394450
needs: [set-os, prepare-nonhindcast-cache]
395451
runs-on: ubuntu-latest

.github/workflows/pypi.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,11 @@ jobs:
2727
- run: python -m pip install build --user
2828
- run: python -m build --sdist --wheel --outdir dist/ .
2929

30+
- name: Test wheel contents before upload
31+
run: |
32+
pip install 'mhkit[all]' --no-index --find-links dist/
33+
python -c "from mhkit import wave, river, tidal, dolfyn, power, loads, mooring, acoustics, qc, utils; print('All modules imported successfully')"
34+
3035
- name: Upload to Test PyPI
3136
if: github.event_name != 'release' && github.repository_owner == 'MHKiT-Software'
3237
uses: pypa/gh-action-pypi-publish@release/v1

mhkit/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111

1212
configure_warnings()
1313

14-
__version__ = "v1.0.0"
14+
__version__ = "v1.0.1"
1515

1616
__copyright__ = """
1717
Copyright 2019, Alliance for Sustainable Energy, LLC under the terms of

pyproject.toml

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,8 +126,22 @@ examples = [
126126
Homepage = "https://github.com/MHKiT-Software/mhkit-python"
127127
Documentation = "https://mhkit-software.github.io/MHKiT"
128128

129+
[tool.setuptools.packages.find]
130+
# This section controls how this python package is built for pip installations
131+
# Getting this section wrong results in an incomplete package that is missing submodules
132+
# This can be tested by simply running python -m build in the root directory
133+
# and then inspecting the generated .tar.gz and .whl files in the dist/ directory
134+
#
135+
# https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#finding-simple-packages
136+
# What this does: (per the above docs):
137+
# - `where = ["."]`: Search for packages starting from the root directory
138+
# - `include = ["mhkit*"]`: Only include packages that match the pattern `mhkit*`
139+
# - this includes `mhkit`, `mhkit.wave`, `mhkit.river`, etc.
140+
# - Setuptools will automatically discover all directories containing `__init__.py` files
141+
where = ["."]
142+
include = ["mhkit*"]
143+
129144
[tool.setuptools]
130-
packages = ["mhkit"]
131145
zip-safe = false
132146
include-package-data = true
133147

0 commit comments

Comments
 (0)