Some transitive "extras" dependencies are not automatically included to the target

[hartikainen]

🐞 bug report

Affected Rule

py_{test,binary,library}

Is this a regression?

I'm not sure. I think things work differently between venvs_site_packages=yes and venvs_site_packages=no. I started seeing this issue when switching over to venvs_site_packages=yes.

Description

The "Using PyPI" section in rules_python-documentation says:

‘Extras’ dependencies
Any “extras” specified in the requirements lock file will be automatically added as transitive dependencies of the package. In the example above, you’d just put requirement("useful_dep") or @pypi//useful_dep.

After switching to using venvs_site_packages=yes, I see that the above is not the case for all packages. For example, consider the following simple requirements.in:

etils[eapp,epath]

and its compiled requirements.txt as follows:

Details

# This file was autogenerated by uv via the following command:
#    uv pip compile --no-strip-extras --generate-hashes --emit-index-url ./requirements.in -o ./requirements.txt
--index-url https://pypi.org/simple

absl-py==2.3.1 \
    --hash=sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9 \
    --hash=sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d
    # via etils
docstring-parser==0.17.0 \
    --hash=sha256:583de4a309722b3315439bb31d64ba3eebada841f2e2cee23b99df001434c912 \
    --hash=sha256:cf2569abd23dce8099b300f9b4fa8191e9582dda731fd533daf54c4551658708
    # via simple-parsing
etils[eapp, epath]==1.13.0 \
    --hash=sha256:a5b60c71f95bcd2d43d4e9fb3dc3879120c1f60472bb5ce19f7a860b1d44f607 \
    --hash=sha256:d9cd4f40fbe77ad6613b7348a18132cc511237b6c076dbb89105c0b520a4c6bb
    # via -r ./requirements.in
fsspec==2025.9.0 \
    --hash=sha256:19fd429483d25d28b65ec68f9f4adc16c17ea2c7c7bf54ec61360d478fb19c19 \
    --hash=sha256:530dc2a2af60a414a832059574df4a6e10cce927f6f4a78209390fe38955cfb7
    # via etils
importlib-resources==6.5.2 \
    --hash=sha256:185f87adef5bcc288449d98fb4fba07cea78bc036455dd44c5fc4a2fe78fed2c \
    --hash=sha256:789cfdc3ed28c78b67a06acb8126751ced69a3d5f79c095a98298cd8a760ccec
    # via etils
simple-parsing==0.1.7 \
    --hash=sha256:225e6b35252d68f7894716101fe3bd7e6dd3d30ab7b1c3c023f77a42dbe1336f \
    --hash=sha256:5276e6c90c157362dd0173d1eecebe58361a66b457129cc9bba13b78a4e85092
    # via etils
typing-extensions==4.15.0 \
    --hash=sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466 \
    --hash=sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548
    # via
    #   etils
    #   simple-parsing
zipp==3.23.0 \
    --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \
    --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166
    # via etils

Based on the documentation referred to above, I'd expect the following py_test target to run without problems:

Details

load("@rules_python//python:defs.bzl", "py_test")


py_test(
    name = "etils_test",
    srcs = ["etils_test.py"],
    deps = [
        "@pypi//etils",
    ],
)

where etils_test.py is defined as:

from etils import eapp
from etils import epath

eapp.better_logging()

However, I get the following error:

Details

bazel run \
  --verbose_failures \
  //:etils_test
INFO: Analyzed target //:etils_test (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:etils_test up-to-date:
  bazel-bin/etils_test
INFO: Elapsed time: 0.133s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: external/bazel_tools/tools/test/test-setup.sh ./etils_test
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //:etils_test
-----------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test_stage2_bootstrap.py", line 499, in <module>
    main()
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test_stage2_bootstrap.py", line 493, in main
    _run_py_path(main_filename, args=sys.argv[1:])
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test_stage2_bootstrap.py", line 287, in _run_py_path
    runpy.run_path(main_filename, run_name="__main__")
  File "<frozen runpy>", line 287, in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/etils_test.py", line 1, in <module>
    from etils import eapp
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/eapp/__init__.py", line 17, in <module>
    from etils.eapp.dataclass_flags import make_flags_parser
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/eapp/dataclass_flags.py", line 25, in <module>
    with _internal.check_missing_deps():
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/external/rules_python++python+python_3_12_x86_64-unknown-linux-gnu/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/epy/_internal.py", line 48, in check_missing_deps
    reraise_utils.reraise(
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/epy/reraise_utils.py", line 112, in reraise
    raise new_exception.with_traceback(e.__traceback__) from e.__cause__
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/epy/_internal.py", line 46, in check_missing_deps
    yield
  File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/eapp/dataclass_flags.py", line 27, in <module>
    from absl import flags
ModuleNotFoundError: No module named 'absl'

Each etils sub-modules require deps to be installed separately (e.g. `from etils import ecolab` -> `pip install etils[ecolab]`)

To make this target work, I need to add several transitive dependency packages to the rule manually. The following target works:

load("@rules_python//python:defs.bzl", "py_test")


py_test(
    name = "etils_test",
    srcs = ["etils_test.py"],
    deps = [
        "@pypi//absl_py",
        "@pypi//etils",
        "@pypi//simple_parsing",
        "@pypi//importlib_resources",
    ],
)

Note how absl_py is clearly a transitive dependency of etils, as can be seen from the compiled requirements.txt. I wonder if I just misunderstand what the documentation means about extras. It's unclear what the useful_dep in the documentation example is because I don't see any reference to useful_dep "in the example above" in the documentation.

🔬 Minimal Reproduction

See full reproduction here: main...hartikainen:rules_python:etils

🔥 Exception or Error

See Above

🌍 Your Environment

Operating System:

$lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04.1 LTS
Release:        24.04
Codename:       noble

Output of bazel version:

$ bazel --version
bazel 8.2.1

Rules_python version:

43a5acf

Anything else relevant?

N/A