-
-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Make it possible for faulthandler to terminate the pytest process on timeout #13679
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At first glance this looks nice
There may be some nitpicks but they relate to preexisting patterns
I think I understand why the |
""" | ||
import os, time | ||
def test_long_sleep_and_raise(): | ||
time.sleep(1 if "CI" in os.environ else 0.1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shall I remove the if "CI" in os.environ
condition that can never hold as long as pytest's CI is configured to not propagate this env var via its tox.ini config?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's actually propagate it snd check why tox doesn't automatically
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CI
is not in the list of variables passed by default: https://tox.wiki/en/latest/config.html#pass_env
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I tried to add CI
to passenv
(see 000876e above in this PR) this broke many other unrelated tests:
https://github.com/pytest-dev/pytest/actions/runs/17271882336/job/49018562088
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In that case, those tests are incorrectly sensitive to said env var. I recommend fixing them all in a separate PR (that would also bypass this env var in tox) by ensuring adequate isolation in them (monkeypatch.delenv('CI')
). Once that PR is merged, you'd be able to return to this one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if the is_ci_env
could be computed early, before the test runs instead of having this check within. It could be checked before the tests implement shielding through os.environ
manipulation. Perhaps, a session fixture in conftest.py
would be useful in general. Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For this particular case, once the CI variable is properly propagated (after #13684 is merged and this PR synced with main
), just increasing the sleep time as originally intended might be enough to make the test stable again and we would not have skip those tests anymore.
Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
Closes #13678.
This would help projects avoid wasting their precious CI resources when running extensive thread-safety tests (which will become more popular in the new free-threading era).
I also changed the logic to mark the test as XFAIL conditionally on the presence of the "CI" environment variable instead of always skipping (related to #7022). This makes it possible to check that the test pass as expected when running locally.
If you think this is too risky, I can remove the condition and always mark the offending params with the
xfail
mark instead of skipping.EDIT: I will do the later right away because I have already observed such a random failure on the CI: https://github.com/pytest-dev/pytest/actions/runs/17269674978/job/49010727146
EDIT 2: that does not work because
xfail_strict = true
is configured inpyproject.toml
. Let me skip instead...