Fix passenv CI in tox ini and make tests insensitive to the presence of the CI env variable

[ogrisel]

See: #13679 (comment)

[ogrisel]

@webknjaz CI is green again!

[webknjaz]

LGTM, but I'll let @RonnyPfannschmidt merge since he was active in the related PR.

[ogrisel]

@RonnyPfannschmidt if you have a few minutes to finalize this review, that would be very appreciated as it would unlock my work to finalize #13679.

[RonnyPfannschmidt]

This looks good

Tho only nitpick that comes to mind is that it could be more declarative

But thats a low hanging fruit followup

[nicoddemus]

I believe we want to remove this env var for all tests, correct?

If so, I suggest you add this autouse fixture to conftest.py:

@pytest.fixture(autouse=True)
def remove_ci_env_var(monkeypatch: MonkeyPatch) -> None:
    """
    Make the test suite insensitive if it is running in CI or not.
    
    If a test requires the variable, it should set it explicitly.
    """
    monkeypatch.delenv("CI", raising=False)

Then you can revert the changes done to the tests.

[ogrisel]

Actually, we might want to use this env var in #13679 to use a longer timeout value in some tests to decrease likelihood of failure on slow CI runners.

[ogrisel]

To be more specific: there is an existing test named test_timeout in test_faulhandler.py that relies on the presence of this variable to adjust the value of the timeout to make it less likely to fail in case the CI runner is too slow (see #7022). However, this strategy never worked as intended because the CI variable propagation was always blocked by the existing configuration in tox.ini.

[webknjaz]

Yeah, I wasn't sure what the impact would be in other places so I didn't suggest making it an autouse fixture.