Noisy React console error output is not automatically suppressed as documented when using test frameworks other than Jest

[jaydenseric]

• react-hooks-testing-library version: v5.0.3

Reproduction:

In a blank project, run:

npm install react-test-renderer @testing-library/react-hooks --save-dev

In test.js:

const { renderHook } = require('@testing-library/react-hooks');

function useExample() {
  throw new Error('Message.');
}

renderHook(() => useExample());

Run:

node test.js

Notice the unexpected console error output in the terminal:

The above error occurred in the <TestComponent> component:

    at TestComponent (/[redacted]/node_modules/@testing-library/react-hooks/lib/helpers/createTestHarness.js:20:5)
    at Suspense
    at ErrorBoundary (/[redacted]/node_modules/react-error-boundary/dist/react-error-boundary.cjs.js:40:35)

React will try to recreate this component tree from scratch using the error boundary you provided, ErrorBoundary.

Problem description:

The documentation says:

To keep test output clean, we patch console.error when importing from @testing-library/react-hooks (or any of the other non-pure imports) to filter out the unnecessary logging and restore the original version during cleanup.
— https://github.com/testing-library/react-hooks-testing-library/blob/master/docs/api-reference.md#consoleerror

In truth, patching only occurs if certain beforeEach and afterEach globals, arbitrary to specific testing frameworks, are present:

react-hooks-testing-library/src/core/console.ts

Lines 6 to 7 in c6de60e

	typeof beforeEach === 'function' &&
	typeof afterEach === 'function' &&

Presumably it's assumed that every project in the world uses Jest 🙃

In another part of the docs:

In order to run tests, you will probably want to be using a test framework. If you have not already got one, we recommend using Jest, but this library should work without issues with any of the alternatives.
— https://github.com/testing-library/react-hooks-testing-library/blob/master/docs/installation.md#testing-framework

“this library should work without issues with any of the alternatives” is not accurate. Plenty of testing frameworks don't mess with globals at all, while some do, but not in the same way as Jest. I use test-director to test my published packages (55.9 kB install size vs 34.4 MB for jest). Sometimes it makes sense to avoid a "framework" altogether and just use a vanilla Node.js script with the Node.js assert API.

Suggested solutions:

1. Do nothing, except update all the documentation to explain specifically what globals need to present for the automatic console error suppression to work, and ideally list what testing frameworks do or don't provide these globals.

   This is not great because it doesn't provide an elegant solution for users with a test environment that doesn't provide these globals; they would be forced to either accept the console noise or hackily define beforeEach and afterEach globals in test scripts just for react-hooks-testing-library to use. It smells to have to copy the globals of an arbitrary testing framework not used in your codebase. The people that specifically chose a testing library that doesn't pollute the global namespace will be opposed to a globals based hack. From a code quality / maintenance perspective, it would not be intuitive without elaborate comments what beforeEach and afterEach refers to or why that workaround code is there.

2. Support console error output suppression for any testing environment. Export functions to monkeypatch and restore the global console, available via the index and deep imports. Internally, these same functions could be used with the automatic beforeEach and afterEach globals detection.

[mpeyper]

Hi @jaydenseric,

Sorry this caused you some strife. So much focus was put into ensuring that the functionality was able to be disabled for those that didn't want it that I failed to consider all the various ways other people might want to use it. While it's true that we assume the majority of our users will be testing with jest we don't aim to exclude any users from using our tools.

I will say that the console logs are actually a side effect of using React and we have always (perhaps only internally) considered the suppression a curtesy to our users rather than a feature per se as the tests do pass with the logs present.

That said, I think option 2 very achievable here and we should strive for that. I also don't see why doing option 1 and improving the documentation around this can't also be done as well as providing an exported utility.

Are you interested in making a PR for this?

[jaydenseric]

@mpeyper

much focus was put into ensuring that the functionality was able to be disabled for those that didn't want it

Thanks for that, it's great to have a pure version available. Normally messing with a native global API like console would be an antipattern.

Are you interested in making a PR for this?

If I end up using @testing-library/react-hooks, maybe. I'm not sure I can make the time for a PR at this very moment; I'm already working (pro bono) full time on several open source projects. That's how I came across this problem.

[mpeyper]

If I end up using @testing-library/react-hooks, maybe. I'm not sure I can make the time for a PR at this very moment; I'm already working (pro bono) full time on several open source projects. That's how I came across this problem.

No worries. It's not a big change so I'm happy to take it on, but have some personal stuff on this weekend and most of next week so won't get to it until after that. If anyone else comes across this please consider it fair game and have a crack. Just leave a message here so we know you're looking at it.

[github-actions]

🎉 This issue has been resolved in version 5.1.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀