Various tests are flaky when run in parallel

[Patagonicus]

I've noticed that tests are very likely to fail on ARM machines when run in parallel with xdist. I was also able to reproduce this on a x86_64 machine by introducing some load. I was unable to reproduce this on either machine when running the tests serially, even with a high load (stress -c 16 on a machine with 4 logical threads).

Please see the NixOS/nixpkgs#91706 for more information.

[scop]

Yep, I'm aware of this, and it happens every now and then in Travis too. Would like to see it fixed and not brushed under the carpet by turning parallelism off, as I think the test suite is too slow already even with modest parallel runners. And I think keeping a problem visible makes it more likely that something will be done about it.