[Feature request]: Speed-up the build by executing the mutation tests in a separate workflow

[martintmk]

Is your feature request related to a specific problem? Or an existing feature?

I think we can speed-up the build by extracting the mutation steps from https://github.com/App-vNext/Polly/blob/main/.github/workflows/build.yml into separate workflows.

Describe the solution you'd like

I believe we can introduce two new workflows:

• mutation-tests-core.yml - in this workflow we would test all new Polly v8 projects.
• mutation-tests-legacy.yml - in this workflow we would test only Polly project.

This way the mutation tests is decreased by half on top of of reduced build times. With this change our PR builds could be done in less than 20 minutes.

Additional context

No response

[martincostello]

Suggest core not v8, then we don't need to rename it if we get to Polly 9.

[rorypierce]

So I should probably familiarize myself with the code base first... But as somebody trying to get a 10k meter perspective: would you be able to clarify a mutation test in the context of this particular project/build pipeline? I've done a reasonable amount of TDD and or PDD (pain driven development lol ) but I haven't seen the word "mutation" used in software parlance since I was taking genetic algorithms in undergrad (just to clarify my context). Thanks.

[martincostello]

The mutation tests are currently run by this cake task, which uses Stryker to run mutations on our different unit test projects:

Polly/build.cake

Line 170 in 98f29c8

Task("__RunMutationTests")

[martintmk]

but I haven't seen the word "mutation" used in software parlance since I was taking genetic algorithms in undergrad (just to clarify my context). Thanks.

Essentially, mutation tests are tests for unit tests :D

Copied from official docs:

TL; DR: Mutation testing introduces changes to your code, then runs your unit tests against the changed code. It is expected that your unit tests will now fail. If they don't fail, it might indicate your tests do not sufficiently cover the code.

https://stryker-mutator.io/docs/

[ShawnWu33]

Hi @martintmk I am happy to take this if nobody is working on,
Just want a bit clarification about the suggested solution as below.
As I can see currently we ran following mutation tests:

Polly/build.cake

Lines 179 to 183 in 4e7d099

	TestProject(File("../src/Polly.Core/Polly.Core.csproj"), File("./Polly.Core.Tests/Polly.Core.Tests.csproj"), "Polly.Core.csproj");
	TestProject(File("../src/Polly.RateLimiting/Polly.RateLimiting.csproj"), File("./Polly.RateLimiting.Tests/Polly.RateLimiting.Tests.csproj"), "Polly.RateLimiting.csproj");
	TestProject(File("../src/Polly.Extensions/Polly.Extensions.csproj"), File("./Polly.Extensions.Tests/Polly.Extensions.Tests.csproj"), "Polly.Extensions.csproj");
	TestProject(File("../src/Polly.Testing/Polly.Testing.csproj"), File("./Polly.Testing.Tests/Polly.Testing.Tests.csproj"), "Polly.Testing.csproj");
	TestProject(File("../src/Polly/Polly.csproj"), File("./Polly.Specs/Polly.Specs.csproj"), "Polly.csproj");

Is that only the last line Polly.proj belongs to legacy workflow and all the rest belongs to core workflow?