[TODO] nim-dustmite source code minimization tool for automated bug reduction

[timotheecour]

As I was trying to reduce a compiler error that only triggers in my full project and is hard/tedious to minimize manually, it occurred to me we could try to develop a source code minimization tool to automate that.

in D, that tool exists, see https://github.com/CyberShadow/DustMite and docs https://github.com/CyberShadow/DustMite/wiki ; developped by @CyberShadow
It's used extensively by the community for investigating bugs; it's especially invaluable for those that trigger in your whole project but are hard to isolate to a minimal bug (which is the 1st step to solving it). I've used it myself a lot for great benefit.

It turns out dustmite is not super tied to D language (doesn't need to know about D's semantics) ; in fact it can reduce C++ code too as it only operates at much shallower level.

• First step could be adapting/forking dustmite to work with Nim code

@CyberShadow any help / pointers in this thread would be most welcome!

• second step (optional, not needed for that purpose...) could be porting that forked dustmite to nim

links

• http://delta.tigris.org/
• https://github.com/CyberShadow/DustMite
• also works great in combo with nim-digger nim-digger: program to build nim at any revision (including forks/PRs) (cf D's digger); useful for git bisect timotheecour/Nim#332

[Araq]

IIRC @zielmicha had a tool that does this.

[Araq]

I'm not too happy these tools as they tend to produce "too minimal" test cases that then get fixed but in the real, larger example the problem remains. Minimal test cases cause an "overfit".

[CyberShadow]

@CyberShadow any help / pointers in this thread would be most welcome!

DustMite has, more or less, pluggable splitters:

https://github.com/CyberShadow/DustMite/blob/7403f0d053b96b2c21950df038923b5d335d557f/splitter.d#L75-L82

There is already support for reducing files in a second "syntax" (patch files). D mode should, to some extent, work with any curly-brace languages already though.

I'm not too happy these tools as they tend to produce "too minimal" test cases that then get fixed but in the real, larger example the problem remains. Minimal test cases cause an "overfit".

In our experience this happens very rarely (less than 1% of test cases). In any case, this mistake can be done by humans as well, who are not aware of the exact internal mechanics that cause the bug.

What happens more commonly is that, along the way of reducing the intended bug, DustMite generates a test case that triggers a different bug with similar symptoms (e.g. a compiler segfault), and then gets sidetracked into reducing that bug. In such cases, a more strict test script (e.g. which checks the segfault stack trace) is the solution.

https://github.com/CyberShadow/DustMite/wiki/Detecting-a-specific-segfault

[krux02]

What is a "compiler ICE"?

[krux02]

I added the Won't fix label, because that is how I understand the status quo.

[Araq]

@timotheecour Feel free to submit one, it's not a bad idea, but we don't intent to work on one.