simplify translate-c implementation by introducing a new pseudo-ast data structures

[andrewrk]

Right now working on translate-c code is awkward because we have to simultaneously create AST nodes, which is a tree structure, while outputting a list of tokens and source in a linear order. If we had a more specialized AST data structure for translate-c, we could generate the token list and source from it instead of having to duplicate the logic.

It ends up causing bugs like this: #6707

And this hack:

zig/lib/std/zig/ast.zig

Lines 26 to 28 in 245d98d

	/// translate-c uses this to avoid having to emit correct newlines
	/// TODO get rid of this hack
	generated: bool = false,

To explain another way, the current process is:

• clang analysis => zig AST + source + token list => std.zig.render

Proposed process is:

• clang analysis => TranslateCNode Tree => zig AST + source + token list => std.zig.render

The translate-c node tree data structure does not need to map to zig AST, it just has to be whatever is convenient for the translate-c implementation.

This probably would have a small performance penalty; the main objective with this proposal is to increase maintainability.

cc @Vexu

[tadeokondrak]

Is it worth having any sort of tree over string output?

[andrewrk]

In my experience the tree structure feels helpful for the implementation, and the linear string output is the tricky part to manage.

[LemonBoy]

A typed C AST is also useful to simplify the boolToInt/intToBool insertion logic and perform several other simplifications over the C code.

[Vexu]

This would definitely make translate-c easier to maintain by removing all the strange appendTokens needed to please Zig fmt. However generating two ASTs and adding some more type analysis seems like it could be more than a small performance penalty with something like zig translate-c /usr/include/epoxy/gl.h -I/usr/include already taking ~22s.

[LemonBoy]

it could be more than a small performance penalty with something like zig translate-c /usr/include/epoxy/gl.h -I/usr/include already taking ~22s.

Did you profile it? What are the bottlenecks?
Keep in mind that if the caching system works as expected the overall impact is near to zero.

[pixelherodev]

It'd also let us write a C-AST -> ZIR path for stage2 and allow zig cc to work without LLVM ;)

[pixelherodev]

However generating two ASTs and adding some more type analysis seems like it could be more than a small performance penalty with something like zig translate-c /usr/include/epoxy/gl.h -I/usr/include already taking ~22s.

If it gets slower, but more correct, then that is performance well spent.