Feature/fixembedding #4176
Merged
Feature/fixembedding #4176
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… rather than allocate to pointers Signed-off-by: Jim.Idle <[email protected]>
…uch as LexerATNConfig instead of by pointer This is the first of many changes that switches the embedded class structure that was copying Java class hieracrchy from allocations/new to proper embedding such that any struct is allocated with one allocation not two or more. Main PR will cover what this means. Signed-off-by: Jim.Idle <[email protected]>
… of pointer Saves an extra allocation and helps the GC Signed-off-by: Jim.Idle <[email protected]>
…dding Saves many allocations and grbage collections Signed-off-by: Jim.Idle <[email protected]>
Should reduce allocation count by tons. Signed-off-by: Jim.Idle <[email protected]>
…ntlr#4169) * doc: Updates to some of the Go doc comments to start a ful ldocumentation cleanup Signed-off-by: Jim.Idle <[email protected]> * doc: More documentation fixes. Using this as a method of forcing myself to read every line of code in the runtime, and therefore discover mistakes in the original implementation. And, of course, actually working docs for the Go runtime, can only be a good thing. Signed-off-by: Jim.Idle <[email protected]> * doc: More documentation fixes Also changes the exporet level of a some variables and funcs that were not correct, even though no user has currently needed them it would seem. Signed-off-by: Jim.Idle <[email protected]> * doc: Many updates to document exported fuctions correctly and reformat the ingerited Java code It looks like a massive amount of changes, but it is almost all doc or changing exports or renaming unused paramters etc to make the Go linter happy. No actual code changes yet. Signed-off-by: Jim.Idle <[email protected]> * doc: More additions and corrections to the Go documentation for the runtime Signed-off-by: Jim.Idle <[email protected]> * doc: Final clean of exported func and type documentation There will be more to do here as there are a lot of things that are hidden internal to the antlr package that probably should not be. There are also a lot of exported funcs and types without any documentation, that will eventually need to be cleaned up. Signed-off-by: Jim.Idle <[email protected]> * Changed Parser typings (antlr#4149) Signed-off-by: Josua Frank <[email protected]> Co-authored-by: Josua Frank <[email protected]> Signed-off-by: Jim.Idle <[email protected]> * fix: Fixes the failing go runtime test suite which was missing the /v4 off the replace option on the go.mod file (antlr#4163) Arrrgh! Signed-off-by: Jim.Idle <[email protected]> * present antlr before versioning (antlr#4156) Signed-off-by: Jim.Idle <[email protected]> * fix: Prevent use of labels such as start= from generating code that clashes with builtin funcs (antlr#4161) Signed-off-by: Jim.Idle <[email protected]> * Feature/gotestfix (antlr#4168) * fix: Fixes the failing go runtime test suite which was missing the /v4 off the replace option on the go.mod file Arrrgh! Signed-off-by: Jim.Idle <[email protected]> * present antlr before versioning (antlr#4156) Signed-off-by: Jim.Idle <[email protected]> * fix: Prevent use of labels such as start= from generating code that clashes with builtin funcs (antlr#4161) Signed-off-by: Jim.Idle <[email protected]> * fix: Cater for the fact that some test rules use start as a label or rule name As a fix for other cvode gen errors when start, end, or exception are used as label names, they are now translated to have a suffix of `_` at code gen time. However, the runtime tests sometimes use start as a rule name and so we must now cater for this in the tests. Signed-off-by: Jim.Idle <[email protected]> --------- Signed-off-by: Jim.Idle <[email protected]> Co-authored-by: ericvergnaud <[email protected]> Signed-off-by: Jim.Idle <[email protected]> --------- Signed-off-by: Jim.Idle <[email protected]> Signed-off-by: Josua Frank <[email protected]> Co-authored-by: Josua Frank <[email protected]> Co-authored-by: Josua Frank <[email protected]> Co-authored-by: ericvergnaud <[email protected]>
…uch as LexerATNConfig instead of by pointer This is the first of many changes that switches the embedded class structure that was copying Java class hieracrchy from allocations/new to proper embedding such that any struct is allocated with one allocation not two or more. Main PR will cover what this means. Signed-off-by: Jim.Idle <[email protected]>
… of pointer Saves an extra allocation and helps the GC Signed-off-by: Jim.Idle <[email protected]>
Should reduce allocation count by tons. Signed-off-by: Jim.Idle <[email protected]>
Signed-off-by: Jim.Idle <[email protected]>
Signed-off-by: Jim.Idle <[email protected]>
Signed-off-by: Jim.Idle <[email protected]>
Rework of the variaous PredictionContexts has reduced memory allocations to between 30% and 50% of previous version. Signed-off-by: Jim.Idle <[email protected]>
Signed-off-by: Jim.Idle <[email protected]>
Signed-off-by: Jim.Idle <[email protected]>
|
@parrt - Builds are still failing in the cpp artifact upload stage. The go builds are all good, so I think that this PR is good to go. It is a major improvement on memory allocations, with quite a bit more to come. |
|
yay! |
jimidle
pushed a commit
to jimidle/antlr4
that referenced
this pull request
Mar 28, 2023
* feat: Createa n Init routine for BaseATNConfig so we can embed sructs rather than allocate to pointers Signed-off-by: Jim.Idle <[email protected]> * feat: Change BaseATNConfig to be properly embedded in other structs such as LexerATNConfig instead of by pointer This is the first of many changes that switches the embedded class structure that was copying Java class hieracrchy from allocations/new to proper embedding such that any struct is allocated with one allocation not two or more. Main PR will cover what this means. Signed-off-by: Jim.Idle <[email protected]> * feat: Change embedding for ATNBaseSimulator to true embedding instaed of pointer Saves an extra allocation and helps the GC Signed-off-by: Jim.Idle <[email protected]> * feat: Switch the use of pointers to embedded ATN states to true embeddding Saves many allocations and grbage collections Signed-off-by: Jim.Idle <[email protected]> * fix: Correct the way that PredictionContext is compared for merge Should reduce allocation count by tons. Signed-off-by: Jim.Idle <[email protected]> * Feature/docclean Greatly improve the godoc comments in the runtime (antlr#4169) * doc: Updates to some of the Go doc comments to start a ful ldocumentation cleanup Signed-off-by: Jim.Idle <[email protected]> * doc: More documentation fixes. Using this as a method of forcing myself to read every line of code in the runtime, and therefore discover mistakes in the original implementation. And, of course, actually working docs for the Go runtime, can only be a good thing. Signed-off-by: Jim.Idle <[email protected]> * doc: More documentation fixes Also changes the exporet level of a some variables and funcs that were not correct, even though no user has currently needed them it would seem. Signed-off-by: Jim.Idle <[email protected]> * doc: Many updates to document exported fuctions correctly and reformat the ingerited Java code It looks like a massive amount of changes, but it is almost all doc or changing exports or renaming unused paramters etc to make the Go linter happy. No actual code changes yet. Signed-off-by: Jim.Idle <[email protected]> * doc: More additions and corrections to the Go documentation for the runtime Signed-off-by: Jim.Idle <[email protected]> * doc: Final clean of exported func and type documentation There will be more to do here as there are a lot of things that are hidden internal to the antlr package that probably should not be. There are also a lot of exported funcs and types without any documentation, that will eventually need to be cleaned up. Signed-off-by: Jim.Idle <[email protected]> * Changed Parser typings (antlr#4149) Signed-off-by: Josua Frank <[email protected]> Co-authored-by: Josua Frank <[email protected]> Signed-off-by: Jim.Idle <[email protected]> * fix: Fixes the failing go runtime test suite which was missing the /v4 off the replace option on the go.mod file (antlr#4163) Arrrgh! Signed-off-by: Jim.Idle <[email protected]> * present antlr before versioning (antlr#4156) Signed-off-by: Jim.Idle <[email protected]> * fix: Prevent use of labels such as start= from generating code that clashes with builtin funcs (antlr#4161) Signed-off-by: Jim.Idle <[email protected]> * Feature/gotestfix (antlr#4168) * fix: Fixes the failing go runtime test suite which was missing the /v4 off the replace option on the go.mod file Arrrgh! Signed-off-by: Jim.Idle <[email protected]> * present antlr before versioning (antlr#4156) Signed-off-by: Jim.Idle <[email protected]> * fix: Prevent use of labels such as start= from generating code that clashes with builtin funcs (antlr#4161) Signed-off-by: Jim.Idle <[email protected]> * fix: Cater for the fact that some test rules use start as a label or rule name As a fix for other cvode gen errors when start, end, or exception are used as label names, they are now translated to have a suffix of `_` at code gen time. However, the runtime tests sometimes use start as a rule name and so we must now cater for this in the tests. Signed-off-by: Jim.Idle <[email protected]> --------- Signed-off-by: Jim.Idle <[email protected]> Co-authored-by: ericvergnaud <[email protected]> Signed-off-by: Jim.Idle <[email protected]> --------- Signed-off-by: Jim.Idle <[email protected]> Signed-off-by: Josua Frank <[email protected]> Co-authored-by: Josua Frank <[email protected]> Co-authored-by: Josua Frank <[email protected]> Co-authored-by: ericvergnaud <[email protected]> * feat: Change BaseATNConfig to be properly embedded in other structs such as LexerATNConfig instead of by pointer This is the first of many changes that switches the embedded class structure that was copying Java class hieracrchy from allocations/new to proper embedding such that any struct is allocated with one allocation not two or more. Main PR will cover what this means. Signed-off-by: Jim.Idle <[email protected]> * feat: Change embedding for ATNBaseSimulator to true embedding instaed of pointer Saves an extra allocation and helps the GC Signed-off-by: Jim.Idle <[email protected]> * fix: Correct the way that PredictionContext is compared for merge Should reduce allocation count by tons. Signed-off-by: Jim.Idle <[email protected]> * doc: Merge documentation updates Signed-off-by: Jim.Idle <[email protected]> * feat: Rework predictions tructs to use emedding instead of pointers Signed-off-by: Jim.Idle <[email protected]> * feat: more reworking of PredictionContext for embedding Signed-off-by: Jim.Idle <[email protected]> * feat: Ensure that EmptyPredictionContext is correctly initialized Rework of the variaous PredictionContexts has reduced memory allocations to between 30% and 50% of previous version. Signed-off-by: Jim.Idle <[email protected]> * feat: Change from use of type casting to using stored type Signed-off-by: Jim.Idle <[email protected]> * feat: Convert CommonToken to true emedding rather than pointers Signed-off-by: Jim.Idle <[email protected]> --------- Signed-off-by: Jim.Idle <[email protected]> Signed-off-by: Josua Frank <[email protected]> Co-authored-by: Josua Frank <[email protected]> Co-authored-by: Josua Frank <[email protected]> Co-authored-by: ericvergnaud <[email protected]> Signed-off-by: Jim.Idle <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
feat: Convert major structs to use embedded parent structs rather than allocating pointers
Many of the structs representing things like transitions, contexts, atn etc emulated Java inheritance by using pointers to instances of parent structs within their definitions. This doubles, triples and even quadruples the number of allocations needed to be made to build the recognizer structs and the prediction contexts.
This PR replaces the pointers with true embedding in the structs that are the hotspots within the runtime.
Seems like a big PR but in fact all the changes are doing the same thing.