-
Notifications
You must be signed in to change notification settings - Fork 13.8k
llama : move vocab, grammar and sampling into separate files #8508
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
d4f8f52 to
516746a
Compare
db39019 to
0049b1a
Compare
da7f831 to
dc96d90
Compare
ec7c6d9 to
8c5f2c2
Compare
Are we mitigating breakages this time? |
8c5f2c2 to
0c14b04
Compare
0c14b04 to
39fbaf9
Compare
|
This is a first step in partitioning |
slaren
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good for a first step, it will probably need more work to completely decouple the different components. Some notes:
llama_get_vocab,llama_get_samplingare unused and probably should be removedllm_load_vocabshould eventually be moved tollama-vocab.cpp- The symbols in
unicode.handunicode-data.hshould have allama_prefix, or moved to a namespace LLAMA_API_INTERNALinllama.hshould be removed, and tests should include the private headers instead
I think it's reasonable if you can avoid breakages |
I started implementing that in #8643. Will look to merge this PR in the meantime to avoid resolving bigger conflicts |
…g#8508) * llama : move sampling code into llama-sampling ggml-ci * llama : move grammar code into llama-grammar ggml-ci * cont ggml-ci * cont : pre-fetch rules * cont ggml-ci * llama : deprecate llama_sample_grammar * llama : move tokenizers into llama-vocab ggml-ci * make : update llama.cpp deps [no ci] * llama : redirect external API to internal APIs ggml-ci * llama : suffix the internal APIs with "_impl" ggml-ci * llama : clean-up
Some refactoring attempts, mainly trying to reorganize the
llamacode to prepare for #5214 and #5215API Changes:
llama_sample_grammar->llama_grammar_sampleSummary:
llama_vocabtollama-vocab.h/.cppllama.cpptollama-vocab.cppllama_sample_implementation tollama-sampling.h/.cppllama_grammar_implementation tollama-grammar.h/.cppTODO:
Makefileheader deps_implfor consistencyThe reason for this change is to be able to more easily distinguish public from private calls and not rely on function overloads. For example:
llama.cpp:llama_set_rng_seed->llama-samlping.cpp:llama_set_rng_seed_impl) is to decouplellama-sampling.cppfromllama_contextConflicting PRs:
Follow-up PRs:
_internalsuffixes to_impl