make add_special_tokens/skip_special_tokens default value is true which align with hf setting #446
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
XiaobingSuper
force-pushed
the
xiaobing/add_special_tokens
branch
from
9530bf9 to
a1e8233
Compare
XiaobingSuper
force-pushed
the
xiaobing/add_special_tokens
branch
from
a1e8233 to
84ff2d1
Compare
XiaobingSuper
changed the title
|
Hi @XiaobingSuper , thanks a lot for your great contribution! We've integrated your changes in PR #454 and credited you as co-author, hence I'm going to close this PR. Let me know if you have any questions, and thanks again for help make TensorRT-LLM better. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For HF
Tokenizer.encode, theadd_special_tokensdefault value isTrue(https://huggingface.co/transformers/v2.11.0/main_classes/tokenizer.html#transformers.PreTrainedTokenizer.encode), but tensorrtllm_backend default value isFlase, and if the user doesn't set it atpreprocessing/config.pbtxt, there may have some issues: #445 and #434. I think we need to align the behavior of tensorrtllm_backend with HF and not expect the user to set it before launching a model server(same asTokenizer.decode).