Skip to content

model : support LiquidAI LFM2 hybrid family #13805

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

model : support LiquidAI LFM2 hybrid family #13805

wants to merge 2 commits into from

Conversation

tdakhran
Copy link

Summary

Add support for LiquidAI LFM2 model family. For more information about models, please read the blog post.

  • Support hybrid LFM2-350M, LFM2-700M, and LFM2-1.2B models.
  • Add ShortConvBlock.
  • Modify construct_transformer to construct hybrid architectures.
  • Move FeedForward to avoid cyclid dependency

Instructions are in examples/models/lfm2/README.md.

Test plan

All commands in README.md are tests.

❯ python -m examples.models.llama.runner.native \
  --model lfm2_700m \
  --pte lfm2_700m_8da4w.pte \
  --tokenizer ~/.cache/huggingface/hub/models--LiquidAI--LFM2-700M/snapshots/ab260293733f05dd4ce22399bea1cae2cf9b272d/tokenizer.json \
  --tokenizer_config ~/.cache/huggingface/hub/models--LiquidAI--LFM2-700M/snapshots/ab260293733f05dd4ce22399bea1cae2cf9b272d/tokenizer_config.json \
  --prompt "<|startoftext|><|im_start|>user\nWho are you?<|im_end|>\n<|im_start|>assistant\n" \
  --params examples/models/lfm2/config/lfm2_700m_config.json \
  --max_len 128 \
  -kv \
  --temperature 0.3

...
I'm an AI designed to assist with generating text based on the prompts you provide. I'm a type of language model, but I don't have a physical form or consciousness. I operate based on complex algorithms and vast amounts of training data. How can I help you today? If you have a specific question or need assistance with something, feel free to ask!
...

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 29, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13805

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ No Failures

As of commit 32441b0 with merge base 1520f9f (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla Meta CLA
Copy link

meta-cla bot commented Aug 29, 2025

Hi @tdakhran!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

jackzhxng
jackzhxng approved these changes Aug 29, 2025
View reviewed changes
Copy link
Contributor

@jackzhxng jackzhxng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, can you try running with the C++ runner? https://github.com/pytorch/executorch/blob/main/examples/models/llama/README.md#step-3-run-on-your-computer-to-validate

examples/models/lfm2/convert_weights.py Outdated
}


def lfm_2_tune_to_meta(state_dict: Dict[str, torch.Tensor]) -> Dict[str, torch.Tensor]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just call this lfm_2_to_meta, I think some of the others are only named that way because we used to use TorchTune to load the checkpoints

examples/models/lfm2/convert_weights.py Outdated

from torchtune.models.convert_weights import get_mapped_key

_LFM_2_FROM_META = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
_LFM_2_FROM_META = {
_LFM_2_TO_META = {

And reverse keys and values

examples/models/lfm2/convert_weights.py Outdated
Dict[str, torch.Tensor]: State dict in Meta's format.
"""
converted_state_dict = {}
inverted_mapping_dict = {v: k for k, v in _LFM_2_FROM_META.items()}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
inverted_mapping_dict = {v: k for k, v in _LFM_2_FROM_META.items()}
inverted_mapping_dict = {v: k for k, v in _LFM_2_FROM_META.items()}

Remove, use _LFM_2_TO_META

@mergennachin
Copy link
Contributor

Please rebase to main and resolve the conflicts

Don't merge yet, I will have to import internally and run a few internal tests.

@mergennachin mergennachin added ciflow/trunk release notes: llm To capture llm specific changes in release notes labels Aug 29, 2025
@tdakhran
model : support LiquidAI LFM2 hybrid family
57f82c5 
Add support for [LiquidAI LFM2](https://huggingface.co/collections/LiquidAI/lfm2-686d721927015b2ad73eaa38) model family.
For more information about models, please read [the blog post](https://www.liquid.ai/blog/liquid-foundation-models-v2-our-second-series-of-generative-ai-models).

- Support hybrid LFM2-350M, LFM2-700M, and LFM2-1.2B models.
- Add `ShortConvBlock`.
- Modify `construct_transformer` to construct hybrid architectures.
- Move FeedForward to avoid cyclid dependency
@tdakhran
Address PR feedback
32441b0
@tdakhran tdakhran force-pushed the tarek/feat/lfm2_upstream branch from 73aa3b8 to 32441b0 Compare August 30, 2025 18:53
@tdakhran
Copy link
Author

@jackzhxng @mergennachin thanks for the review, I've rebased and addressed the feedback.

All commands in README.md work, including the C++ runner. However, the C++ runner output differs from the Python runner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jackzhxng jackzhxng jackzhxng approved these changes

@larryliu0820 larryliu0820 Awaiting requested review from larryliu0820 larryliu0820 is a code owner

@swolchok swolchok Awaiting requested review from swolchok swolchok is a code owner

@mergennachin mergennachin Awaiting requested review from mergennachin mergennachin is a code owner

@lucylq lucylq Awaiting requested review from lucylq lucylq is a code owner

@kimishpatel kimishpatel Awaiting requested review from kimishpatel

Assignees
No one assigned
Labels
release notes: llm To capture llm specific changes in release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tdakhran @mergennachin @jackzhxng