Skip to content

Llamafile comparison? #8

@hrstoyanov

Description

@hrstoyanov

@mukel thank you for creating this project! I would like to discuss the following topics:

  1. Please enable the Discussions tab for posts like this, which are not real "issues"

  2. Do you plan on releasing Llama3 code?

  3. Do you plan on quantized llama models with Java vector api?

  4. Can you run a benchmark against llamafile, the vector version of which (AVX, neon) claims to be the performance king for inference.
    (I am deciding between using your project or wrapping around the llamafile c code with Java 22 foreign function apis)

  5. Do you plan to implant model training as well? If so, take a look at Andrey's LLM.c repo

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions