Skip to content

add option to run vf-eval with prepared completions or by providing ground truth #215

@rasdani

Description

@rasdani

add option to run vf-eval in "offline" mode with prepared completions or by providing ground truth.
helps with testing and avoids inference, if not necessary.

"answer" can already be provided by dataset, just pass it through to vf-eval.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions