RLVR AIM Event - Verifiers!

NOTE: ⚠️You will need a GPU for the following demo!⚠️

CREDITS:

This is an example extracted from the verifiers library. All credit to the work of the verifiers team!

Pre-Reqs

NOTE: You'll need to ensure you have uv installed before proceeding.

From the verifiers library Setup step - let's take the one we need:

uv add 'verifiers[all]' && uv pip install flash-attn --no-build-isolation

This will ensure we can train the model on our local machine using the verifiers trainer (using vf.GRPOTrainer) and environments.

Running the Example

Now, we can follow a few simple steps - we can install the environment, which, in this case - is the verifiers example environment for GSM8K.

NOTE: The environment is more than just the dataset, which we discuss in detail in the accompanying event!

uv run vf-install gsm8k

Next, we can train our model using the vf.GRPOTrainer on our GPU! You can dive deep into this process here in the verifiers documentation.

CUDA_VISIBLE_DEVICES=0 uv run accelerate launch --num-processes 1 \
    --config-file configs/zero3.yaml train_gsm8k.py

That's all! It's really that easy.

CONCLUSION:

Again, all credit to the verifiers library!