New feature requests

[XkunW]

New features we'd like to support:

• Tool Calling
• Structured Outputs
• Reasoning Outputs
• Automatic Prefix Caching
• Speculative Decoding
• Quantization

For any other features not mentioned here, please add them to this thread

[scarere]

Writing this here so that i don't forget. Track slurm job ids in a log file. Let users call vec-inf status list or something of the like to see all jobs started with vec-inf so that i don't need to copy paste the slurm-id or run squeue to find it.