Support for Passing Codec-Specific Options

[mrezabayat]

TorchCodec currently provides no way to forward codec-specific parameters to libavcodec when initializing a decoder via avcodec_open2. This limits advanced use cases that require control over backend behavior, decoding quality, or performance tuning.

TorchCodec embeds FFmpeg's libavcodec into its native C++ backend (_core). When a video stream is added, VideoStreamOptions is populated by custom_ops.cpp (Python binding) with preset fields:

Width, height

Thread count

Dimension order

Optional color-conversion library

Target device

These are passed to SingleStreamDecoder::addVideoStream, which sets up the decoder and calls:

avcodec_open2(codec_context, codec, /* options = */ nullptr);

Since no AVDictionary is passed, libavcodec uses default settings. This prevents codecs from receiving extra options.

Proposed Solution

To enable codec-specific parameter forwarding:

Extend VideoStreamOptions
Add new fields (or a generic options dictionary) for key-value parameters.

Update custom_ops.cpp
Modify the Python binding to accept and forward these additional options from Python to C++.

Update SingleStreamDecoder

    Construct an AVDictionary* from the new options.

    Pass it to avcodec_open2() as the third parameter.

Benefit

This enables users to:

Fine-tune codec behavior

Make TorchCodec more extensible and FFmpeg-aligned

Willing to Contribute

Yes – I’m interested in implementing this feature and can start with guidance or approval from maintainers.

[NicolasHug]

Hi @mrezabayat , thank you for the feature request.

To help us scope the feature, can you share more details about the codec-specific options you would like to use, and which use-cases you hope to enable from those?

[mrezabayat]

Hi @NicolasHug

Thanks for your response!

The motivation for this feature is to support VC6, a proprietary decoder we’ve integrated via FFmpeg. For example, I need to pass decoder-specific options like:

backend – to control GPU vs CPU decoding

loq – to select decoding quality level (LoQ-X), a speed–quality tradeoff used internally by VC6

Since these are registered as AVOptions on the VC6 decoder, they can only be applied via AVDictionary passed to avcodec_open2, which TorchCodec currently does not support.

To help clarify this further, here’s a concrete example from an upstream FFmpeg decoder — the NVIDIA cuvid decoder — that registers multiple runtime decoder options through an AVOption array:

static const AVOption options[] = {
    { "deint",    "Set deinterlacing mode", OFFSET(deint_mode), AV_OPT_TYPE_INT,   { .i64 = cudaVideoDeinterlaceMode_Weave    }, cudaVideoDeinterlaceMode_Weave, cudaVideoDeinterlaceMode_Adaptive, VD, "deint" },
    { "weave",    "Weave deinterlacing (do nothing)",        0, AV_OPT_TYPE_CONST, { .i64 = cudaVideoDeinterlaceMode_Weave    }, 0, 0, VD, "deint" },
    { "bob",      "Bob deinterlacing",                       0, AV_OPT_TYPE_CONST, { .i64 = cudaVideoDeinterlaceMode_Bob      }, 0, 0, VD, "deint" },
    { "adaptive", "Adaptive deinterlacing",                  0, AV_OPT_TYPE_CONST, { .i64 = cudaVideoDeinterlaceMode_Adaptive }, 0, 0, VD, "deint" },
    { "gpu",      "GPU to be used for decoding", OFFSET(cu_gpu), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, VD },
    { "surfaces", "Maximum surfaces to be used for decoding", OFFSET(nb_surfaces), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, VD | AV_OPT_FLAG_DEPRECATED },
    { "drop_second_field", "Drop second field when deinterlacing", OFFSET(drop_second_field), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VD },
    { "crop",     "Crop (top)x(bottom)x(left)x(right)", OFFSET(crop_expr), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, VD },
    { "resize",   "Resize (width)x(height)", OFFSET(resize_expr), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, VD },
    { NULL }
};

To use any of these options (gpu, deint, crop, etc.), FFmpeg users typically pass them through a dictionary passed to avcodec_open2() — which TorchCodec currently does not expose.

By allowing options to be forwarded via VideoStreamOptions and propagated into an AVDictionary, TorchCodec could support not only VC6-specific configuration, but also advanced tuning for decoders like libdav1d, libvpx, cuvid, etc.

Let me know if you need more details, and I’m happy to contribute the implementation if it fits into your roadmap!