Skip to content

Conversation

dvrogozh
Copy link
Contributor

This PR is built on top of #831 which provides stand-alone implementation for FilterGraph. To review cuda specific change, navigate to the last commit as the first 2 are identical to #831. I'll keep it as a draft for initial discussion till #831 is hopefully merged.

The changes in the last commit are:

  1. Added DeviceInterface::initializeFiltersContext() API which returns device specific FilterGraph initialization settings
  2. Extended SingleStreamDecoder to call FilterGraph pipeline if DeviceInterface::initializeFiltersContext() returned valid description.

Basically idea behind this change is the following. Let device interface to perform trivial and performance optimized conversions in the convertAVFrameToFrameOutput() method. If device interface can not handle conversion in the convertAVFrameToFrameOutput(), it can setup ffmpeg filters pipeline by returning valid description in initializeFiltersContext().

I tested the pipeline on 10-bit videos, h264 and h265. However, this setup should be valid for 12-bit videos which I did not try.

Note that this PR does not actually contradict another attempt to add 10-bit support in #790 as using filters is more generic approach and #790 usage of NPP can be considered as optimization for specific cases.

Finally, I would like to pay attention on 2 things:

  1. On ffmpeg-n4.4 filters pipeline walls back to CPU as scale_cuda does not support format conversion in this ffmpeg version (was added from n5.0)
  2. More work might be needed including changes on ffmpeg level to actually align output frames with CPU implementation. I do see the difference in the outputs likely due to different handling of color standards. This is something I was not able to overcome at the moment.

CC: @scotts @NicolasHug @eromomon

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Aug 26, 2025
@dvrogozh
Copy link
Contributor Author

I forgot to mention that I did not change cpu device interface yet in a similar way to cuda (initializeFiltersContext() not implemented for cpu). This would be a next step if this design will be accepted.

Dmitry Rogozhkin and others added 4 commits August 28, 2025 18:01
FFmpeg filter graphs allow to cover a lot of use cases including
cpu and gpu usages. This commit moves filter graph support out of
CPU device interface which allows flexibility in usage across
other contexts.

Signed-off-by: Dmitry Rogozhkin <[email protected]>
@dvrogozh
Copy link
Contributor Author

I've added one more commit to update CPU device interface to use filters path introduce in SingleStreamDecoder and DeviceInterface. Still a draft as #831 needs to be reviewed first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant