How to use STM32MP157C GPU Hardware Acceleration with FFmpeg

[farid]

We want to use FFmpeg for video processing (encoding, decoding, scaling, format convertion, etc) on a STM32MP157C based custom board. 

I understand STM32MP157C has a display controller and a Vivante GPU that supports OpenGL-ES and OpenVG to accelerate graphics-based applications.

We are not using any display (LCD/HDMI) with the board and the display controller is disabled in device tree.

We want to use the GPU's hardware acceleration feature for video processing through FFmpeg.

We will compile the FFmpeg from source and want to include STM32MP157C GPU Hardware accelerator, that can be access by "-hwaccel" option in ffmpeg command.

We don't know yet how to implement this. I looked at PM0263 (STM32MP157 GPU application programming manual) but not much helpfull.

Is there any OpenGL API that exposes GPU's hardware acceleration that we can integrate with ffmpeg?

Has anyone used STM32MP157C GPU with FFmpeg or any video processing application?

Any help is much appreciated.

[Erwan SZYMANSKI]

Hello @farid​ ,

In fact, the -hwaccel option of FFmpeg in the context of a PC will target a specific hardware encode/decode video IP, which is in the graphic card but not in the GPU itself. It gives you the same interface/API to call this feature, but it is important to distinguish the two.

In the context of our STM32MP15, we have a GPU but we do not include this specific IP to make video encoding/decoding. As a result, you will not be able to offload video management on our GPU with FFmpeg. Our graphic team already looked for another solution to do video encoding directly with our GPU, but there were not satisfactory results.

This task must be done in software.

I hope these explanations will help you.

Kind regards.

Erwan.

[farid]

Hi Erwan,

Thanks for your response.

I understand we can not use FFmpeg. Can we access GPU's hardware acceleration feature directly through OpenGL-ES 2.0 without Wayland/Weston framework and no display connected?

Thanks for your help.

/Farid

[Erwan SZYMANSKI]

Hi @farid​ 

We can access to OpenGLES 2.0 API (located in /usr/include/GLES2/), which is the same as described in OpenGLES 2.0 reference manual. However, without Wayland/Weston or without any graphic context in general, I don't know if you can call the API properly without any trouble.

I would like to add that I have spoken with the graphics team about your topic, and they remind us that our GPU is not designed to do any work other than graphics. Trying to offload another job to it will be difficult and the performance will not be higher than in software.

I hope this information can help you.

Erwan