Rendering time is abysmal

[jchernus-fikst]

Hi again, (posting twice today)

I am looking at the performance of my STM32H747 system to better understand it. I didn't think there were any problems because I am not visually seeing any artifacts, but I always like to check. As it turns out, my system's rendering times are abysmal.

I have many different screens in my application, and for this test the system cycles from one to another every 3 seconds. I used the Performance Measurement pins to measure VSYNC_FREQ, RENDER_TIME, FRAME_RATE, and MCU_ACTIVE. 

VSYNC looks healthy, measured at 62,352HZ (set to ~62,4Hz).

The MCU appears to be barely active - there are 10ns or less blips occurring frequently.

Frame rate is a consistent 3 seconds (I don't know if that's normal).

But the rendering time is abysmal! The screens are 480x320 and some take hundreds of milliseconds to render when switching screens.

This is the system as it's switching screens:

This is system when there are no updates to render:

The simplest screen takes 25ms to render and it contains a 480x320 background image. The most complicated screen takes 218ms to render and it contains a 480x320 background image, 8 text areas (some contain wildcards), 3 flex buttons, and a few icons, (and one hidden background image that's not being used).

I sense that I am doing something very wrong, I wanted to ask if there are common places to look. 

Things that come to mind for me,

• speed of writing or reading from RAM (I'm presently using a burst length of 1, and I can put in the work to get this up to 8),
• caching the background images - sometimes the background don't even change between screens,
• I have additional images on each screen, but they're hidden using the left-hand panel - these shouldn't be affecting the render time, right?
• I am using vector fonts (and a custom font at that - Roboto Regular and Robot SemiBold),

Thanks!

Julia

 

 

[GaetanGodart]

Hello @jchernus-fikst ,

 

 

Here is a typical measurement from a random board with a regular UI:

Here is the example of measurement that you find on our documentation:

 

As you can see, they are very similar unlike your measurement.

The VSYNC gets high about 60 times per second (you do that right).
The render time starts when the VSYNC gets high (you do that right).
The framerate is the same as the VSYNC (not for you).
The MCU _ACTIVE starts when VSYNC gets high, aka we start rendering as soon as possible (this is not the case for you, perhaps your MCU is busy with something else).

Your framerate should be 60 per second like your VSYNC.
What happens if you try to change screen every second? Are you able to do that or did you set up your board to only be able to render every 3 seconds (I doubt that because there is a render time peak every 16ms)?

Your render time picks every 16ms showing that TouchGFX want to render something every 16ms but it is extremely fast to render, perhaps there is nothing to render as MCU_ACTIVE is not even getting a peak everytime.

Regarding the screenshot during the transition, we can see that the render starts (and last 200+ ms) and once it is finished it does not transfer directly, it waits for the next VSYNC to start a render and then transfer (because FrameRate changes). It is normal because to transfer we wait for VSYNC so we skip frames if the rendering takes too long.

However, we see that when the rendering finishes, the MCU is not even active so there is definitely something wrong.

 

You say that it takes 25ms to render a simple image in 480*320*2 bytes. That is way too much, perhaps you are right, this is a memory limitation if this asset is in external memory.

Can you test to put this asset in internal memory and see if it renders faster?
You can also try to just make a screen with boxes, circles, lines and other widgets that do not require to be stored in memory (this way the memory won't be used, only the rendering will be limiting).

 

It is recommended to disable DCache until you have a stable project.

 

If you have an image that is hidden by some other element(s), the hidden image will not impact the render time as long is the other element on top do not have an alpha value.
So if everything on top is RGB565 it will be fine, but if the element on top is ARGB8888 then we will do blit operations.

 

Vector font do increase render time but your render time is already abnormal for a simple image (25ms).
Note that texture mapper and VSG (including vector font but I think it depends on the size) takes quite some time to render.

 

Can you check the priority of your tasks? Perhaps there is something that has a way higher priority than TouchGFX and is taking all the RTOS's resources.

 

I hope you can investigate and try to change screen every tick (60 times per seconds) with a simple UI.

 

 

Regards,

[jchernus-fikst]

Thank you so much, @GaetanGodart, for your lengthy reply. I have finally found some time to work on this task, here is what I've found, and a few answers to your questions:

1. TouchGFX Task is the highest priority task (3) on the core that it's running in (M7). There is a priority 4 task and some priority 5 interrupts.
2. D-Cache and I-Cache are both disabled.
3. I have turned off vector font rendering, as we don't actually need it.

Experiment 1 - Images in Internal Flash

I reduced the GUI to two screens with one image each (240 x 320, as I can't fit two images of size 480 x 320). I placed these images into internal flash. I have a full-size invisible flex button covering each screen which I use to change screens.

Other very wonky things happen, but the render time is reduced to 2ms when using double framebuffers. Each of the captures below show one change of screen. I included two because the FRAME_RATE behaviour toggles, and I wanted to show both.

Wonky things that happen:

• VSYNC_FREQ experiences a glitch - you'll see a discontinuity in its cycle
• I would expect every RENDER_TIME to be 2ms but three of them are and the rest are a negligible amount (12us)
• FRAME_RATE is still behaving incorrectly
• MCU is still never active

Experiment 2 - Image in external vs internal flash

For this one, I have one screen with no images and one invisible flex button (that I use to change screens). And the second screen has an image of a turtle (stored in internal RAM) and the same invisible flex button.

The capture below is for moving from the blank screen to the turtle screen.

The capture below is for moving from the turtle screen to the blank screen.

Experiment 3 - Transitions

I used the same screens as experiment two, but used a block transition every second:

And here with a wipe transition:

Experiment 4 - Shapes instead of images

Per your recommendation, I tried adding shapes instead of images to see how this affects the system.

The capture is seen below, with 12ms of render time on the double framebuffer setup.

I feel that something is drastically wrong with the setup of our project, but I don't know if I should be looking at hardware or software for the culprit. Would love to hear any advice you have! 

Thanks,

Julia

[jchernus-fikst]

@GaetanGodart any thoughts on the matter?

[mathiasmarkussen]

Hello,

TouchGFX will only render areas of the screen that has actually changes since the last render. This is the reason that you render time is 0 for many frames. If you want to continously render a screen for testing purposes, you can create an interaction on it as shown in my screen shot.

The frame rate signal is a representation of how often a newly rendered frame is shown on the screen (the buffers are swapped for double buffer). It will toggle when a render starts and toggle again at the vsync after the render finishes. For LTDC screens without double buffering you will experience tearing if render (half period) time is higher than vsync(full period), for double buffer it will lead to lower frame rates, so choppy animations and so on.

It looks a lot like a memory bottleneck on either you flash or ram.

Can you elaborate on your external memory setup?

Where are your frame buffers placed?

Have you enabled DMA2D and/or GPU2D for TouchGFX?

I believe it should be fine to enable the icache.

[jchernus-fikst]

Hi @mathiasmarkussen, thank you for your reply!

You colleague stated that Frame Rate should match Vsync, and that something seems very wrong about the MCU never being used - this is why I've been trying to investigate what is wrong with our system. 

What you are saying makes sense to me, the point of invalidating objects in TouchGFX is so that you don't re-render everything on every frame. 

We have sped our system up, so nothing is taking 200+ ms to render, but we are still seeing some take upwards of 20ms.

Can you please confirm, aside from the rendering taking too long, does this look like a healthy performance from our system? I am changing the 320 x 480 screen twice during this time - each screen has a full-size background image as well as other image-based assets (buttons, non-vector fonts, images) on top.

Here is a zoomed-in capture of the first screen change, you'll see that Vsync gets stretched:

You'll also notice that there are several long renders, which I didn't expect for one screen change.

If you say this looks healthy, I'll speed our QSPI flash up a little bit more and call it done!

Thanks again for your help,

Julia

PS: Replies to your questions:

Can you elaborate on your external memory setup? QSPI Flash @ 100MHz, FMC (16-bit) SDRAM @ 100MHz and we are not using burst writes (should we be?)

 

Where are your frame buffers placed? SDRAM, like so:

Have you enabled DMA2D and/or GPU2D for TouchGFX? Yes, see above

[mathiasmarkussen]

Hello, I'm back from easter break :)

How does your performance look if you invalidate the entire screen as I showed earlier each frame in your UI?

I would like to confirm that each frame would then take the same amount of time as changing to and from it, unless you are using transitions between frames?

I can see you have had MCU load showing in your measurements earlier. I would expect MCU load to be very low in a scenario like this, since the MCU hands off all asset transfers to DMA2D, which transfers the assets directly to flash. I would expect at least a little bit of MCU work for each frame though, especially with overlapping elements. How quickly are you sampling the measurements?

Regarding the stretched vsync, my guess is that the screen changes are triggered by touch screen input, and you are reading those in an interrupt triggered by a GPIO pin with higher priority than the ltdc interrupts that controls the vsync output.

If that is the case, I would recommend just setting a flag in the interrupt and checking the state of that flag in STM32TouchController::sampleTouch() and reading the touch points and resetting the flag if the interrupt has set the flag.

You can also try to set another trigger for the transition, for example a hardware button press or a certain amount of ticks passed.

[Marc_LM]

I might know why the MCU active pin is never toggled.

Check for "MCU_ACTIVE" or "touchgfx::HAL::getInstance()->setMCUActive".
For me, I am using FreeRTOS and it toggles the pin on every IdleTaskHook().

I recommend checking Designer's Demo9 with any of the dev-kit you have.
It should contain this macro.

[jchernus-fikst]

Hey @mathiasmarkussen,

Thanks for your reply. You said:

> "Regarding the stretched vsync, my guess is that the screen changes are triggered by touch screen input, and you are reading those in an interrupt triggered by a GPIO pin with higher priority than the ltdc interrupts that controls the vsync output."

That's absolutely what's happening, thank you for your insight here. I am going to fix this promptly!

Once I fix this, I'll measure to confirm, at which time I'll get answers to the other two questions you asked.

Thanks,

Julia

[unsigned_char_array]

FRAME_RATE toggles when changing something on the screen. If there are 2 buffers I think it only toggles when swapping frame buffers. If something changes each frame then FRAME_RATE should be half the refresh_rate/vsync rate of the display. If nothing changes it doesn't toggle. If rendering takes too long it can take longer for it to toggle. Effectively a frame skip.
MCU_ACTIVE needs to be added manually if I recall correctly. In my application I don't use an operating system, so I created my own idle loop. The OSWrappers call the idle loop when it is waiting for something to finish (usually DMA). this takes more stack, so be aware, but it allows you to do things during idle. Pseudocode:

main
{
...
touchgfxRegisterIdleTask(mainLoop);
while(1)
{
 mainLoop();
 MX_TouchGFX_Process();
}
}

static void mainLoop()
{
HAL_GPIO_WritePin(MCU_IDLE_GPIO_Port, MCU_IDLE_Pin, GPIO_PIN_SET);
// handle non-touchgfx things here
HAL_GPIO_WritePin(MCU_IDLE_GPIO_Port, MCU_IDLE_Pin, GPIO_PIN_RESET);
}

void MX_TouchGFX_Process(void)
{
 touchgfx_taskEntry();
}

void touchgfx_taskEntry()
{
if (OSWrappers::isVSyncAvailable())
{
 hal.backPorchExited();
}
}

static void (*idleTask_p)(void) = nullptr;
extern "C"
{
void touchgfxRegisterIdleTask(void (*idleTask)(void))
{
idleTask_p = idleTask;
}
}

void OSWrappers::takeFrameBufferSemaphore()
{
//while (fb_sem);
//fb_sem = 1;

if(fb_sem)
{
if(idleTask_p != nullptr)
{
//touchgfx::HAL::getInstance()->setMCUActive(false);
//HAL_GPIO_WritePin(MCU_IDLE_GPIO_Port, MCU_IDLE_Pin, GPIO_PIN_SET);

while(fb_sem)
{
idleTask_p();
}
//touchgfx::HAL::getInstance()->setMCUActive(true);
//HAL_GPIO_WritePin(MCU_IDLE_GPIO_Port, MCU_IDLE_Pin, GPIO_PIN_RESET);
}
else
{
while(fb_sem);
}

}
fb_sem = 1;
}

/*
* Release the frame buffer semaphore.
*/
void OSWrappers::giveFrameBufferSemaphore()
{
fb_sem = 0;

// //HAL_GPIO_WritePin(MCU_IDLE_GPIO_Port, MCU_IDLE_Pin, GPIO_PIN_SET);
// HAL_GPIO_WritePin(MCU_ACTIVE_GPIO_Port, MCU_ACTIVE_Pin, GPIO_PIN_SET);
// uint32_t begin = __HAL_TIM_GET_COUNTER(&htim2);
// uint32_t diff;
// do{
// diff = __HAL_TIM_GET_COUNTER(&htim2) - begin;
// }while(diff<16);
// //HAL_GPIO_WritePin(MCU_IDLE_GPIO_Port, MCU_IDLE_Pin, GPIO_PIN_RESET);
// HAL_GPIO_WritePin(MCU_ACTIVE_GPIO_Port, MCU_ACTIVE_Pin, GPIO_PIN_RESET);

if(idleTask_p != nullptr)
{
idleTask_p();
}
}

void OSWrappers::tryTakeFrameBufferSemaphore()
{
fb_sem = 1;

// //HAL_GPIO_WritePin(MCU_IDLE_GPIO_Port, MCU_IDLE_Pin, GPIO_PIN_SET);
// HAL_GPIO_WritePin(MCU_ACTIVE_GPIO_Port, MCU_ACTIVE_Pin, GPIO_PIN_SET);
// uint32_t begin = __HAL_TIM_GET_COUNTER(&htim2);
// uint32_t diff;
// do{
// diff = __HAL_TIM_GET_COUNTER(&htim2) - begin;
// }while(diff<32);
// //HAL_GPIO_WritePin(MCU_IDLE_GPIO_Port, MCU_IDLE_Pin, GPIO_PIN_RESET);
// HAL_GPIO_WritePin(MCU_ACTIVE_GPIO_Port, MCU_ACTIVE_Pin, GPIO_PIN_RESET);
//
if(idleTask_p != nullptr)
{
idleTask_p();
}

}

 

In my application (custom board) enabling the cache was essential for performance. Setting it up was a bit of a hassle since you need to modify the linker script and also enable the MPU to mark which memory regions need to be cached. Without it the LTDC was reading memory faster than the memory controller could process causing artifacts. Also rendering took too long.