Issues with SVG image rendering

[apadom]

Hi,

I have a project derived from the demo6 from ST on the STM32U5G9J-DK1 devkit, where I want to display SVG images. The only difference is that the rounded display of the original kit has been substituted with another rounded display of the same resolution (480x480 pixels) but smaller. The PCB of the display is then different as well as the display driver chip which is driven through MIPI-DSI commands (and not using video mode of MIPI-DSI like the original demo is). I'm using latest version of TGFX 4.25.0.

All the subdemos of this demo6 project are working the same as on the original display, but not the SVG subdemo. All the .SVG images of this specific sub demo, which should animate and start bouncing on the screen, are simply not rendered at all. I only see the background image which is a .PNG file. And I can come back on the main screen as the touch display is working.

I tried to strip down this demo6 project and display a simple background .PNG image and on top of this last, some .SVG images. The background image is again displayed as it should, but none of the .SVG images.

As I've seen some other having difficulties with .SVG images I tried to change the portrait/landscape mode, without seeing any change. I tried to modify the canvas buffer size, and I didn't see any change. I checked the code inside MX_DCACHE2_Init() function and I have the two lines which were advised there.

I'm running out of ideas on the reasons why .SVG images cannot be rendered. So far, the only difference I can see is around these MIPI-DSI commands usage but since all the rest of subdemos are working as expected, I don't think that this can be related. There must be something not going well before this stage, when all the images should be rendered inside the framebuffer. But I don't understand why this is related to .SVG images rendering.

Any idea or hint are welcome.

[unsigned_char_array]

Have you tried rendering a static SVG image? Please include the code.

[apadom]

As written in my post, yes. I did try to simply put .SVG images on top of a .PNG image as a background. So all static. And the results were the same.

[unsigned_char_array]

Does it fail to render in the simulator too?
Please share code so we can reproduce.

[apadom]

In all cases everything is rendered correctly under the simulator.
The issue only happens on the target.

Prior to share code with you, I'm sorry to ask you this but, you're not an ST employee, that's correct?

By the way, it's mainly the demo6 project which exists inside the designer for this specific board. And then some changes to use the MIPI DSI commands mode instead of the video mode. 

 

[unsigned_char_array]

 

---

@apadom wrote:

Prior to share code with you, I'm sorry to ask you this but, you're not an ST employee, that's correct?

---

I'm not. Since you mentioned you basically stripped an example project I don't understand why you are hesitant sharing code.

 

---

@apadom wrote:

In all cases everything is rendered correctly under the simulator.
The issue only happens on the target.

---

Interesting. If the SVG wasn't static I would suspect it might not render quickly enough. But since it is static I don't understand why it is not rendering.

Perhaps cache is not cleared? Do you use double buffering? Have you configured enough stack? What about only using a very basic SVG image such as a single square.

[apadom]

> I'm not. Since you mentioned you basically stripped an example project I don't understand why you are hesitant sharing code.

I have two versions derived from this demo6 project. 

The first one is a stripped down version of this demo6, on which I added new screens and artwork from a designer. I cannot share this work without an NDA. Therefore my question. I'm sure you understand.

I don't have much resources taken in this first project compared to the original demo6. I would even say that it's less since I have less assets and of smaller size in bytes (and SVG are even less complex then what you see in this demo6). And I have a mix of .PNG and .SVG. And they all render correctly under the Simulator. And only on the target all .SVG assets are not rendered at all. 

It's very simple. There's no animation. It's a simple .SVG image object dropped on top of a .PNG background.

The second version is the demo6 project you can find on designer, with minimal adaptations to make the display and touchscreen work, as I explained in my original post. The only changes linked to the new display are this MIPI-DSI commands.

I moved back to the original demo6 just to check that it was not my screens design or their contents which would explain this weird behavior on the target. But obviously not, since the demo6 and its specific SVG demo isn't working on target as well.

> Perhaps cache is not cleared? Do you use double buffering? Have you configured enough stack? What about only using a very basic SVG image such as a single square.

Yes, double buffering is being used.
The .SVG image is basic in first project: a circle filled with a color and a very simple vector icon in it.
You mention the stack size: does a .SVG image rendering would be more complex that it would require more code and calls? 

You mention the cache not being cleared. Are you referring to the same SRAM cache disabling as what I linked in my original post?

[unsigned_char_array]

---

@apadom wrote:

> Perhaps cache is not cleared? Do you use double buffering? Have you configured enough stack? What about only using a very basic SVG image such as a single square.

Yes, double buffering is being used.
The .SVG image is basic in first project: a circle filled with a color and a very simple vector icon in it.
You mention the stack size: does a .SVG image rendering would be more complex that it would require more code and calls? 

---

Yes. That's my reasoning. I don't know if you use FreeRTOS or a loop. But you need to make sure the rendering task has enough stack. If it doesn't there is no guarantee it will work. I recently checked my stack usage and found out I didn't have enough stack so I needed to increase it. I think I got around 3 or 4k of stack usage so I set my stack to 8k just in case.

 

---

@apadom wrote:

You mention the cache not being cleared. Are you referring to the same SRAM cache disabling as what I linked in my original post?

---

I don't know how you configured your MPU and if you have caching enabled or not. I have a project where I have external RAM and flash that requires cache for performance (otherwise there were artifacts, even with double buffering). But the MPU configuration was complicated. And I needed to write my own cache clear functions that only cleared cache of the parts of the frame buffer that were changed. So either disable it completely or configure it.properly.

[apadom]

I remarked something a bit odd regarding the .SVG images part of the demo6:

For all .PNG files, there are some settings you can alter.
But nothing for .SVG files.

If I would want to put .SVG resources in a specific section (external or internal flash), it doesn't seem possible there. Same for the rotation directly from the designer.

I guess it means that .SVG resources location need to be managed by user code.
Or? 

[unsigned_char_array]

---

@apadom wrote:

I remarked something a bit odd regarding the .SVG images part of the demo6:

For all .PNG files, there are some settings you can alter.
But nothing for .SVG files.

That's because an SVG image is not stored as an image. SVG is stored as render instructions (basically a list of primitive shapes to render sequentially). Images can either be stored as uncompressed bitmaps or some type of compression that requires the mapping of a color index to a palette(the chromart peripheral supports color mapping). But for SVG that is not possible. Storing them as render instructions is a form of compression. But it takes a lot more time to render.So you are trading storage memory vs time.

[apadom]

@unsigned_char_array 
Thank you for your explanations.

So if I understand you well, these instructions for the rendering of .SVG resources, are part of the code in all cases? 
Why not considering these instructions in an external RAM (or Flash) for instance? For me the reading of instructions in an external RAM (or Flash), in order to get the final rendered results should be possible, no?

My concern here is that, if I create all my assets to .SVG files in order to space storage space (vs bitmap formats), regardless of this extra time for the rendering (which I can afford), the fact that I can't decide where I'm locating these assets, for instance on an external RAM (or Flash), will be an issue in the end.

Do you agree?

[apadom]

@unsigned_char_array 
Thank you for these explanations on the cache and the stack. I'll have a special look at these.

[mathiasmarkussen]

Regarding your inability to show SVG images, I suspect you may have disabled or downprioritized the GPU2D error interrupt. That interrupt is not only used for errors, and is needed to initialize the SVG rendering logic. You should have that enabled, and I would advise giving it the same priority as the normal GPU2D interrupt.

If that does not solve your issue, can you share how and where you hook in to transfer data to your screen?

 

[apadom]

Thank you @mathiasmarkussen for your answer.

Both are with the same level of interrupts as you highlighted.

I will send you privately the relevant code you asked.
I just have to fix a hardware issue prior to that.

[mathiasmarkussen]

Another thing to not is that the SVGs are rendered to a stencil buffer by the GPU before being transferred to the frame buffer. You may need to handle that in you linker script, if you are not using the ones provided with the Board Setup. For example, for gcc, we include this:

 FramebufferSection (NOLOAD) :
 {
 *(TouchGFX_Framebuffer TouchGFX_Framebuffer.*)
 *(.gnu.linkonce.r.*)
 . = ALIGN(0x4);

 *(Nemagfx_Stencil_Buffer Nemagfx_Stencil_Buffer.*)
 *(.gnu.linkonce.r.*)
 . = ALIGN(0x4);

 *(Nemagfx_Memory_Pool_Buffer Nemagfx_Memory_Pool_Buffer.*)
 *(.gnu.linkonce.r.*)
 . = ALIGN(0x4);
 } >RAM

 

[apadom]

@mathiasmarkussen 

I think you had a good catch here :)

I checked the differences in the linkerscript file between what's generated for this demo and what was modified by us for our particular display. And there's exactly these sections in linkerscript which are handled differently:

That said, I'm discovering as well that our project is based on ThreadX instead of FreeRTOS. Probably because it comes from early trials with the A9 chip variant. I'm unsure if this can explain these differences. But at least I will now focus on this particular "nemagfx" component in the code.

I noticed something a bit odd: in the genuine demo6 for the G9 board setup, everything related to the framebuffer initialization is handled inside: 

\TouchGFX\Target\generated\TouchGFXGeneratedHAL.cpp

While for this stencil and memory pool buffer, everything is inside:

\TouchGFX\Target\generated\nema_hal.c

Is this related to this discussion? As it's the same "feature" (buffers initialization) I would have seen these at the same place. I have the feeling that NemaGFX is somehow treated apart.

As soon as I have my hardware fixed I will be able to test different memory initialization path for these two buffers for NemaGFX and I will update this post if the SVG rendering issue is fixed.

[mathiasmarkussen]

Nema_hal is for hardware abstraction, and especially initialization of GPU2D and its related buffers specifically, whereas TouchGFXGeneratedHAL/TouchGFXHAL are for handling the frame buffers that are actually drawn to the display. In general, the TouchGFX library takes care of copying from the nema buffers to the general frame buffers, so user code should be handled in TouchGFXHAL.cpp.

Regarding OS selection, that should not be an issue. You can change it to FreeRTOS using CubeMX, if you want, there is some information here: Real Time Operating System | TouchGFX Documentation

But ThreadX should work fine - for the other U5G9 discovery kit, we supply Board Setups for both FreeRTOS and ThreadX.

[mathiasmarkussen]

You can certainly add that line to your project, it would definitely not hurt. But as far as I remember this is related to us finding that the interrupt was not setup correctly to actually fire on some relevant errors, not related to this.

How do you update the screen? Have you set a window on the entire screen and sending the entire frame buffer on every vsync, or are you doing something else?

[apadom]

@mathiasmarkussen 

> Have you set a window on the entire screen and sending the entire frame buffer on every vsync

Yes, we have a TE (tearing effect) line coming back from the display module.
DSI interrupt is enabled to handle this TE signal. On every possible screen update we send one of the two complete buffers.
The MIPI-DSI works at 30fps.

Inside: \TouchGFX\Target\TouchGFXHAL.cpp

extern "C" {

 void HAL_DSI_TearingEffectCallback(DSI_HandleTypeDef* hdsi)
 {
 GPIO::set(GPIO::VSYNC_FREQ); // sets the GPIO for debugging and performance measurement
 HAL::getInstance()->vSync(); // counts vSync

 // signalVSync should be called from the display driver when the display is ready for the next frame.
 // VSYNC period indicates the end of the actual frame display and that the two buffers need to be swapped.
 OSWrappers::signalVSync();

 if(refreshRequested && !displayRefreshing)
 {
 refreshRequested = false;
 if(frontBufferIdx < 0){
 // swap the frame buffers
 GPIO::toggle(GPIO::MCU_ACTIVE); // used for debugging frame buffer swap
 frontBufferIdx = backBufferIdx;
 backBufferIdx = 1 - frontBufferIdx;
 }
 // Transfer pixel data.
 HAL_DSI_Refresh(hdsi);
 displayRefreshing = true;
 }else{
 if(!displayRefreshing){
 GPIO::clear(GPIO::VSYNC_FREQ); // clear the GPIO for debugging and performance measurement
 }
 }
 }

 void HAL_DSI_EndOfRefreshCallback(DSI_HandleTypeDef* hdsi)
 {
 displayRefreshing = false;
 if(frontBufferIdx >= 0)
 {
 // set the LTDC frame buffer in preparation for the next transmission
 // of pixel data by the DSI host
 setLtdcFrameBuffer(frameBufferAddresses[frontBufferIdx]);
 frontBufferIdx = -1;
 }
 GPIO::clear(GPIO::VSYNC_FREQ); // clear the GPIO for debugging and performance measurement
 }

 void HAL_LTDC_ErrorCallback(LTDC_HandleTypeDef *hltdc){
 // set the red LED in case of error
 HAL_GPIO_WritePin(LED_RED_GPIO_Port , LED_RED_Pin, GPIO_PIN_SET);
 }

 void HAL_DSI_ErrorCallback(DSI_HandleTypeDef *hdsi)
 {
 // set the red LED in case of error
 HAL_GPIO_WritePin(LED_RED_GPIO_Port , LED_RED_Pin, GPIO_PIN_SET);
 }

 void setLtdcFrameBuffer(uint32_t adr)
 {
 // sets the LTDC frame buffer
 // this frame buffer will be transmitted by the DSI host
 __HAL_DSI_WRAPPER_DISABLE(&hdsi);
 LTDC_LAYER(&hltdc, 0)->CFBAR = adr;
 __HAL_LTDC_RELOAD_CONFIG(&hltdc);
 __HAL_DSI_WRAPPER_ENABLE(&hdsi);
 }

} // extern c

But most probably the SVG renderer doesn't produce anything in any of these two buffers.

I hope to get my hardware issue fixed by tomorrow so I can really try new things on the target.