Any example of FIQ implementation under Linux?

[A.M.]

I am trying to achieve hard real-time under 100us on the STM32MP157f-eval, specifically, on the Cortex A7 dual core.

Note: I specifically target the Cortex A7 cores because the M4 is already taken for other tasks which requires much smaller latency.

What I did so far is:

• Build the Yocto openst-linux
• Add Preempt-RT patch
• Added several software: irqbalance, tuna, hiccups... to monitor and tune the system.
• Isolate one core
• Tested high-priority process, also IRQ. Tested with internal clock and external oscilloscope.
• Tried to disable most drivers

After all those efforts, the jitter improved but I still get ~500us jitter when applying load for enough time.

------------------------------

I found the following text which brings me to think that FIQ (Fast Interrupts reQuests) are available on this hardware and could possibly be a solution to my quest for low latency and low jitter.

https://bootlin.com/blog/fiq-handlers-in-the-arm-linux-kernel/

Is there some example code about using those FIQ? Ideally under Linux, for example to drive an GPIO.

Related work:

https://stackoverflow.com/questions/32194264/bcm2708-rpi-rasbpian-fiq-not-triggered

[PatrickF]

Hi @A.M.​ ,

did you use https://wiki.st.com/stm32mpu/wiki/X-LINUX-RT_OpenSTLinux_Expansion_Package ?

FIQ as 'fast' interrupt concept should not be used in recent ARM Cortex-A usages, especially when there is a GIC interrupt controller.

As far as I know, instead of IRQ/FIQ, there is now in GIC the concept of 'group' . In most implementations, group0 is mapped to 'I' for kernel/application drivers and group1 is mapped to 'F' for secure interrupts (i.e. managed by secure monitor or OpTEE).

FIQ with a recent Cortex-A using a GIC will not bring any visible latency gain. FIQ was originally (look back to pre-Cortex processors like ARM7 or ARM9) intended to be used by a single interrupt source, without any interrupt controller nor vectors to gain some cycles (i.e. reading interrupt controller to know the interrupt source and branching). This is not anymore relevant in much more complex systems including hundreds of interrupt sources, fast and multiple processors, large caches and security mechanisms.

Regards,

[A.M.]

Hi @PatrickF​, and thank you for your answer.

> "Did you use https://...";

yes, and i also added few software like Tuna, irqbalance, hiccups, etc..

99.99% of the time, the latency is under 50us, however, over long time I get >1.4ms. I managed to get down to a maximum latency <500us with core isolation, but some interrupts can not be moved, so the isolation is only partial.

My guess is that some driver take priority over my RT interrupt, and triggers this long latency. I see only one-two context switch (probably the testing application being switched to non-isolated core). It seems there is no memory paging.

On an other side, Linux does not seem to implement IRQ priority. Consequently, the approaches that I would like to try for improving jitter are:

1) FIQ

2) Direct access to the GIC to set higher priority to my target IRQ

3) Hook the IVT of Linux and give priority to my IRQ.

I found that FIQ has priority over normal IRQs, which would be ideal (even if the base 30us latency is not improved, having an interrupt with higher priority would probably solve the issue).

If FIQ is not the proper solution, what would be your recommended approach to achieve hard real time? (50us is fine, but I consistently get >500us when running my test with load (e.g. playing video) for 24h).

[Christophe Guibout]

Hi @A.M.​ ,

Could you please tell me on which ecosystem version you are?

If you're based on openstlinux-5.15-yocto-kirkstone-mp1-v22.11.23 (v4.1.0) with X-LINUX-RT expansion package (PREEMPT_RT patches), we observed a regression on latency since new random driver integrated in kernel-5.15.44 which involves non-interruptible activities on optee side: waiting a correction, I would recommend disabling CONFIG_HW_RANDOM_OPTEE from the kernel.

BR,

Christophe

[A.M.]

Hi @Christophe Guibout​ 

I am using the following:

Linux built with Yocto project from STM and applying the RT patch form STM.

I get the following version:

$uname -a

Linux stm32mp1-rt 5.15.67-rt49 #1 SMP PREEMPT_RT Mon Mar 27 08:17:56 UTC 2023 armv7l armv7l armv7l GNU/Linux

I don't think that the observed regression maters: with the default IRQ managed by GIC and Linux, many other drivers IRQs might get precedence on our RT software, thus, it's only a question of time to get 2 or maybe 3 IRQs in between, with a consequent jitter impact.

If I am correct, FIQ will always get priority on other IRQs right? so the jitter (not saying the latency) should be drastically reduced.

[Christophe Guibout]

Hi @A.M.​,

You're correct: FIQ will always get priority on other IRQs.

With Cortex A7 running both Optee and Linux kernel, FIQ are dedicated to secure world, IRQ are for normal interrupts:

https://optee.readthedocs.io/en/latest/architecture/core.html#native-and-foreign-interrupts

Since STM32 MPU ecosystem release v4.1.0, optee code size has slightly increased, and more code paging could be seen leading to a latency degradation on MP15 every time a request to optee is performed (compared with v4.0.0).

The solution will be to put optee code into DDR for MP15 (this is already the case for MP13), and we plan to deliver it in the next ecosystem release (v5.0.0).

BR,

Christophe

[nettercm]

Hi @A.M.   Did you get this resolved, i.e. are you able to use the FIQ?   I'm using a STM32MP13x  (single core;  no co-processor) and I'm also interested in using FIQ for certain use cases

[A.M.]

Hi @nettercm,
No, we went for another solution (using xenomai). Mainly because FIQ makes use of GPLv2 headers that is incompatible with our proprietary requirement.
Consequently, my previous message was between the last efforts we spend on attempting FIQ.

If you would achieve something, share it please, I would be very interested in knowing how it works.

A-M.