ST SSD Mobilenet v1 return an IRQ error

[Creator]

Hi,

 

When changing from a Tiny YOLOv2 model to ST SSD Mobilenet v1 I strangely got the following error:

Bus interface interrupt
BUSIF0 ERR: 0x10
BUSIF1 ERR: 0x0
ATON_STD_IRQHandler()@763: irqs=0xlx
assertion "0" failed: file "Y:/VM/stm-workspace/fall-guard-v2.0.0/Lib/AI_Runtime/Npu/ll_aton/ll_aton_runtime.c", line 767, function: __LL_ATON_RT_IrqErr

 

I tried other models but only Mobilenet v1 always returns this error. Is there a specific config required?

 

Thanks,

Gino

[Creator]

My testing kind of showed that the issue is related to the automated generated network.c file.

 

Tthe object detection of the getstarted project works. When using this network.c file in my project the error doesn’t occur.

 

has someone experience in using a st ssd mobilenet v1 model in his multithreaded application?

 

thanks

[Julian E.]

Hello @Creator ,

 

Can you provide a bit more context?

What do you mean changing  a Tiny YOLOv2 model to ST SSD Mobilenet v1? 

Are you talking about the getting started object detection for N6? Or with ST Model Zoo?

 

Can you described the steps you followed and which models worked please?

 

have a good day,

Julian

 

 

[Creator]

Hi @Julian E. 

I've trained a st_ssd_mobilenetv1 model for object detetection. My purpose is running it on the STM32N6570-DK.

Testing with the getting started object detection sample gave good results.

 

However my application needs multithreaded functionality. Therefore I was changing the ai-people-detection-v1.0.0 sample from ST. This sample is standard using the Tiny YOLO V2 person detetction model.

Running any st_ssd_mobilenetv1 model (even the modelzoo models) the __LL_ATON_RT_IrqErr occurs.

In this part of the code it goes wrong:

/* run ATON inference */
 ts = HAL_GetTick();
 /* Note that we don't need to clean/invalidate those input buffers since they are only access in hardware */
 ret = LL_ATON_Set_User_Input_Buffer_Default(0, capture_buffer, nn_in_len);
 assert(ret == LL_ATON_User_IO_NOERROR);
 /* Invalidate output buffer before Hw access it */
 CACHE_OP(SCB_InvalidateDCache_by_Addr(output_buffer, nn_out_len));
 ret = LL_ATON_Set_User_Output_Buffer_Default(0, output_buffer, nn_out_len);
 assert(ret == LL_ATON_User_IO_NOERROR);
 LL_ATON_RT_Main(&NN_Instance_Default);
 inf_ms = HAL_GetTick() - ts;

 /* release buffers */
 bqueue_put_free(&nn_input_queue);
 bqueue_put_ready(&nn_output_queue);

LL_ATON_RT_Main(&NN_Instance_Default);

==> This call triggers the issue. It's referenced via LL_ATON_Set_User_Output_Buffer_Default to the auto generated network.c file.

 

Even tough the sample is prepared for st_ssd_mobilenetv1 postprocessing, I wonder if anyone has tested this.

 

Thanks,

Gino

 

 

 

 

[SlothGrill]

Hi @Creator ,

sorry to pop here from nowhere, on my side, i've tested to do a "validate on target" with the .tflite you provided.

It seems to work on the 2.1.

 

You say the `.c` file generated by 2.0 works and replacing it with the one generated by 2.1 does not work, right ? 

If you do not do anything else, this kind of behaviour is sound:

• The .c file generated is generated by a different version of the Neural-Art compiler between v2.0 and v2.1
• The .c file calls functions from the ll_aton files (that, too, have changed between v2.0 and v2.1)
• Eventually, the .c file also calls functions that comes in the "NetworkRuntimexxxxxx.a" library

(in short, code generated by 2.1 expects some behaviours from the 2.1 ll_aton files/or the 2.1 lib, but those behaviours are likely to be a tiny bit different from the v.2.0 versions)

So, migrating the codebase from one version of stedgeai to the next one is usually not as straightforward as dropping-in the network.c file compiled with another version of the tool.

I don't know if you did that or not, but if not, these are one of the minimal mandatory steps to do when migrating:

• Replace ll_aton files (in <stedgeai>/Middlewares/ST/AI/Npu/ll_aton)
• Replace the library for software layers (in <stedgeai>/Middlewares/ST/AI/Lib/<toolchain>/ARMCortexM55)

 

Please tell us if this helps: looking at your github, it looks like the "ll_aton" files in Lib/AI_Runtime/Npu/ll_aton are aligned with 2.0 release, so it is likely the project won't work with .c files compiled by v2.1 compiler.

Thanks.

[Creator]

Hi @SlothGrill ,

I replaced the Inc,lib and npu folder in the AIruntime project.

Soo the project used: NetworkRuntime1010_CM55_GCC.a

This also resulted in the same error:

Bus interface interrupt
BUSIF0 ERR: 0x10
BUSIF1 ERR: 0x0
ATON_STD_IRQHandler()@842: irqs=0xlx
assertion "0" failed: file "Y:/VM/stm-workspace/fall-guard-v2.0.0/Lib/AI_Runtime/Npu/ll_aton/ll_aton_runtime.c", line 846, function: __LL_ATON_RT_IrqErr

 

My program that uses version 2.1 i've made available here if you want to check: https://www.swisstransfer.com/d/3ff38e4b-0222-4883-ba0b-eb62d3023505

 

Thanks,

Gino

ps. the main project page: FallGuard: Using Edge AI to Monitor Safety with Full Privacy | Elektor Magazine

[SlothGrill]

Hello

Thanks for the input, and the tests you did.

Thanks also for the full project.

• It looks like the "network.c" in this project has been generated with version 2.0 (different from your github version) but you have updated the libs/runtime files: so this is also unlikely to work for the same reasons as above (this time, the "network.c" is too old vs. the libs & runtime that are too new) -> don't mix versions or expect issues.
• This project provided is missing some files, could you please update and give a project that compiles so we can reproduce the issue ? (you can trash the STM32CubeIDE/Debug and Model\* (except the network.c), binary * (except the .bin) to make the archive smaller)
• Is it possible to reduce your example to something that does not need a camera to be connected to the board ? but just runs an inference on the NPU (input = what's in the memory at the time of inference), so we can debug the AI part alone ? thanks.

Thanks.

[GRATT.2]

Hi @Creator, 

I successfully ran st_ssd_mobilenet_v1_025_192_int8.tflite on the people detection project. Please find attached the patch allowing for such support. 

Steps: 

1. In the Model folder, launch the generate-n6-model.sh script. 
2. Flash the weights (network_data.hex) using STM32CubeProgrammer. 
3. Build the app by running make in the root folder. 
4. Put the board in dev mode and load build/Project.elf using ST-LINK_gdbserver. 

Note: make sure to use the last stedgeai release (v2.1). 

Guillaume

[Creator]

Hi @GRATT.2 ,

With this latest version of people dettection project I was also able to run my model and am able to detect a fall. 

I've uploaded a working version here: https://www.swisstransfer.com/d/5a548741-4d5a-43eb-9cf1-b3fc665dd493

I am still not sure what went wrong in my project because I kind of did everyting that was stated in the patch file. This version of people detect had extra buffer definitions

#define NN_OUT_NB 3

#define NN_OUT0_SIZE (6825 * 3 * 4)

#define NN_OUT1_SIZE (6825 * 4 * 4)

#define NN_OUT2_SIZE (6825 * 4 * 4)

==> Can this be related?

 

Thanks for the solution, from this point I can continue my fall detection project.

Grts,

Gino

[Creator]

Hi, @GRATT.2 , @Julian E. , @SlothGrill ,

Since the people detection project V2.0.0 is based on freertos and the nxduo library is not compatible with freertos, I've looked further into the bug.

The root cause seemed to be in the nn_thread_fct function and how the output buffer is managed.

Now I have a working version using threadx of my project. It's updated on github: ginodecock/fall-guard-od: Detect a fall by using object detection using the stm32n6570-dk board

Can you guys check within ST that the people detection project V1.0.0 also can get this fix?

 

Thanks,

Gino

 

 

 

 

[Julian E.]

Hello @Creator,

 

A patch was published in the last version of the people detection project V2.0.0 that should fix.

 

Have a good day,

Julian