LL_ATON_RT_Reset_Network() call between inferences

[ERROR]

Hi,

On my test setup with the NN (MiniResNet model from STM32 Model Zoo),

I observed the following behavior:

Without calling LL_ATON_RT_Reset_Network() after each inference, only the first inference runs a few ms. All subsequent calls return almost instantly (execution time ≈ 0), and no inference is actually performed.

With LL_ATON_RT_Reset_Network() invoked either before or after each inference (as in the example below), every inference is executed normally.

Questions:

1. Is calling LL_ATON_RT_Reset_Network() always required to ensure correct inference execution? Are there any cases where the  LL_ATON_RT_Reset_Network() call is not needed?

2. In the case of stateful neural networks (e.g. RNN), is it safe to call LL_ATON_RT_Reset_Network()? Will the network’s state remain intact, or is it cleared/reset as well?

 while(1){
 /* ------------- */
 /* - Inference - */
 /* ------------- */
 /* Pre-process and fill the input buffer */
 //_pre_process(buffer_in);
 /* Perform the inference */
 LL_ATON_RT_Reset_Network(&NN_Instance_Default);
 printf("Starting inference");
 do {
 /* Execute first/next step */
 ll_aton_rt_ret = LL_ATON_RT_RunEpochBlock(&NN_Instance_Default);
 /* Wait for next event */
 if (ll_aton_rt_ret == LL_ATON_RT_WFE)
 LL_ATON_OSAL_WFE();
 } while (ll_aton_rt_ret != LL_ATON_RT_DONE);
 /* Post-process the output buffer */
 /* Invalidate the associated CPU cache region if requested */
 //_post_process(buffer_out);
 /* -------------------- */
 /* - End of Inference - */
 /* -------------------- */
 }

 

[Julian E.]

Hello @ERROR,

 

After doing an inference you need to call the function LL_ATON_RT_Reset_Network(&NN_Instance_Default).

The model is a list of epochs (or steps), if you don't reset it, you are stuck doing the last epoch, which mean that it will not do anything.

There is no case where you don't need to call it.

 

For RNNs like models, you need to add an output to your model to get the network state and pass it back as an input.

As you pointed, the model will be reset, so you need to handle it manually.

 

Have a good day,

Julian

[ERROR]

Hi @Julian E. 

Thank you indeed for your precise answer!

Good luck!