How does the aiValidation in X-CUBE-AI run?

[MChan.10]

We are running X-CUBE-AI on an stm32F746 eval board (and a number of other stm32 eval boards). We are using version 7.1.0, 7.2.0, and 7.3.0, and using the generated source code for CubeIDE.

We then pulse a GPIO whenever ai_mnetwork_run() runs, which would call the network's inference run function, and we use this GPIO pulse to know when to measure, for example, current draw on the board to measure power usage during inference.

Anyway, my question is that in the source code in aiValidation.c, we see a definition for

#if !defined(USE_OBSERVER)
#define USE_OBSERVER 0 /* 0: remove the registration of the user CB to evaluate the inference time by layer */
#endif
 
#if defined(USE_OBSERVER) && USE_OBSERVER == 1
#ifndef HAS_INSPECTOR
#define HAS_INSPECTOR
#endif
 
#ifdef HAS_INSPECTOR
#define HAS_OBSERVER
#endif
#endif

where we can remove or not the INSPECTOR / OBSERVER which seems to observe the inference run for statistics.

Does this observer add any extra time during the inference run itself? Or does it let the inference run cleanly and then later gather the information about the inference run?

[jean-michel.d]

Hi MChan,

Yes when the OBSERVER is enabled, extra cycles are used to manage a registered callback allowing to generate the statistics per layer. Statistics are not stored on the board, there are uploaded to host after each layer. Number of cycles can be important due to the usage of the serial link and if the dump of the data is also enabled. These extra cycles are removed at the end of the inference to compute the results. In your case, if you want to evaluate the inference time with a GPIO, USE_OBSERVER should be disabled.

br,

Jean-Michel

[MChan.10]

Thank you Jean-Michel.

Is there any way we can have the best of both worlds? If I modified the aiPbCmdNNRun() function in aiValidation.c, where the first run is as written (with observer), and then after I do a second run of the network inference where I DO NOT bind the observer, can we get a clean run where the callbacks do hot happen?