Explanation of 'validation on target' output from X-CUBE-AI command line interface with TFLM runtime

[jana_urmi]

Board Used: STM32 Nucleo-L476RG

Workflow: I configured the board, added X-CUBE-AI library in Validation mode, added network in .tflite format with TFLM runtime and generated the code using STMCubeMX. Built the project and flashed the firmware with STMCubeIDE. With the X-CUBE-AI CLI, I validated the tflite model on target.

Issue: I cannot understand the profiling results obtained here. Sample output is shown in the picture
1. For the duration value, what is meant by 'by sample'?

2. And what do the values in brackets represent for the duration? 

3. What is meant by HOST duration and how is it different from normal duration? 

4. In the TFLM support page on X-CUBE-AI documentation for the 8.1.0 version (screenshot attached), the output snippet provided for AI Validation application does not match the output I get from the CLI in terms of how the duration is displayed, which makes it even harder to understand the output I'm getting. Is the output snippet from a different version of the library?

 

[jean-michel.d]

Hello,

1 - By default to evaluate the inference time, 10 inferences are executed with different random inputs and only the average duration is reported. This is why "by sample" is indicated.  

2. In brackets, for the N inferences/samples, the min, max and std deviation values are reported.

3. This is not fully related to the normal duration; this indicates only the global time (host point of view) to perform an inference. By sample, it includes the time to send the data to the target (It is dependent of the size of tensor and the speed of the connection with the board), to execute the inference itself, the dump of the intermediate results (if available) and the time to dump the outputs.

4. We understand. We will try to be more vigilant in the next release to align the documentation.

br,

JM

[jana_urmi]

Hello,

Thank you so much for clearly explaining!!