FP-AI-PDMWBSOC incorrect inferencing results

[t_mo]

I have a STEVAL-PROTEUS device that I collected data with and NanoEdgeAI returned a model that has very high accuracy (almost 100%). In fact, during the validation step on NEAI, the emulator returns very accurate results as well. However, in the deployment step, after compilation (I left everything as default but these are the settings), and replacing the the NCC stub library provided in FP-AI-PDMWBSOC, the results are very poor and totally unlike the results from the emulator.

Has anyone seen this or has any idea what might be wrong?

[Julian E.]

Hello @t_mo ,

 

In my opinion, here are possible explanation for the issue that you are facing:

• In the FP-AI-PDMWBSOC, make sure that you are using the same sensor, data rate, buffer size and range that you were using when logging data.
• Make sure to also change the knowledge.h
• Make sure that the neai_classification_init returns NEAI_OK
• You can try to use buffers you used during the benchmark as input for your microcontroller and see if it performs as expected or not. It needs some work, but if you know how to do it, it can be a solution.

 

If it doesn't help, you can use the datalogger source code instead of the function pack code if you want. You will find a project for every board supported by the NanoEdge AI Studio data logger. Just find the Proteus and select your sensor. (make sure to clone the project with the command in the readme)

stm32ai-nanoedge-datalogger/Projects/STEVAL-PROTEUS1/STEVAL-PROTEUS1_ISM330DHCX/Core/Src/main.c at main · stm32-hotspot/stm32ai-nanoedge-datalogger · GitHub;

 

In the proteus datalogging code, change your settings (datarate, buffer size and range) and you will see that there is variable NEAI_MODE that controls if you are datalogging or using the code for anomaly detection.

To change the code from anomaly detection to n class classification, it is pretty easy. You need to:

• Include the knowledge
• delete the learning part of the code (ncc don't require a learning phase)
• replace the anomaly detection functions with the ncc functions (very similar) see: https://wiki.st.com/stm32mcu/wiki/AI:NanoEdge_AI_Library_for_n-class_classification_(nCC)

 

You can also take a look at this ncc tutorial, it is not the same board, but the process is the same:

https://wiki.st.com/stm32mcu/wiki/AI:How_to_create_a_multi-state_vibrations_classifier_using_NanoEdge_AI_studio

 

If you still need help, do not hesitate to reach out.

Julian

 

[t_mo]

Thanks for the reply, Julian. I would like to address your points above for the function pack since I am trying to evaluate the model without having to change too much code - 

• After flashing FP-AI-PDMWBSOC binary to the device, I opened the NEAI Classification page option on my mobile app, and selected the same sensor, ODR, and accelerometer sensitivity used during data logging. Is this what you are referring to?
• At this step, I was following the steps in the PDMWBSOC quick start guide. After compiling the library, I replaced the knowledge_ncc.h and NanoEdgeAI_ncc.h files in ...\en.en.fp-ai_pdmwbsoc\STM32CubeFunctionPack_AI_PDMWBSOC_v2.0\Middlewares\ST\NanoEdge_AI_Library\Inc. as well as the libneai_ncc.a file in ...\en.en.fp-ai_pdmwbsoc\STM32CubeFunctionPack_AI_PDMWBSOC_v2.0\Middlewares\ST\NanoEdge_AI_Library\Lib.
• Could you clarify how I can check that neai_classification_init returns NEAI_OK? For more context, I attached the sensor to a fan and am simply trying to identify whether the fan is off/low/medium/high speed. Once flashed to the device, it can detect very well when the fan is off, but it is classifying low/medium/high speed as high speed. It does not look like there are any issues running, but it seems to me that the library is outputting an incorrect value. 

I wonder if there is some step that I am missing that is not specified in the quick start guide. 

Thank you!