While trying 'Validate on Desktop' for tflite model, we are getting the error : "TOOL ERROR Didn't find op for built-in opcode "FULLY_CONNECTED" for version '9' Registration failed".

[NS K.1]

Hello,

We are running a tflite model (generated from tf.Keras) following the example here : https://www.digikey.in/en/maker/projects/tinyml-getting-started-with-stm32-x-cube-ai/f94e1c8bfc1e4b6291d0f672d780d2c0

While running "Validate on Desktop", we are getting the above mentioned TOOL ERROR. The generated report has been attached.

Thanks in advance.

[jean-michel.d]

Hello,

First, the raised error is due to the version of the TFLite interpreter which is embedded in the X-CUBE-AI pack to perform the evaluation of the generated c-model. In X-CUBE-AI 5.2, the version of the TF Lite interpreter is TF2.3. I suppose that to generate the TFLite file, you have used a more recent version +2.4. In this case, the generated TFLite file is generated with a version 9 of the FULLY_CONNECTED operator.

Other point, according the referenced link to train and to convert the model, the OPTIMIZE_FOR_SIZE (no more documented since TF2.3)

should be not used. Because it is not really supported in X-CUBE-AI and particularly for a tiny run-time, including also tensorflow lite for micro-controller. In this case, only the weights are quantized, and the C run-time requires to implement the hybrid kernels to manage the weights in int8 and the activations in float. This option is only interesting to reduce the size of the deployed TFLite file, but it requests a runtime with extra memory and computing resources to execute it (like Mobile device with Android or iOS). Only the following Python code should be used to create a TF Lite float version of the Keras model. In this case, with TF +2.4, the generated TF Lite will be generated with version 1 of the FULLY_CONNECTED operator, fully compliant with X-CUBE-AI 5.2. You can note that in this case, the Keras model (h5 file) can be directly imported in X-CUBE-AI and the generated c-model will be exactly the same.

converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()

Now to quantize the original h5 model, you need to applied the TF Lite converter options to have a full integer post-training quantization for all operators (see https://www.tensorflow.org/lite/performance/post_training_quantization )

import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8 # or tf.uint8
converter.inference_output_type = tf.int8 # or tf.uint8
tflite_quant_model = converter.convert()

Last point, for info, the generated c-model for the TF Lite model (with FC v9) is valid and can be used. If you want to valid it, with your own data, the following command line can be used to avoid to execute the original model thanks the --no-exec-model option:

$ stm32ai validate -m <tflite_model_file> -vi user_data.npz --no-exec-model [--mode stm32]

br,

Jean-Michel