stedgeai and "s16 x s8" TFlite models

[ERROR]

Hi,

I have got a TFlite model with s16 activations and s8 weights.

1. Without NPU. I run this command

stedgeai.exe generate --target stm32n6 --name network -m %1 --compression none --verbosity 1 -O time --output ./X-CUBE-AI

and the utility produces a network with "float" activations and weights (and adding int -> float at the start, and float -> int at the end).

Can I use some command line options to force the utility to generate "integer" layers?

2. With NPU. I run this command

stedgeai.exe generate --target stm32n6 --st-neural-art n6-noextmem@C:/Users/asdfg/STM32Cube/Repository/Packs/STMicroelectronics/X-CUBE-AI/10.2.0/scripts/N6_scripts/user_neuralart.json --name network -m %1 --compression none --verbosity 1 -O time --output ./X-CUBE-AI_NPU/App

and the utility outputs the error "NOT IMPLEMENTED: Unkonwn layer format for layer Input_0"

With NPU, is it possible to use int16x8 TFlite models?

Regards!

 

[Julian E.]

Hello @ERROR,

 

We do not support 16x8.

If you use the NPU or not, the elements in the core used are not the same.

 

In your tests, it seems that without using the NPU, the core supports a float32 fallback but with it, it does not.

I would say that it is not surprising as the core for the NPU only support int8 and some layer in SW fallback (so it is more restrictive).

 

I was told that 16x16 and 8x16 could be supported by the hardware, but it is not currently supported by the tool chain.

It is in the roadmap, but I would not expect to see it soon.

 

Have a good day,

Julian

[ERROR]

Hi @Julian E. 

Thank you for the swift and full explanation.

Regards!