STM32N6xy: NPU for audio DSP?

[LCE]

Heyho,

I'm not a SW / DSP guy, but I have to help selecting a new MCU.

Mostly we need gigabit ethernet, SAIs / I2S for audio input.

But for some future applications we might need some DSP power for online audio processing, some heavy filtering and FFT stuff.

I wonder if the N6 's NPU can help with that. In all the documentation / ads I mostly see graphics applications.

[Tuomas95]

Hello,

The N6 has Helium vector acceleration which can be used for the DSP processing.

[LCE]

This does not seem to help with typical audio processing, using FIR filters, FFT, ... - or so it seems to me.

Made rather for 2D graphics stuff.

[Tuomas95]

It is exactly made for audio processing (among other things), see the documentation here, FFT and FIR are supported.

[DmitryR]

Hi @LCE,

 

Just like AI, DSP functions are linear combination of input data. Or I should better say that AI is like DSP but much larger. FIR is a single artifical neuron without activation function. FFT is one layer of perceptron. So an AI accelerator may easily be used for DSP. If you search Keras of Tensorflow you will quickly find there DSP functions that probably may be directly imported to STM32N6 project as CubeMX supports both Keras and TFlite.

 

Helium may also be used, but it is relatively week in compare to AI. AI accelerator in STM32N6 contains 4 convolutional accelerators with 18 16*16 MACs each which gives incredible for DSP 72 MACs per clock. That means for example that it could calculate a 63-tap FIR each clock. I am sure that it will be enough not only for audio but for at least some of SDR applications. 

 

But before you choose it I would recomment to create a CubeMX project, load models you need there and at least validate them. Even better will be to buy a kit and test it in hardware. 

 

With best regards,

Dmitry

[LCE]

Thanks to all so far!

In CubeMX I found out that 1 pin is shared between SAI (data in) and ETH (clock) - and I need both.

So for now that rules out the N6...

Maybe I'll come back later to the N6 and work around that using I2S and some terrible PCB routing...

[mƎALLEm]

Hello @LCE ,

Please accept as solution if your question has been answered.

For SAI/ETH shared pin, you've already posted that question here.

Thank you