STM32 model zoo help

[jzzunn]

Im trying to deploy an audio detection model i trained and developed using yamnet onto the B-U585I-IOT02A board that differentiates between normal and abnormal machine sounds. I am trying to implement the AI on stm32cubeide but dont know how to do it. I saw a freertos guide or something but dont really know how to implement or use it? 

 

[jzzunn]

Duplicate - merged.

---

I get this error when trying to open the .ioc file. I opened the project of freertos from stm32 model zoo for audio. 

Invalid Input: Must be project's active .ioc file.

Project's 'Projects' active one is 'Projects.ioc' file

[Julian E.]

Hello @jzzunn 

 

it seems that you use the ST Model Zoo.

In this case, please look at this tutorial on how to deploy a model on a B-U585I-IOT02A

stm32ai-modelzoo-services/audio_event_detection/deployment/README.md at main · STMicroelectronics/stm32ai-modelzoo-services

 

If you have any issues following this tutorial, feel free to reach out to me!

 

Have a good day,

Julian

[jzzunn]

I dont want to use the deployment tutorial because i am trying to do it with my own model file that i converted using stm32 cloud ai

 

[Julian E.]

First, you can deploy your own model with it. Model Zoo will generate the same files as Stm32 Dev Cloud did.

Then, about your ioc issue, where did you get this file?

Julian

[jzzunn]

I just added the project from the freertos folder of stm32 model zoo didnt change anything, but i think it was because the ioc file was placed outside the actual project for some reason

 

[jzzunn]

This is my model, the feature extraction layers uses YAMNet's pretrained feature extraction and adding my own classification layers on top:

def extract_features(audio_file, sample_rate=16000:(

    """Extract YAMNet embeddings from an audio file"""

    # Load audio file

    audio, sr = librosa.load(audio_file, sr=sample_rate, mono=True)

   

    # Ensure audio is the right length (at least 1 second)

    if len(audio) < sample_rate:

        audio = np.pad(audio, (0, sample_rate - len(audio)))

   

    # Get YAMNet embeddings

    yamnet_model = hub.load('https://tfhub.dev/google/yamnet/1')

    scores, embeddings, _ = yamnet_model(audio)

   

    return embeddings

 

Class YAMNetTransferModel:

def __init__(self:(

        # Load the YAMNet model from TensorFlow Hub

        self.yamnet_model = hub.load('https://tfhub.dev/google/yamnet/1')

        self.input_shape = (None, 1024)  # YAMNet embeddings shape

       

        # Build the classifier model on top of YAMNet embeddings

        self.model = self._build_model()

   

    def _build_model(self:(

        """Build a classifier on top of YAMNet embeddings"""

        # Input layer for YAMNet embeddings

        inputs = keras.layers.Input(shape=self.input_shape)

       

        # Add layers for classification

        x = keras.layers.GlobalAveragePooling1D()(inputs)

        x = keras.layers.Dense(128, activation='relu')(x)

        x = keras.layers.Dropout(0.3)(x)

        x = keras.layers.Dense(64, activation='relu')(x)

        x = keras.layers.Dropout(0.2)(x)

        outputs = keras.layers.Dense(2, activation='softmax')(x)  # 2 classes: normal and anomalous

       

        # Create and compile the model

        model = keras.Model(inputs=inputs, outputs=outputs)

        model.compile(

            optimizer=keras.optimizers.Adam(learning_rate=0.001),

            loss='sparse_categorical_crossentropy',

            metrics=['accuracy']

        )

       

        return model

[jzzunn]

How do i implement this model to the custom_model.py code because i dont have feature extraction layers, im just importing yamnet's feature extraction from tensorflowhub. Besides, Im trying to develop a webserver that can deploy models to the stm32 board from the server itself so i would prefer not to use model zoo config files etc because it is not very user friendly 

[Julian E.]

Hello @jzzunn ,

 

I think that the best way for you to deploy your model is to use X Cube AI from STM32CubeMX.

Take a look at this document: UM2526Getting started with X-CUBE-AI Expansion Package for Artificial Intelligence (AI) (https://www.st.com/en/embedded-software/x-cube-ai.html#documentation)

 

With this, you should be able to generate a template application that run an inference of your model with a random input and outputs via serial some metrics.

 

In the generated code, the API related to AI is explained here:

https://stedgeai-dc.st.com/assets/embedded-docs/embedded_client_api_legacy.html 

 

Have a good day,

Julian