Nano edgeAI software

[pradeepaan]

Hi everyone,

I am trying to train a large dataset in NanoEdge AI Studio, where each class is around 500 MB in size.
Totally 5 class.

However, I am facing limitations related to the data structure, memory usage, and training workflow in NanoEdge AI Studio. I would like guidance on:

• Recommended data structuring or segmentation methods for very large datasets

• Best practices to handle or reduce large data sizes (windowing, feature extraction, downsampling, etc.)

• Any tool limitations or configuration settings to be aware of

• Possible alternative approaches if direct training with such large data is not supported

If anyone has experience training large datasets with NanoEdge AI Studio, your guidance or references would be greatly appreciated.

Thank you in advance for your support.

Best regards,
Vijepradeepan

[Julian E.]

Hi @pradeepaan,

 

ould you clarify what you mean by “limitations”?
Are you seeing any specific errors, or does it just feel slow because of the large amount of data?

I’ve used significantly larger datasets with NanoEdge without functional issues (aside from longer benchmark times).

 

I don’t know your exact use case, but some general guidelines are documented here:

AI:Datalogging guidelines for a successful NanoEdge AI project - stm32mcu

 

Also, you don’t need to wait until every benchmark completes. In the first few minutes (typically between 10 minutes and 1 hour), you can already get a good sense of the final performance. For example, if accuracy is around 50% after that time, it’s unlikely to become good later. In that case, stop the benchmark, refine your data, and start a new one.

Given your 500 MB of data per class, you could create several buffers, shuffle them, and begin with a subset, say 50 MB. Run a benchmark, check the results, then gradually increase the data volume, monitor the evolution, and iterate.

This is inherently an iterative process.

 

Have a good day,

Julian