camera doesn't detect- Deploying custom object detection model on stm32n6 nucleo board
Hi, I am trying to deploy a custom object detection model on the STM32N6-Nucleo board.
I started with this:
https://github.com/STMicroelectronics/stm32ai-modelzoo-services/tree/main/object_detection
The training gets done, I get the performance metrics,
python stm32ai_main.py
The deployment code works fine, and I get the deployment complete message. But when I change from development mode to flash boot mode. It doesn't detect the camera.
When I program the demo camera.hex file
https://github.com/STMicroelectronics/x-cube-n6-camera-capture/tree/main/Binary/NUCLEO-N657X0-Q
or object detection .hex file,
https://github.com/STMicroelectronics/STM32N6-GettingStarted-ObjectDetection/tree/main/Binary/NUCLEO-N657X0-Q/USB-UVC-Display
the camera works perfectly.
Below are the training and deployment configuration files. These are the only files I changed. I would really appreciate any help in this regard. Thank you,
# training configure.yaml file:
general:
project_name: custom_food_yoloxn_256
logs_dir: logs
saved_models_dir: saved_models
display_figures: false
gpu_memory_limit: 16
num_threads_tflite: 12
global_seed: 127
operation_mode: chain_tqe
model:
model_type: st_yoloxn
model_name: st_yoloxn_d033_w025
input_shape: (256,256,3)
dataset:
format: darknet_yolo
dataset_name: darknet_yolo
class_names: [beverage, food]
exclude_unlabeled: true
download_data: false
max_detections: 50
train_images_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/images/train
train_annotations_path: path/r/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/labels/train
val_images_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/images/val
val_annotations_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/labels/val
test_images_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/images/test
test_annotations_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/labels/test
test_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/images/tfs_labels/test
quantization_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/images/test
preprocessing:
rescaling:
scale: 1/255
offset: 0
resizing:
aspect_ratio: fit
interpolation: nearest
color_mode: rgb
data_augmentation:
random_contrast:
factor: 0.4
random_brightness:
factor: 0.3
random_flip:
mode: horizontal
random_translation:
width_factor: 0.15
height_factor: 0.15
fill_mode: reflect
interpolation: nearest
random_rotation:
factor: 0.02
fill_mode: reflect
interpolation: nearest
random_crop:
crop_center_x: (0.25, 0.75)
crop_center_y: (0.25, 0.75)
crop_width: (0.6, 0.9)
crop_height: (0.6, 0.9)
change_rate: 0.9
training:
dropout: null
batch_size: 64
epochs: 100
optimizer:
Adam:
learning_rate: 0.0025
callbacks:
LRWarmupCosineDecay:
initial_lr: 1.0e-05
warmup_steps: 20
max_lr: 0.00125
hold_steps: 20
decay_steps: 300
end_lr: 1.0e-06
EarlyStopping:
monitor: val_loss
patience: 60
restore_best_weights: true
verbose: 1
quantization:
quantizer: TFLite_converter
quantization_type: PTQ
quantization_input_type: uint8
quantization_output_type: int8
export_dir: quantized_models
postprocessing:
confidence_thresh: 0.300
NMS_thresh: 0.5
IoU_eval_thresh: 0.5
plot_metrics: true
max_detection_boxes: 100
mlflow:
uri: ./tf/src/experiments_outputs/mlruns
hydra:
run:
dir: ./tf/src/experiments_outputs/custom_food_yoloxn_256
# deploying custom model configure file
general:
project_name: custom_food_yoloxn_256
logs_dir: logs
saved_models_dir: saved_models
display_figures: false
gpu_memory_limit: 16 #24
num_threads_tflite: 12
global_seed: 127
operation_mode: deployment
model:
model_type: st_yoloxn
model_path: C:/stm32zoo/stm32ai-modelzoo-services/object_detection/tf/src/experiments_outputs/custom_food_yoloxn_256/quantized_models/quantized_model.tflite
dataset:
format: darknet_yolo
dataset_name: darknet_yolo
class_names: [beverage, food]
exclude_unlabeled: true
download_data: false
max_detections: 50
train_images_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/images/train
train_annotations_path: path/r/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/labels/train
val_images_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/images/val
val_annotations_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/labels/val
test_images_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/images/test
test_annotations_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/labels/test
test_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/images/tfs_labels/test
quantization_path: path/yolov5/yolo_dataset_prep/yolo_ready_dataset_416/images/test
preprocessing:
rescaling:
scale: 1/255
offset: 0
resizing:
aspect_ratio: fit
interpolation: nearest
color_mode: rgb
data_augmentation:
random_contrast:
factor: 0.4
random_brightness:
factor: 0.3
random_flip:
mode: horizontal
random_translation:
width_factor: 0.15
height_factor: 0.15
fill_mode: reflect
interpolation: nearest
random_rotation:
factor: 0.02
fill_mode: reflect
interpolation: nearest
random_crop:
crop_center_x: (0.25, 0.75)
crop_center_y: (0.25, 0.75)
crop_width: (0.6, 0.9)
crop_height: (0.6, 0.9)
change_rate: 0.9
training:
dropout: null
batch_size: 64
epochs: 100
optimizer:
Adam:
learning_rate: 0.0025
callbacks:
LRWarmupCosineDecay:
initial_lr: 1.0e-05
warmup_steps: 20
max_lr: 0.00125
hold_steps: 20
decay_steps: 300
end_lr: 1.0e-06
EarlyStopping:
monitor: val_loss
patience: 60
restore_best_weights: true
verbose: 1
quantization:
quantizer: TFLite_converter
quantization_type: PTQ
quantization_input_type: uint8
quantization_output_type: int8
export_dir: quantized_models
postprocessing:
confidence_thresh: 0.300
NMS_thresh: 0.5
IoU_eval_thresh: 0.5
plot_metrics: true
max_detection_boxes: 10
tools:
stedgeai:
optimization: balanced
on_cloud: False
path_to_stedgeai: C:/ST/STEdgeAI/4.0/Utilities/windows/stedgeai.exe
path_to_cubeIDE: C:/ST/STM32CubeIDE_2.0.0/STM32CubeIDE/stm32cubeide.exe
deployment:
c_project_path: ../application_code/object_detection/STM32N6/
IDE: GCC
verbosity: 1
hardware_setup:
serie: STM32N6
board: NUCLEO-N657X0-Q
output: "UVCL"
mlflow:
uri: ./tf/src/experiments_outputs/mlruns
hydra:
run:
dir: ./tf/src/experiments_outputs/custom_food_yoloxn_256