Example for Azure RTOS USBX device video (UVC)

[Johannes]

Hello everyone

Does anyone have example code for Azure RTOS USBX device class video (UVC)?

I was able to add Video to my USBX demo application on Nucleo H7A3

There was sample code for CDC_ACM and MSC (mass storage), this worked fine. But for device class video, I was not able to find any example code.

CubeIDE 1.12.1 generated two files

ux_device_video.c

ux_device_video.h

But both files only contain prototypes of callback functions which need to be filled

Like this:

/**
 * @brief USBD_VIDEO_StreamRequest
 * This function is invoked to manage the UVC class requests.
 * @param video_stream: Pointer to video class stream instance.
 * @param transfer: Pointer to the transfer request.
 * @retval status
 */
UINT USBD_VIDEO_StreamRequest(UX_DEVICE_CLASS_VIDEO_STREAM *video_stream,
 UX_SLAVE_TRANSFER *transfer)
{
 UINT status = UX_SUCCESS;
 
 /* USER CODE BEGIN USBD_VIDEO_StreamRequest */
 UX_PARAMETER_NOT_USED(video_stream);
 UX_PARAMETER_NOT_USED(transfer);
 /* USER CODE END USBD_VIDEO_StreamRequest */
 
 return status;
}

How to fill it, that it works?

Thanks

Johannes

[mohamed.ayed]

Hi @Johannes​, as i see in your project app_usbx_device.c Tx Fifo of endpoint 4 ( EP video IN) is missed, you can see in referance project

https://github.com/STMicroelectronics/x-cube-azrtos-h7/blob/f25d657a4de4aac6dba48df28b96238479d73ca0/Projects/NUCLEO-H723ZG/Applications/USBX/Ux_Device_Video/USBX/App/app_usbx_device.c#L245

[Johannes]

@mohamed.ayed​ : I must admit, I don't understand:

If I have 4 endpoints (IN) I need one TX fifo for each endpoint?

How do these fifo indexes correspond to the endpoints?

 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 0, 0x10);
 
 /* Set Tx FIFO 2 */
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 1, 0x10);
 
 /* Set Tx FIFO 3 */
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 2, 0x20);
 
 /* Set Tx FIFO 4 */
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 3, 0x80);

[mohamed.ayed]

in your project there is 5 endpoint IN and 3 endpoint OUT you can see it in ux_device_descriptor.h

endpoint IN

• endpoint 0 (mandatory for each USB device ) 0x80U
• #define USBD_MSC_EPIN_ADDR               0x81U
• #define USBD_CDCACM_EPINCMD_ADDR           0x82U
• #define USBD_CDCACM_EPIN_ADDR              0x83U
• #define USBD_VIDEO_EPIN_ADDR             0x84U

endpoint OUT

• endpoint 0 (mandatory for each USB device ) 0x00U
• #define USBD_MSC_EPOUT_ADDR               0x01U
• #define USBD_CDCACM_EPOUT_ADDR             0x03U

endpoint 1 and endpoint 3 is bidirectional

So for OUT endpoint size is regrouped in RxFiFo:

 HAL_PCDEx_SetRxFiFo(&hpcd_USB_OTG_HS, 0x200);

For IN endpoint size TxFiFo:

 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 0, 0x10);
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 1, 0x10);
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 2, 0x20);
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 4, 0x80);

please note that real value is multiplied x4

for example for RxFifo we have 0x200 so value is 512 *4 = 2048

In HS mode size of PCD USB fifo (sum of fifo value) sould be < 4096

[Johannes]

@mohamed.ayed​ : thank you for explaining.

Sorry, but I am still confused.

I have 5 IN-endpoints. So I need 5 TX Fifos?

does the Fifo number in HAL_PCDEx_setTxFifo have to match the endpoint number?

 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 0, 0x10); //->0x80
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 1, 0x10); //->0x81
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 2, 0x20); //->0x82
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 3, 0x20); //->0x83
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 4, 0x80); //->0x84

Thank you for your help

Johannes

[Johannes]

I tried to more TX fifos like described above. This does not solve the problem with the video device.

I still need to move the video endpoint to the top of the list of endpoint descriptors to make it work

uint8_t UserClassInstance[USBD_MAX_CLASS_INTERFACES] = {
 CLASS_TYPE_VIDEO,
 CLASS_TYPE_MSC,
 CLASS_TYPE_CDC_ACM,
};

[mohamed.ayed]

@Johannes​ Yes, if you see HAL_PCDEx_SetTxFiFo API description fifo parameter (The number of Tx fifo), this is the endpoint Number .

https://github.com/STMicroelectronics/x-cube-azrtos-h7/blob/f25d657a4de4aac6dba48df28b96238479d73ca0/Drivers/STM32H7xx_HAL_Driver/Src/stm32h7xx_hal_pcd_ex.c#L70

Regards,

Mohamed

[Johannes]

I was able to implement the video demo

https://github.com/STMicroelectronics/x-cube-azrtos-h7/tree/main/Projects/NUCLEO-H723ZG/Applications/USBX/Ux_Device_Video

into my code using Nucleo H7A3

Does anyone know, how the demo data for the video is generated or encoded?

see file "stream1.h"

Are these "just" JPEG images put into a sequence of 28 images?

 
const uint8_t *tImagesList[] = {image0, image1, image2, image3, image4, image5, image6, image7, image8, image9, \
 image10, image11, image12, image13, image14, image15, image16, image17, \
 image18, image19, image20, image21, image22, image23, image24, image25, \
 image26, image27, image28};
 
uint16_t tImagesSizes[] = {JPEG0_SIZE, JPEG1_SIZE, JPEG2_SIZE, JPEG3_SIZE, JPEG4_SIZE, JPEG5_SIZE, JPEG6_SIZE, \
 JPEG7_SIZE, JPEG8_SIZE, JPEG9_SIZE, JPEG10_SIZE, JPEG11_SIZE, JPEG12_SIZE, \
 JPEG13_SIZE, JPEG14_SIZE, JPEG15_SIZE, JPEG16_SIZE, JPEG17_SIZE, JPEG18_SIZE, \
 JPEG19_SIZE, JPEG20_SIZE, JPEG21_SIZE, JPEG22_SIZE, JPEG23_SIZE, JPEG24_SIZE, \
 JPEG25_SIZE, JPEG26_SIZE, JPEG27_SIZE, JPEG28_SIZE};

[mohamed.ayed]

Hi @Johannes​, To have encoded video.

1- split your video into images with ffmpeg libraries (see https://ffmpeg.org)

2- select your images and convert them to hex array you can use free tool for that.

BR

Mohamed

[Johannes]

Thank you mohamed. That was helpful. I was able to build my own video.

Is there an example project to use an uncompressed video stream?

CubeIDE allows for a setting "uncompressed"

 

is there an example for that?

[mohamed.ayed]

Hi Johannes,

It's mentioned in readme file

https://github.com/STMicroelectronics/x-cube-azrtos-h7/blob/main/Projects/NUCLEO-H723ZG/Applications/USBX/Ux_Device_Video/README.md

if you you open ioc of application (download https://www.st.com/en/embedded-software/x-cube-azrtos-h7.html) change VIDEO_STREAMING_FORMAT to MJPEG and change video width and height finally generate code and include stream2.h in project instead of stream1.h

- stream2 FORMAT : UNCOMPRESSED and Width = 176, Height = 144 and the descriptor shall be updated.

[Johannes]

Hello muhamad

Sorry, but I am still confused.

Stream2.h is an uncompressed stream. You say; i still should set the stream format to MJPEG?

I used MJPEG for "stream1.h" (compressed stream) This works.

For stream2.h don't I need to switch to "uncompressed"?

What about all the other parameters for the stream of "stream2.h"?

Bits per Pixel?

Format YUV2 or NV12?

Color Primaire?

TRF Characteristics?

Matrix Coefficients?

 

Thanky you very much for your help

Johannes