USBX Audio Class frame done callback not called

[nico23]

I've set up my STM32U5A9J-DK to use ThreadX and connect to my PC via USBX and Audio Class.

I'm correctly see the Audio Device on my PC and I can easily stream audio to it. The only thing is that, for some reason, the callback for new frame done is not fired

	audio_stream_parameter[0].ux_device_class_audio_stream_parameter_callbacks.ux_device_class_audio_stream_change
	= USBD_AUDIO_PlaybackStreamChange;

	audio_stream_parameter[0].ux_device_class_audio_stream_parameter_callbacks.ux_device_class_audio_stream_frame_done
	= USBD_AUDIO_PlaybackStreamFrameDone;

The strage thing is that USBD_AUDIO_PlaybackStreamChange is correctly called

VOID USBD_AUDIO_PlaybackStreamChange(UX_DEVICE_CLASS_AUDIO_STREAM *audio_play_stream,
 ULONG alternate_setting)
{
 /* USER CODE BEGIN USBD_AUDIO_PlaybackStreamChange */

 /* Do nothing if alternate setting is 0 (stream closed). */
 if (alternate_setting == 0)
 {
 return;
 }

 BufferCtl.state = PLAY_BUFFER_OFFSET_UNKNOWN;

 /* Start reception (stream opened). */
 ux_device_class_audio_reception_start(audio_play_stream);

 /* USER CODE END USBD_AUDIO_PlaybackStreamChange */

 return;
}

Instead, USBD_AUDIO_PlaybackStreamFrameDone is never called

VOID USBD_AUDIO_PlaybackStreamFrameDone(UX_DEVICE_CLASS_AUDIO_STREAM *audio_play_stream,
 ULONG length)
{
 /* USER CODE BEGIN USBD_AUDIO_PlaybackStreamFrameDone */

 UCHAR *frame_buffer;
 ULONG frame_length;

 /* Get access to first audio input frame. */
 ux_device_class_audio_read_frame_get(audio_play_stream, &frame_buffer, &frame_length);

...
}

[T_Hamdi]

Hello @nico23 

To help you fix the issue where the ux_device_class_audio_stream_frame_done callback is not triggered, I recommend referring to the STM32WBA USBX Audio example available here:

STM32WBA BLE USBX Audio Example .

[FBL]

Hi @nico23 

Would you attach minimum firmware to reproduce the behavior? 

[nico23]

Hi @FBL 

Here you are https://github.com/NicoCaldo/NovaSonus_ThreadX

I'm seeing that my memory allocation is one-third the one in the example

#define USBX_DEVICE_MEMORY_STACK_SIZE 20 * 1024

vs

#define USBX_APP_MEM_POOL_SIZE 60 * 1024
#define USBX_MEMORY_STACK_SIZE 53 * 1024

Could it be the issue?

 

[FBL]

Hi @nico23 

I suggest isolate the USB audio functionality by disabling other middleware like TouchGFX to rule out conflicts and narrow down the issue. Would you please provide a minimal project based on this STM32U5A9J-DK board, without any other peripherals configured? This will help us assist you more efficiently.

[nico23]

So I have investigated a bit, and when I start to stream to the board, I'm getting a correct

Audio Stream Change: alt=0
Audio Stream Change: endpoint_dir=OUT
Audio Stream Change: alt=1
ux_device_class_audio_reception_start with status 0

 (I'm logging what's going on) 

Audio starts streaming, but the callback is not called. If I pause the execution of the program, I'm seeing the ux_class_audio_stream_thread is always SUSPENDED in waiting for a semaphore (?)

 

(I've disabled ThouchGFX thread)

[nico23]

Checking into the actual source code, I've found the _ux_utility_error_callback_register. I've implemented it, and it turned out that on startup, when the USB is connected to my PC, I'm seeing the error

USB Error: INTERRUPT level, HCD context, TIMEOUT (0x2/0x8/0x4)

Meanwhile, when I'm starting the audio stream from the PC, I'm getting

USB Error: INTERRUPT level, CLASS context, TRANSFER (0x2/0x7/0x23)

Now, I'm seeing the second error is a transfer error, probably due to the fact that at the beginning, something went into a timeout. My question is, why am I seeing an HCD context error if I'm using the USBX in Device Only mode?

After tracing the issue, I've understood that the error is called inside

_ux_utility_mutex_on
_ux_system_mutex_on(&_ux_system -> ux_system_mutex);
memory = _ux_utility_memory_allocate(UX_NO_ALIGN, UX_REGULAR_MEMORY, sizeof(UX_SLAVE_CLASS) * UX_MAX_SLAVE_CLASS_DRIVER);

The tx_mutex_get returns 0x4 which is triggered by 

if (TX_THREAD_GET_SYSTEM_STATE() != ((ULONG) 0))
{

	/* A non-thread is trying to suspend, return appropriate error code. */
	status = TX_WAIT_ERROR;
}

From what I'm understanding, the error occurs because ux_device_stack_initialize is executed before ThreadX is running. Am I right, or am I missing something?

[nico23]

Despite the TIMEOUT at the initialization (I don't think it's the main issue of my frame_done not being called), I'm seeing that when I try to start an audio flux from my PC to the USB Audio driver, I'm seeing the error handler being called with the following stack:

status = _ux_device_stack_transfer_request(transfer, max_packet_size, max_packet_size); 

triggers the error as the status is 38 (what does it mean?). Inside the function, I've found

status = dcd -> ux_slave_dcd_function(dcd, UX_DCD_TRANSFER_REQUEST, transfer_request);

which is the one that returns 38.

In ux_api.h, the only value I was able to find that matches the return value was

#define UX_TRANSFER_BUS_RESET 0x26

 From here, I'm stuck as the ux_slave_dcd_function is defined as 

typedef struct UX_SLAVE_DCD_STRUCT
{

 UINT ux_slave_dcd_status;
 UINT ux_slave_dcd_controller_type;
 UINT ux_slave_dcd_otg_capabilities;
 UINT ux_slave_dcd_irq;
 ULONG ux_slave_dcd_io;
 ULONG ux_slave_dcd_device_address;
 UINT (*ux_slave_dcd_function) (struct UX_SLAVE_DCD_STRUCT *,UINT, VOID *);
 void *ux_slave_dcd_controller_hardware;

} UX_SLAVE_DCD;

[FBL]

Hi @nico23 

It seems the issue is related to mutex acquisition failing because USBX initialization is called before ThreadX kernel is fully running. This is a known limitation using USBX.

[nico23]

So why does your example code do just that?

en.x-cube-azrtos-h7-v3-3-0.zip\Projects\STM32H743I-EVAL\Applications\USBX\Ux_Device_Audio2.0_PlayBack\

https://www.st.com/en/embedded-software/x-cube-azrtos-h7.html

Or am I reading it wrong? As I've followed your example above

[FBL]

Hi @nico23 again, 

I agree example firmware should be updated (internal ticket 204777). As of now, the recommendation is to move the  MX_USBX_Host_Init() to USBX_APP_Host_Init().

[nico23]

I can confirm that invoking USBX_APP_Device_Init within a thread fixes the first issue.

Now I'm still experiencing the return code 38 from the dcd function https://community.st.com/t5/stm32-mcus-embedded-software/usbx-audio-class-frame-done-callback-not-called/m-p/856896/highlight/true#M70159

[nico23]

So, I think ultimately there's a bug in the USBX api when it comes to setting frequency.

If I'm choosing to support a single frequency, i.e. 48k, why does the device in the sampling frequency control SET request break and return if multiple frequencies are not supported?

It does the same while checking if it's a fixed single frequency. 

 /* Sampling frequency control, SET request. */
 if ((request_type & UX_REQUEST_DIRECTION) == UX_REQUEST_OUT &&
 (control_selector == UX_DEVICE_CLASS_AUDIO20_CS_SAM_FREQ_CONTROL))
 {
 switch(request)
 {
 case UX_DEVICE_CLASS_AUDIO20_CUR:

 /* Check request parameter. */
 if (request_length != 4)
 break;

 /* Check if multiple frequency supported. */
 if (control -> ux_device_class_audio20_control_sampling_frequency != 0)
 break;

...

 /* Check if it's fixed single frequency. */
 if (n_sub <= 1 && res == 0)
 break;

Isn't it more correct to first check if the frequency the host is trying to set matches the only frequency the device can set? If not, it breaks and returns, stalling the device.

What's your take @FBL @T_Hamdi 

[FBL]

Hi @nico23 

An internal ticket 222275 is submitted to dedicated team regarding the HCD context issue.

Regarding the stall, would you simplify your project to narrow down the issue to simply streaming via SAI like the example N6 here.

[FBL]

Hi @nico23 

I think the order of initialization regarding setting the audio frequency and control is different. 

If this function is called too early, the USBX audio class control handler may see a zero or invalid frequency, causing it to stall on frequency SET requests.

The example firmware sets explicit audio playback parameters (sample rate, bits per sample, volume) after USBX init, which may also be required to keep the device and USBX stack synchronized.