USB stops receiving and/or transmitting data after short time using Async endpoint

[HHans.1]

I have a problem using ISOC ASYNC USB endpoint.

I am using Audio Class 2.0

3 endpoints, (To Host, Feedback to host, from host)

If I only enable receiving data from host then I will receive the data.

If I only enable transmitting data to host then I will transmit the data.

If I enable both transmit and receiving then transmit will always transmit data, but receive will only receive x amount and then stop. It does not trigger ISOINComplete or any other known fault.

Has anyone had similar problems?