StreamBuffers in AzureRTOS

[Davide Dalfra]

Hi Folks

I am doing some test to see if it worth to start using AzureRTOS on a new system we're going to develop.

I have a serial stream coming from a device (RTCM stream) that need to be read by a MCU and then sent over a UDP stream.

The UDP/Net part is (almost ok), while i'm struggling a bit to port a uart manager i wrote in the past for FreeRTOS.

The uart from the device is a bit tricky and i receive frequently fragmented packet that need to be reassembled to be correctly decoded by a parser.

With FreeRTOS i used to do that with stream-buffers or queues , where UART DMA RX isr pushes in half and cplt events the received bytes on the streambuffer (or queue).

If the amount of received bytes was not hit by the half or cplt event i have a last-byte timeout timer that when triggered just push the remaining bytes on the same streambuffer.

As example, the uart receiveve was launched with size=64 in circular mode.

Every half or cpltd events 32 bytes was sent to the streambuffer, and let's say that if 6 byte was received , those 6 were sent to the streambuffer after 10ms (last_byte_timeout, managed by checking dma rx counters).

I tried the queue approach on AzureRTOS but unfortunately the queue element size seems to be maximum 16 bytes which can't be used for my purpose.

I do see some post on MS KB that are talking about passing reference by pointer but this would means adding another abstraction layer to the system.

Does in AZ RTOS exist something similar to StreamBuffers or a way to use queues with a higher element size?

Thanks

Davide

[Davide Dalfra]

P.s.: i was having a look at the CMSIS wrapper that has been made around for a quick migration from FreeRTOS.

Queues wrapper is broken by design.

[Piranha]

If there is just a single writer and a single reader, you can do it even better without RTOS or any locking:

https://ferrous-systems.com/blog/lock-free-ring-buffer/

https://www.snellman.net/blog/archive/2016-12-13-ring-buffers/

Can be combined with sending a thread notification/event to trigger the check/read of the buffer.

Maybe consider reading the DMA buffer directly in the processing thread, which prepares the UDP packet, so that there is just one copy operation, not two or more.

> 10ms (last_byte_timeout, managed by checking dma rx counters)

Why not use IDLE or RTO interrupts?

https://github.com/MaJerle/stm32-usart-uart-dma-rx-tx

[Davide Dalfra]

lock free ringbuffer are very good, however i did not have only one consumer.

I have one producer (uart/dma) and two consumers (udp and another serial where i need to write the same packet).

About RTO and IDLE those are very good ideas. As far as i remember IDLE is not that good if you have slower bitrate transmission, while RTO is not available on my mcu (STM32F429NIHx).

I'm trying an approach combined by block-pool & queue(by pointer, no longer reference). Basically i'm trying to write what's already working on FreeRTOS.

[Piranha]

> two consumers (udp and another serial where i need to write the same packet)

A consumer is a code that takes the data out of the ring buffer. If you need the same data for two destinations you cannot have multiple consumers.

> IDLE is not that good if you have slower bitrate

Not with a slower baud-rate, but with an intermittent data. In the worst case one can get an IDLE interrupt after every byte. But, as the data rate is reduced anyway, I don't see it as a problem. And in addition for many systems reacting quicker is desirable.

[Haithem Rahmani]

Hi @Davide Dalfra,

Actually you can create a message queue in Threadx with a message size greater than 16 Bytes.
you'll need to define the flag TX_DISABLE_ERROR_CHECKING  to map the tx_queue_create() to _tx_queue_create() in the tx_api.h.
the _tx_queue_create() accepts any size of the message. The restriction is done in the _txe_queue_create() function. (check here).

Hope this helps.

regards
Haithem.

 

[Davide Dalfra]

Hi, 

Thanks for your reply! Is there any specific reason why that was enabled by default? 

 

Regards

Davide

[Haithem Rahmani]

Hi,
that's the default configuration of ThreadX, the flag TX_DISABLE_ERROR_CHECKING  adds extra check on parameters to ensure params consistency prior to function calls.

[Stefano Oliveri]

Hi,

the size limit for a queue message in ThreadX is not 16 bytes. From the API documentation:

• message_size: Specifies the size of each message in the queue. Message sizes range from 1 32-bit word to 16 32-bit words. Valid message size options are numerical values from 1 through 16, inclusive.

That means the max size is 16 * (32 bit) = 64 byte

Best regards,

Stefano

[Davide Dalfra]

Hi Stefano,

Yes you're right. 
However i have implemented an approach that uses block-pool + queue to store "large message". It's a sort of wrapper  that you can use like a normal queue (cmsis-os api based) with an arbitrary message size.

Regards

Davide

[Stefano Oliveri]

Hi Davide,

yours is an interesting ideas. In the past, working also with FreeRTOS, I had different use cases where I need to exchange messages bigger than 64 bytes. If your API are generic it will be interesting to give a look, maybe integrate it in a application framework. 

Another thing that make me curious is when you write "cmsis-os api based". It is not clear to me if you are using CMSIS-OS in your project, and, if yes, why?

In any case, it is nice to read that you fix your issue and your are moving forward with STM32 ;)

Best regards,

Stefano