SDIO + FATFS + DMA + FreeRTOS advice, currently inconsistent write times + stalling other tasks

[LM.2]

Hey,

I have been working on a data logger project that requires sensors to be read at 10Hz, the data timestamped and stored to an SD card. I am using FreeRTOS and a Streambuffer is used to pass data from the measurement task to the storage task. The storage task writes to a daily log file whenever there are at least 1024 bytes in the stream buffer. This usually takes around 4-5ms however sometimes much longer with the highest I've logged being 551ms. Even when using DMA these long writes block other tasks. For example the measurement task that is meant to sample every 100ms will take 110-220ms between measurements.

What is the proper way to implement FatFS on SD cards using DMA without blocking other tasks? Are there any examples of this. What I have works the majority of the time, It can run for multiple hours before any long writes happen. Also I have logged the sensor read timing over UART and it is consistently 100ms between samples if I disable SD card writes. I also have stored a sample number with each measurement and no data is lost during the long writes, it just stalls enough to delay the measurements from happening on time.

I have the measurement task running at priority 4 and the storage task as priority 3. It's not critical for me to have fast SD writes as long as they don't block the functionality of other tasks. With all sensors enabled I will be reading about 2 kB/s worth of data.

I am using a STM32L486ZGTx processor.

Thanks.

[TDK]

Writes to SD cards are not guaranteed to be quick. Due to the nature of flash storage, there will be occasional writes which take much longer than others. There is no way around this.

You will need to implement a buffer which can store multiple values and write those to the card asynchronously when it is not busy.

[LM.2]

I'm using a Streambuffer which is large enough to store seconds worth of data. A write event is triggered whenever there are at least 1024 bytes of data in this buffer. Then only 1024 bytes are extracted from the buffer and sent to the SD card (I read that it was better to do in multiples of 512 bytes). Data can still be added to the streambuffer while a write is happening.

My issue is that during these occasional very long writes it blocks other timing critical functions from running on time. I also don't get any feedback that anything has went wrong other than the 100x increase in time to write. I call f_write and it returns FR_OK just much later than expected.

[LM.2]

Update.

I had forgotten to add portYIELD_FROM_ISR to the timer interrupt triggering my measurement task. Now timing isn't interrupted however the SD write freezes for multiple seconds (possibly forever) and my buffer overflows.

Are SD cards just not the way to store data in timing critical applications? Why is the timing so inconsistent.

The measurement task has small CPU utilisation, It just needs to do that at within a millisecond window. At up to 20Hz with overall usage around 3%.

There should be plenty of remaining time to write data without interrupting the measurements.