FileX + LevelX Log Circular Buffer Implementation STM32U585

[PJose.4]

Hi Everyone
Greetings

We are using an STM32U585 MCU and need to maintain a log of about 200 KB using FileX. Once the 200 KB storage is full, I don’t want LevelX to reclaim the entire file and erase it. Instead, I need the log to behave like a circular buffer, where only the oldest block of log data is erased and overwritten with new data. How this requirement can be achieved. For this requirement how much size shall be allocated  by fileX(im using an external flash of 8MB).

Thanks and Regards

Philip

[MFARH.1]

Hello @PJose.4 ,

To implement a 200 KB log functioning as a circular buffer with FileX on the STM32U585:

- FileX does not natively support circular buffer functionality. You need to handle this at the application level by writing to a fixed-size file and wrapping around to the beginning once the end is reached, thereby overwriting the oldest data.

- LevelX manages the physical flash memory (erasing, wear leveling) but does not delete files. Your circular buffer management will work seamlessly alongside LevelX.

Regarding size allocation, it is advisable to allocate a partition slightly larger than 200 KB, for example between 256 KB and 512 KB, to accommodate both the data and the file system overhead.

BR,

Maher

[PJose.4]

Hi @MFARH.1 

Greetings
Thanks for your reply
1. regarding the previous reply we are implementing circular buffers from our application level. We have implemented 2 100 kb files for storage of the logs and when the second one gets filled, first one gets erased and vice versa to replicate the circular buffer method.

2. Free sectors are calculated during media initialization. As logs are written to flash, the number of free sectors decreases. If the available free space drops below two blocks, a reclaim operation is triggered. During this reclaim, the first block will be erased, leading to the loss of driver initialization data and log entries. How can this issue be avoided?

3. It’s not feasible to allocate between 256 KB and 500 KB for a 200 KB file because writing each log entry reduces free sectors by 64 or 96 sectors(sector size 512) due to wear levelling. In this case, what would be the recommended partition size  and how prevent reclaim and data loss?

Thanks for the continued support
BR
Philip

[MFARH.1]

Hello  @PJose.4 

 

To prevent data loss during the reclaim operation, several solutions can be considered:

• Reserve a specific area of the partition for critical data that will not be erased automatically.
• Additionally, periodically backing up critical data to another memory (such as external memory) allows restoring this data in case it is erased during reclaim.

Regarding partition size, it is recommended to allocate significantly more space than the actual data size (for example, more than 500 KB for a 200 KB buffer) to provide sufficient margin for wear leveling and reduce the frequency of reclaim operations. Furthermore, optimizing the size of write operations and implementing over-provisioning (reserved space unused by the application) helps limit fragmentation, data loss, and extends the lifespan of the flash memory.

 

Regards,

Maher

 

[PJose.4]

Hello @MFARH.1 
Greetings

I'm attempting to allocate 600 KB of storage for a 200 KB file using FileX. Below is the configuration I’m using with fx_media_format() to initialize the media, as well as how the NOR flash geometry is being set up. Could you please review and let me know if there are any issues or anything I might have missed in the media initialization, the flash geometry setup or erase implementation?

fx_media_format(&nor_ospi_flash_disk, // Media control block pointer 
 fx_stm32_levelx_nor_driver, // LevelX NOR driver entry function 
 (VOID*) LX_NOR_OSPI_DRIVER_ID, // Driver ID (0x02) 
 (UCHAR*) fx_nor_ospi_media_memory, // Media buffer (uint32_t fx_nor_ospi_media_memory([FX_NOR_OSPI_SECTOR_SIZE/sizeof(uin32_t)])
 sizeof(fx_nor_ospi_media_memory), // Size of media buffer 
 FX_NOR_OSPI_VOLUME_NAME, // Volume name (e.g., "STM32_NOR_OSPI_FLASH_DISK") 
 FX_NOR_OSPI_NUMBER_OF_FATS, // Number of FATs (1)
 32, // Root directory entries 
 FX_NOR_OSPI_HIDDEN_SECTORS, // Hidden sectors (0) 
 LX_STM32_OSPI_FLASH_SIZE / FX_NOR_OSPI_SECTOR_SIZE, // Total sectors (e.g., 0x800000 / 0x200 = 16384) 
 FX_NOR_OSPI_SECTOR_SIZE, // Sector size (512 bytes)
 8, // Sectors per cluster
 1, // Number of heads; 
 1); // Sectors per track 

 

Flash geometry setup:

nor_flash->lx_nor_flash_base_address = (ULONG*)(LX_STM32_OSPI_FLASH_FILE_SYS_OFFSET); // Base address = 0x400000
nor_flash->lx_nor_flash_total_blocks = total_blocks; // total_blocks = (LX_STM32_OSPI_FLASH_FILE_SYS_SIZE / LX_STM32_OSPI_SECTOR_SIZE)//(600 KB / 64 KB) = 9 blocks
nor_flash->lx_nor_flash_words_per_block = block_size / sizeof(ULONG); // 64 KB per block
nor_flash->lx_nor_flash_driver_read = lx_ospi_driver_read_sector;
nor_flash->lx_nor_flash_driver_write = lx_ospi_driver_write_sector;
nor_flash->lx_nor_flash_driver_block_erase = lx_ospi_driver_erase_block;
nor_flash->lx_nor_flash_driver_block_erased_verify = lx_ospi_driver_block_erased_verify;
nor_flash->lx_nor_flash_driver_system_error = lx_ospi_driver_system_error;
rnor_flash->lx_nor_flash_sector_buffer = &ospi_sector_buffer[0]; // ospi_sector_buffer[LX_STM32_OSPI_SECTOR_SIZE / sizeof(ULONG)] 

Erase implementation (lx_stm32_ospi_erase) .// every time 4k will erase

INT lx_stm32_ospi_erase(UINT instance, ULONG block, ULONG erase_count, UINT full_chip_erase){
 INT status = 0;

 if (full_chip_erase)
 {
 if (OSPI_FLASH_DEV_NAME.EraseChip() != ARM_DRIVER_OK)
 {#if OSPI_FLASH_STATUS
 printf("Error while EraseChip OSPI Flash Interface\r\n");#endif status = 1;
 }
 }
 else {
 for (ULONG i = 0; i < erase_count; i++)
 {
 uint32_t sector_address = LX_STM32_OSPI_FLASH_FILE_SYS_OFFSET + ((block + i) * LX_STM32_OSPI_SECTOR_SIZE);//0x400000+(block+i)*(64kb)

 for (int j = 0; j < 16; j++) // Erase 64 KB block in 16 x 4 KB sectors 
 { 
 if (OSPI_FLASH_DEV_NAME.EraseSector(sector_address) != ARM_DRIVER_OK)
 {
 printf("Error while EraseSector OSPI Flash Interface\r\n");
 status = 1;
 }

 sector_address += 4096;
 }
 }
 }

 return status;
}

 

Thanks and Regards

Philip

[MFARH.1]

Hello @PJose.4 ,

The overall implementation, including the media format configuration, flash geometry setup, and erase function, is well developed and aligns with best practices for NOR flash managed by LevelX and FileX. All parameters such as sector size, total sectors, sectors per cluster, and block size are correctly defined and consistent with the hardware specifications.

However, it is critical to ensure that the buffers fx_nor_ospi_media_memory and ospi_sector_buffer are properly aligned to 512 bytes or a multiple thereof. Proper alignment is necessary to prevent data corruption and to guarantee compatibility with the DMA controller, which is essential for reliable and efficient flash operations.

 

Regards,

Maher

[PJose.4]

Hi @MFARH.1 ,
Greetings

As per your suggestion, I have updated fx_nor_ospi_media_memory and ospi_sector_buffer to be multiples of 512. However, during the reclaim process, valid data sectors are still being erased. We are writing 256 bytes of data per write operation. Could you please advise if there are any alternative solutions or configurations that might help?

Thanks for your support
Regards
Philip

[PJose.4]

Hi @MFARH.1 

Greetings

Thanks for the solution. The erase method is working by using sector size instead of block size for the erase function and configuration

Another query in this regard-  when encountering the following error codes during file operations, 

FX_IO_ERROR
FX_BOOT_ERROR
FX_MEDIA_INVALID
FX_FAT_READ_ERROR
FX_FILE_CORRUPT
FX_NOT_OPEN -- this occurs during the file creation
FX_NO_MORE_ENTRIES
FX_NO_MORE_SPACE
FX_SECTOR_INVALID
FX_INVALID_PATH
FX_MEDIA_INVALID
FX_NOT_A_FILE — This occurs even when the file already exists
FX_NOT_FOUND — This also occurs even though the file is present

is there a way to handle them without using fx_media_format()? Are there any other possible solutions?

Thanks and Regards
Philip

[MFARH.1]

Hello @PJose.4 ,

 

Yes, it is possible to handle these FileX errors without reformatting the media. You can use functions such as fx_media_check() to verify and repair the file system. It is also important to properly manage file opening and closing, as well as to validate paths and available space. Finally, in case of corruption, prioritize data backup and targeted repair rather than a full format.

 

Regards,

Maher

[Tesla DeLorean]

The minimum erase block size for most QSPI NOR FLASH in 4KB, and the Write Page is up-to 256-bytes (aligned). Would be better to commit to 4KB sectors rather than 512-byte.

If you're just writing in a loop, which is inherently journalling / wear-leveling, you might want to look at doing something directly. Not sure how the middle-ware maintains statistics / block-reallocation, but a FAT file-system is going to generate a lot of cycling on the FAT tables. And similar juggling if you try to maintain content where the erase block size is greater than the sector size.

NAND typically has some out-of-band data that can be used for age and redirect info.

[PJose.4]

Hi @Tesla DeLorean @MFARH.1 
Greetings

Thanks for the reply 

I’m currently unable to determine how to repair the file system when an error occurs (e.g., FX_IO_ERROR, FX_FILE_CORRUPT ), we are not able to open files or read data, which makes it unclear how a backup can be taken or how targeted repair can be performed.

Could you please clarify:

How to proceed with repairing the file system when read access is no longer possible?

Whether there is any way to recover data or take a backup if the media cannot be read due to the error?

BR
Philip