LoRaWAN FreeRTOS End Node - Context Switching Hangs

[_iit_]

Hello everyone,
I am experiencing a problem with a firmware based on the LoRaWAN FreeRTOS End Node example (FW Package 1.3.0).
The problem is with the FreeRTOS kernel that non-deterministically stops performing context switching. I have already opened a discussion on the FreeRTOS forum where some considerations arose, but at this point I have been advised to continue in this community since the problem would seem to be related to the Pre and Post sleep functions that maneuver the systick when entering and exiting low power. Those functions are custom and written by ST. You can find the discussion here: Context Switching Hangs - Issue - Kernel - FreeRTOS Community Forums
If the link cannot be posted/opened here, I can arrange to report all the details.

Thank you very much

[nouirakh]

Hello @_iit_ ,

Welcome to Community!
Could you please, provide more details on the issue here on STCommunity forum this may be helpful for Community users, however in order to reproduce the issue and work on a solution, can you please provide more details on your setting project and if possible, attach your .ioc file/configuration ?

[_iit_]

I’m using FreeRTOS V10.2.1 with a STM32WLE5 microcontroller running at 48MHz (ARM Cortex-M4) with the GCC-ARM_CM3 porting, packed in the WL package 1.3.0

My application consists of several tasks with different priorities; some of them wait for a direct notification sent by other tasks, one task wait for a queued element, others are notified from ISRs.
After a certain non-deterministic amount of time the context switching stop. By using an internal watchdog, refreshed periodically by a low priority task, this context switching problem will result in a reset. I can also reproduce the problem replacing the watchdog with a software timer to avoid reset so to break and get some informations about running status.
Stack size of each task is enough, set as twice the used one as reported in rtos analyzer during debug sessions. I do not have access to Tracealyzer; i just use the free rtos-views plugin of VScode.
All the enabled interrupt external to FreeRTOS have the same priority and equal to LIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY.
Also, different optimization levels will increase/decrease the frequency of problem during execution time.
Next is freertos config file:

 

/* USER CODE BEGIN Header */
/*
 * FreeRTOS Kernel V10.2.1
 * Portion Copyright (C) 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 * Portion Copyright (C) 2019 StMicroelectronics, Inc. All Rights Reserved.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy of
 * this software and associated documentation files (the "Software"), to deal in
 * the Software without restriction, including without limitation the rights to
 * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
 * the Software, and to permit persons to whom the Software is furnished to do so,
 * subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in all
 * copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
 * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
 * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
 * IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 *
 * http://www.FreeRTOS.org
 * http://aws.amazon.com/freertos
 *
 * 1 tab == 4 spaces!
 */
/* USER CODE END Header */

#ifndef FREERTOS_CONFIG_H
#define FREERTOS_CONFIG_H

/*-----------------------------------------------------------
 * Application specific definitions.
 *
 * These definitions should be adjusted for your particular hardware and
 * application requirements.
 *
 * These parameters and more are described within the 'configuration' section of the
 * FreeRTOS API documentation available on the FreeRTOS.org web site.
 *
 * See http://www.freertos.org/a00110.html
 *----------------------------------------------------------*/

/* USER CODE BEGIN Includes */
/* Section where include file can be added */
/* USER CODE END Includes */

/* Ensure definitions are only used by the compiler, and not by the assembler. */
#if defined(__ICCARM__) || defined(__CC_ARM) || defined(__GNUC__)
 #include <stdint.h>
 extern uint32_t SystemCoreClock;
#endif
#define configENABLE_FPU 0
#define configENABLE_MPU 0

#define configUSE_PREEMPTION 1
#define configSUPPORT_STATIC_ALLOCATION 1
#define configSUPPORT_DYNAMIC_ALLOCATION 1
#define configUSE_IDLE_HOOK 1
#define configUSE_TICK_HOOK 0
#define configCPU_CLOCK_HZ ( SystemCoreClock )
#define configTICK_RATE_HZ ((TickType_t)1000)
#define configMAX_PRIORITIES ( 56 )
#define configMINIMAL_STACK_SIZE ((uint16_t)256)
#define configTOTAL_HEAP_SIZE ((size_t)32768)
#define configMAX_TASK_NAME_LEN ( 16 )
#define configUSE_TRACE_FACILITY 1
#define configUSE_16_BIT_TICKS 0
#define configUSE_MUTEXES 1
#define configQUEUE_REGISTRY_SIZE 8
#define configCHECK_FOR_STACK_OVERFLOW 1
#define configUSE_RECURSIVE_MUTEXES 1
#define configUSE_COUNTING_SEMAPHORES 1
#define configENABLE_BACKWARD_COMPATIBILITY 0
#define configUSE_PORT_OPTIMISED_TASK_SELECTION 0
#define configUSE_TICKLESS_IDLE 1
/* USER CODE BEGIN MESSAGE_BUFFER_LENGTH_TYPE */
/* Defaults to size_t for backward compatibility, but can be changed
 if lengths will always be less than the number of bytes in a size_t. */
#define configMESSAGE_BUFFER_LENGTH_TYPE size_t
/* USER CODE END MESSAGE_BUFFER_LENGTH_TYPE */

/* Co-routine definitions. */
#define configUSE_CO_ROUTINES 0
#define configMAX_CO_ROUTINE_PRIORITIES ( 2 )

/* Software timer definitions. */
#define configUSE_TIMERS 1
#define configTIMER_TASK_PRIORITY ( 2 )
#define configTIMER_QUEUE_LENGTH 10
#define configTIMER_TASK_STACK_DEPTH 256

/* Set the following definitions to 1 to include the API function, or zero
to exclude the API function. */
#define INCLUDE_vTaskPrioritySet 1
#define INCLUDE_uxTaskPriorityGet 1
#define INCLUDE_vTaskDelete 1
#define INCLUDE_vTaskCleanUpResources 1
#define INCLUDE_vTaskSuspend 1
#define INCLUDE_vTaskDelayUntil 1
#define INCLUDE_vTaskDelay 1
#define INCLUDE_xTaskGetSchedulerState 1
#define INCLUDE_xTaskResumeFromISR 0
#define INCLUDE_xTimerPendFunctionCall 1
#define INCLUDE_xQueueGetMutexHolder 1
#define INCLUDE_uxTaskGetStackHighWaterMark 1
#define INCLUDE_eTaskGetState 1

/*
 * The CMSIS-RTOS V2 FreeRTOS wrapper is dependent on the heap implementation used
 * by the application thus the correct define need to be enabled below
 */
#define USE_FreeRTOS_HEAP_4

/* Cortex-M specific definitions. */
#ifdef __NVIC_PRIO_BITS
 /* __BVIC_PRIO_BITS will be specified when CMSIS is being used. */
 #define configPRIO_BITS __NVIC_PRIO_BITS
#else
 #define configPRIO_BITS 4
#endif

/* The lowest interrupt priority that can be used in a call to a "set priority"
function. */
#define configLIBRARY_LOWEST_INTERRUPT_PRIORITY 15

/* The highest interrupt priority that can be used by any interrupt service
routine that makes calls to interrupt safe FreeRTOS API functions. DO NOT CALL
INTERRUPT SAFE FREERTOS API FUNCTIONS FROM ANY INTERRUPT THAT HAS A HIGHER
PRIORITY THAN THIS! (higher priorities are lower numeric values. */
#define configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY 5

/* Interrupt priorities used by the kernel port layer itself. These are generic
to all Cortex-M ports, and do not rely on any particular library functions. */
#define configKERNEL_INTERRUPT_PRIORITY 		( configLIBRARY_LOWEST_INTERRUPT_PRIORITY << (8 - configPRIO_BITS) )
/* !!!! configMAX_SYSCALL_INTERRUPT_PRIORITY must not be set to zero !!!!
See http://www.FreeRTOS.org/RTOS-Cortex-M3-M4.html. */
#define configMAX_SYSCALL_INTERRUPT_PRIORITY 	( configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY << (8 - configPRIO_BITS) )

/* Normal assert() semantics without relying on the provision of an assert.h
header file. */
/* USER CODE BEGIN 1 */
#define configASSERT( x ) if ((x) == 0) {extern void _ResetWithError(char *err_string, uint32_t err_code, char *file, uint32_t line); taskDISABLE_INTERRUPTS(); _ResetWithError("configASSERT", 1, __FILE__, __LINE__);}
#define traceTASK_SWITCHED_IN() do{extern char* __last_executed_taskname; __last_executed_taskname = (char*)(&(pxCurrentTCB->pcTaskName[0]));}while(0)
/* USER CODE END 1 */

/* Definitions that map the FreeRTOS port interrupt handlers to their CMSIS
standard names. */
#define vPortSVCHandler SVC_Handler
#define xPortPendSVHandler PendSV_Handler

/* IMPORTANT: This define is commented when used with STM32Cube firmware, when the timebase source is SysTick,
 to prevent overwriting SysTick_Handler defined within STM32Cube HAL */

#define xPortSysTickHandler SysTick_Handler

/* USER CODE BEGIN Defines */
/* Section where parameter definitions can be added (for instance, to override default ones in FreeRTOS.h) */

#define configAPPLICATION_ALLOCATED_HEAP 0

#if defined(configAPPLICATION_ALLOCATED_HEAP) && (configAPPLICATION_ALLOCATED_HEAP > 1)
 #define _REMOVE_CAST_PRE_PRE_(x)
 #define _REMOVE_CAST_PRE_(x) _REMOVE_CAST_PRE_PRE_ x
 #define _REMOVE_CAST_(x) _REMOVE_CAST_PRE_ x
 #if !defined(configTOTAL_HEAP_SIZE) || _REMOVE_CAST_(configTOTAL_HEAP_SIZE) > 16384
 #error "ERROR: STM32WLE5JBx has only 16KB of RAM1 allocable. Change implementation of FreeRTOS heap management, or change values according to the specific microcontroller!" 
 #endif
#endif

/* USER CODE END Defines */

#if defined(__ICCARM__) || defined(__CC_ARM) || defined(__GNUC__)
void PreSleepProcessing(uint32_t *ulExpectedIdleTime);
void PostSleepProcessing(uint32_t *ulExpectedIdleTime);
#endif /* defined(__ICCARM__) || defined(__CC_ARM) || defined(__GNUC__) */

/* The configPRE_SLEEP_PROCESSING() and configPOST_SLEEP_PROCESSING() macros
allow the application writer to add additional code before and after the MCU is
placed into the low power state respectively. */
#if configUSE_TICKLESS_IDLE == 1
#define configPRE_SLEEP_PROCESSING PreSleepProcessing
#define configPOST_SLEEP_PROCESSING PostSleepProcessing
#endif /* configUSE_TICKLESS_IDLE == 1 */

#endif /* FREERTOS_CONFIG_H */

 

I want to explain better the application:
A middle priority task is continuously notified by an ISR (relative to an EXTI interrupt) and fill a FreeRTOS queue with some values read from spi peripheral (frequency goes from 2.5 Hz to 40Hz). A low priority task will consume this data, waiting on queue. An higher priority task is notified by the ISR inside critical section as described before, with a periodicity of 10 minutes.
Rest of the time microcontroller goes to low power mode (stop2 specifically).

Next are the results of various tests done:

• Installed last version of FreeRTOS kernel: problem persists
• Disabled tickless idle: problem does not appear
• When problem comes, xTickCount stop to be incremented because scheduler remains suspended (uxSchedulerSuspended = 1)
• When problem comes those are the values of some kernel variables:
  

 

-----------------------------------------------------------------------------------
RAM Address| Object | Value |
-----------------------------------------------------------------------------------
0x200022e4 | volatile UBaseType_t uxCurrentNumberOfTasks | 0x11 |
0x20002370 | volatile TickType_t xTickCount | 0x278f6b |
0x200022f8 | volatile UBaseType_t uxTopReadyPriority | 0x0 |
0x20002344 | volatile BaseType_t xSchedulerRunning | 0x1 |
0x200022ec | volatile UBaseType_t uxPendedTicks | 0x934f02 |
0x20002374 | volatile BaseType_t xYieldPending | 0x1 |
0x2000232c | volatile BaseType_t xNumOfOverflows | 0x0 |
0x200022f4 | UBaseType_t uxTaskNumber | 0x11 |
0x20002328 | volatile TickType_t xNextTaskUnblockTime | 0x278f6e |
0x20002324 | TaskHandle_t xIdleTaskHandle | 0x200028a4 (Idle_TCB) |
0x200022f0 | volatile UBaseType_t uxSchedulerSuspended | 0x1 |
-----------------------------------------------------------------------------------

 And this is a partial dump of RAM:

 

 

Tickless mode uses the default implementation from port.c but the pre/post sleep hooks are installed and implemented by ST. Inside the pre/post hooks some peripherals clock are disabled by me.

From FreeRTOS community the PreSleepProcessing() and PostSleepProcessing() functions seem to be the cause of the issue; they say that I'm not supposed to manipulate the SysTick timer in those functions as I might break the careful logic in vPortSuppressTicksAndSleep(). However, it seems strange to me that the error is coming from those two functions considering that being written by ST I imagine that they were verified.

I cannot attach the full .ioc file here, but I can attach a part of it. Tell me which parts can be useful