Question
STM32 + Quectel BG95 Modem: System hangs after ~60 days - UART Idle Line setup never returns
Background
We have a battery-powered device using:
- MCU: STM32 (STM32L4series)
- Modem: Quectel BG95 (cellular)
- Communication: UART with Idle Line Interrupt (HAL_UARTEx_ReceiveToIdle_IT)
- Power mode: Device goes to sleep periodically, wakes up to communicate with modem
Architecture Pattern
Our communication pattern:
- Device wakes from low-power mode
- Call low-level AT command function that:
- Enables UART RX Interrupt with Idle Line detection (HAL_UARTEx_ReceiveToIdle_IT)
- Sends AT command
- Waits for response with timeout
- Disables interrupt (HAL_UART_Abort_IT)
- Process response
- Go back to sleep
This means we re-enable/disable UART Idle Line interrupt on every transaction (potentially hundreds of times per day).
UART + Ring Buffer Setup:
#define RING_BUFFER_SIZE 2048
#define ISR_BUFFER_SIZE 1024
typedef struct {
lwrb_t rb; // lwrb ring buffer
volatile bool rx_data_ready_flag; // Flag set by ISR
uint8_t rb_buffer[RING_BUFFER_SIZE]; // Ring buffer storage
uint8_t isr_buffer[ISR_BUFFER_SIZE]; // Intermediate buffer for Idle Line ISR
} uart_rb_t;
uart_rb_t modem_rb;
ISR Callback:
void HAL_UARTEx_RxEventCallback(UART_HandleTypeDef *huart, uint16_t Size) {
if (huart == &MODEM_UART) {
// Save data from ISR buffer to ring buffer
lwrb_write(&modem_rb.rb, modem_rb.isr_buffer, Size);
modem_rb.rx_data_ready_flag = true;
// Re-enable Idle Line interrupt
int retries = 10;
do {
if (HAL_UARTEx_ReceiveToIdle_IT(&MODEM_UART,
modem_rb.isr_buffer,
sizeof(modem_rb.isr_buffer)) == HAL_OK) {
break;
}
retries--;
} while (retries > 0);
}
}
The Problem
After ~60 days of continuous operation, the device completely froze with no recovery.
Symptoms:
- System printed the last log line: >>StartParse >>>>
- Then complete silence - no further output
- No HardFault triggered (we have handler with reset + logging - it was never called)
- Device required power cycle
Last logs before hang:
<<EndParse <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>StartParse >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
"AT+QISTATE=1,1" --> [response received OK]
<<EndParse <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>StartParse >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
"AT+QISTATE=1,1" --> [response received OK]
<<EndParse <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>StartParse >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
"AT+QISTATE=1,1" --> [response received OK]
<<EndParse <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>StartParse >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[SYSTEM FROZE HERE - no further output]Notice: The command string was not printed in the last call, suggesting the code hung before that point.
Code Structure (simplified)
Parent function:
uint32_t tick = HAL_GetTick();
do {
return_code = modem_send_command_wait_parse_result(
"AT+QISTATE=1,1",
"+QISTATE:",
/* parsing params */,
300 /* timeout ms */
);
if (condition_met) break;
HAL_Delay(100);
} while (HAL_GetTick() - tick < 15000); // 15 second outer timeout
Low-level function structure:
int modem_send_command_wait_parse_result(..., int timeout, ...) {
// Local buffers
char formatted_command[1024] = {0};
char buffer_final[2048] = {0};
unsigned int total_bytes = 0;
printf(">>StartParse >>>>\n");
if (command_to_send != NULL) {
printf("\"%s\" --> ", command_to_send);
}
// Enable UART RX Interrupt with Idle-line detection
int retries = 10;
do {
if (HAL_UARTEx_ReceiveToIdle_IT(&MODEM_UART,
modem_rb.isr_buffer,
sizeof(modem_rb.isr_buffer)) == HAL_OK) {
break;
}
retries--;
} while (retries > 0);
if (retries == 0) {
return_code = -1;
}
uint32_t tick = HAL_GetTick();
// Main receive loop with timeout
while (return_code > 0 && ((HAL_GetTick() - tick) < timeout)) {
// Check flag and read from ring buffer
if (modem_rb.rx_data_ready_flag) {
modem_rb.rx_data_ready_flag = false;
int bytes = lwrb_read(&modem_rb.rb,
&buffer_final[total_bytes],
sizeof(buffer_final) - total_bytes);
total_bytes += bytes;
}
// Parse response, check for expected strings, etc.
// ...
}
// Cleanup
HAL_UART_Abort_IT(&MODEM_UART);
lwrb_reset(&modem_rb.rb);
return return_code;
}
My Questions
- Is re-enabling UART Idle Line interrupt on every transaction a valid approach? Could repeatedly calling HAL_UARTEx_ReceiveToIdle_IT (with 1KB buffer) → HAL_UART_Abort_IT → HAL_UARTEx_ReceiveToIdle_IT (hundreds of times over 60 days) cause UART peripheral corruption?
- The retry loop, when we enable idle line in low level, sometimes returns != HAL_OK (HAL_BUSY or HAL_ERROR). Is this expected behavior, or does it indicate underlying UART state problems that could accumulate over time?
- Race condition in flag handling? The rx_data_ready_flag is set in ISR and cleared in main loop without atomic operations. Could this cause issues:
// Main loop (non-atomic):
if (modem_rb.rx_data_ready_flag) { // Read
modem_rb.rx_data_ready_flag = false; // Write - ISR could interrupt here!
}- Could printf() cause the hang? We use printf extensively for debugging over a separate UART. Could printf buffer overflow or UART TX blocking cause the system to freeze without triggering exceptions?
- HAL_GetTick() overflow handling: After 50 days, HAL_GetTick() wraps around. Our timeout check is (HAL_GetTick() - tick) < timeout. Is this safe with overflow?
- Ring buffer overflow: If the 2KB ring buffer fills up and data is lost, could this cause the expected response string to never arrive, leading to timeout? Though this should be caught by the timeout logic...
Request for advice:
- Are we using UART Idle Line correctly for this use case (repeated enable/disable)?
- Is the flag handling race condition a real concern, or is it benign?
- Any known issues with long-term UART peripheral usage on STM32?
- Could be an issue the OPEN LOG dev board that we have connected to our device in order to collect logs into an SD card?