Parsing Raw Data

[AE104]

Hello,

I need to parse a raw data that I receive UART3 interface. Then, I will send to a PC to show results in serial monitor. Before implementing in the microcontroller, I tested my approach in a C compiler. A sample raw data in the buffer array and here is C code to do this job 

 

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main() {
 char buffer[] = "e\nM0000\nPda7F85F3Fu;ba48D503Dp,10,288\nPda7F9234Bu;ba4E2C324p,10,288\nPda806EC24u;baAE16C6Dp,10,288\nPda807B031u;baB360495p,10,288\n*\n\n";
 char *token;
 token = strtok(buffer, "\n");
 int package_nr = 0;

 while (token != NULL) {
 if (strcmp(token, "e") == 0) {
 printf("Response begin\n");
 } else if (token[0] == 'M') {
 printf("Measuring...\n");
 package_nr = 0;
 } else if (token[0] == 'P') {
 if (package_nr++ == 0) {
 printf("Receiving measurement responses:");
 }
 printf("\n%d. ", package_nr);
 char *P = strchr(token, 'P');
 char *packageLine = P + 1;
 char *running = packageLine;
 char *param;
 while ((param = strtok_r(running, ";,", &running)) != NULL) {
 if (strlen(param) == 0) {
 continue;
 }
 char paramIdentifier[3];
 char paramValue[10];
 strncpy(paramIdentifier, param, 2);
 paramIdentifier[2] = '\0';
 strncpy(paramValue, param + 2, 9);
 paramValue[9] = '\0';
 printf("%s%s ", paramIdentifier, paramValue);
 }
 } else if (strcmp(token, "*") == 0) {
 printf("\nMeasurement completed.\n");
 printf("%d data point(s) received.\n", package_nr);
 } else {
 printf("%s", token);
 }
 token = strtok(NULL, "\n");
 }

 return 0;
}

 

Then I got this result:

 

Response begin
Measuring...
Receiving measurement responses:
1. da7F85F3Fu ba48D503Dp 10 288 
2. da7F9234Bu ba4E2C324p 10 288 
3. da806EC24u baAE16C6Dp 10 288 
4. da807B031u baB360495p 10 288 
Measurement completed.
4 data point(s) received.

 

 

 

To implement this code in the microcontroller, I simply removed the buffer array and replace it with Rx_Buffer which collects all raw data with HAL_UART_RxCpltCallback function. Then, here is the modified code:

 

 while (1)
 {
 /* USER CODE END WHILE */

 /* USER CODE BEGIN 3 */
 
	 char *token;
 char msg[100];
	 token = strtok(RX_Buffer, "\n");

	 while (token != NULL) {
	 if (strcmp(token, "e") == 0) {
	 sprintf(msg, "Response begin\n");
	 HAL_UART_Transmit(&huart2, (uint8_t*)msg, strlen(msg), HAL_MAX_DELAY);
	 } else if (token[0] == 'M') {
	 sprintf(msg, "Measuring...\n");
	 HAL_UART_Transmit(&huart2, (uint8_t*)msg, strlen(msg), HAL_MAX_DELAY);
	 package_nr = 0;
	 } else if (token[0] == 'P') {
	 if (package_nr++ == 0) {
	 sprintf(msg, "Receiving measurement responses:");
	 HAL_UART_Transmit(&huart2, (uint8_t*)msg, strlen(msg), HAL_MAX_DELAY);
	 }
	 sprintf(msg, "\n%d. ", package_nr);
	 HAL_UART_Transmit(&huart2, (uint8_t*)msg, strlen(msg), HAL_MAX_DELAY);
	 char *P = strchr(token, 'P');
	 char *packageLine = P + 1;
	 char *running = packageLine;
	 char *param;
	 while ((param = strtok_r(running, ";,", &running)) != NULL) {
	 if (strlen(param) == 0) {
	 continue;
	 }
	 char paramIdentifier[3];
	 char paramValue[10];
	 strncpy(paramIdentifier, param, 2);
	 paramIdentifier[2] = '\0';
	 strncpy(paramValue, param + 2, 9);
	 paramValue[9] = '\0';
	 sprintf(msg, "%s%s ", paramIdentifier, paramValue);
	 HAL_UART_Transmit(&huart2, (uint8_t*)msg, strlen(msg), HAL_MAX_DELAY);
	 }
	 } else if (strcmp(token, "*") == 0) {
	 sprintf(msg, "\nMeasurement completed.\n");
	 HAL_UART_Transmit(&huart2, (uint8_t*)msg, strlen(msg), HAL_MAX_DELAY);
	 sprintf(msg, "%d data point(s) received.\n", package_nr);
	 HAL_UART_Transmit(&huart2, (uint8_t*)msg, strlen(msg), HAL_MAX_DELAY);
	 } else {
	 sprintf(msg, "%s", token);
	 HAL_UART_Transmit(&huart2, (uint8_t*)msg, strlen(msg), HAL_MAX_DELAY);
	 }
	 token = strtok(NULL, "\n");
	 }
	 return 0;


	}

 

Then I got this result in the serial monitor:

 

Response begin
 e!0006Receiving measurement responses:
 1.

 

So what can be source of the problem?
Thank you,

 

[Pavel A.]

Configure your terminal to handle \n as end of line (automatically add CR).

 

[AE104]

Thank you @Pavel A. ! I set it CR, Both NL and CR but it didn't work. I also recgonized that after Rx_Counter reaches to 8, the code goes and stuck at Hardfault function.

 

[Tesla DeLorean]

Code snippet really doesn't convey broader context as to how you accumulate and hand-off the reception buffer.

The transmit calls here will block, consider creating an output ring buffer / FIFO

On the receive do this also with interrupts so you don't miss characters from you multi-character time blocking code.

Instrument HardFault_Handler() to output actionable data.

Likely faults due to exceeding buffers or pointers, or data buffers being out-of-scope.

Perhaps assumption buffers are NUL terminated when not, and using string functions.

[AE104]

Thank you @Tesla DeLorean , here is the callback function to accumulate the raw data:

void HAL_UART_RxCpltCallback(UART_HandleTypeDef *huart)
{

 if (huart->Instance == USART3) // check if it's USART3 interrupt
 {
 RX_Buffer[RX_Counter++] = RX_Data[0]; // Store received data in buffer
 HAL_UART_Receive_IT(huart, RX_Data, 1); // Ready to receive next byte
 }

}

Can you give me an example for creating an output ring buffer / FIFO?

[Tesla DeLorean]

Providing source code is a bit beyond the scope of my participation here.

You should perhaps bounds check your RX_Counter, and look at how you determine a full/usable condition which you can flag, and move the content to a processing buffer, and restart your buffer acquisition here.