setting a backlog, response issue with TCP Raw Socket Server

[VSomasekhar]

1. How to set the number of simultaneous connections (backlog) in the Raw Socket Listen API? This is the relevant code snippet: /* start tcp listening for echo_pcb */ tcp_echoserver_pcb = tcp_listen( tcp_echoserver_pcb ); In contrast, the BSD/LWIP Sockets Listen API is: int lwip_listen(int s, int backlog)
2. Do I have to enable this macro in opt.h for the above purpose? #define TCP_LISTEN_BACKLOG
3. I am running a modified TCP Echo Server on the Board and sending continuous sequence of Modbus requests from the Modscan32 tool to the Modbus server running on the Board with STM32H723Zet6 MCU. But the response is given only to the first Request. The TCP Server is based on the source in the webpage: https://community.st.com/t5/stm32-mcus-embedded-software/how-to-send-data-lwip-tcp-server-to-client/td-p/119556
4. How to modify the TCP Raw Socket Server mentioned above to send the response for all the request packets? As mentioned, the TCP Raw Socket Server is responding only once, that too after restart.

[STea]

Hello @VSomasekhar,

By tweaking the options in lwipopts.h, it should be possible to receive more incoming connections: MEMP_NUM_TCP_PCB, MEMP_NUM_TCP_PCB_LISTEN, MEMP_NUM_NETCONN, etc...

You can refer to the following link for lwIP internal memory pool management: http://www.nongnu.org/lwip/2_0_x/group__lwip__opts__memp.html.

Refer to Solved: How to tune LwIP to receive more incoming connecti... - STMicroelectronics Community

Regards

 

[STea]

Hello @VSomasekhar,

Any updates on this ?

Regards

 

[VSomasekhar]

Hi,

I've found the specified Macros at: 

STM32CubeIDE\workspace_1.13.2\J1424\Middlewares\Third_Party\LwIP\src\include\lwip\opt.h

Those Macros are already having certain default values. I am not exactly sure what other values should be there for these Macros. I've attached a text file (partof_opts.h) with these Macros definitions in the opts.h file. Please check and mention if these default values are OK.

Let me make the observation more clear:

When I connect the ModScan32 tool to the Modbus server on the Board, the very first request will be answered correctly in a Modbus response packet. The tool continues to send the requests, which are not answered. Since this Modbus server is based on the TCP Raw Socket Echo server, it simply echoes the request data in the response packet for the 2nd request onwards. Hope this helps. 

[STea]

Hello @VSomasekhar ,

This is clearly a problem with your implementation of callbacks when receiving a message. can you share the app implementation in which you construct the Modbus response. as you based your implementation on the TCP Raw Socket Echo server example you need to construct a packet for each message received in tcp_echoserver.c I assume.
Regards

[STea]

Hello @VSomasekhar ,

In the code you shared in tcp_echoserver_recv you treat the first incoming data with a Modbus response, and you start to echo back the messages received (see the comments in code)

/* first data chunk in p->payload */ the first packet recieved
 es->state = ES_RECEIVED_SERVER;

 //Modbus entry point
 pTx = ModbusIxFace(p);

 /* store reference to incoming pbuf (chain) */
 es->p = pTx;
 
 /* initialize LwIP tcp_sent callback function */
 tcp_sent(tpcb, tcp_echoserver_sent);

 /* send back the received data (echo) */
 tcp_echoserver_send(tpcb, es);

 ret_err = ERR_OK;
 }
 else if (es->state == ES_RECEIVED_SERVER)
 {
 /* more data received from client and previous data has been already sent*/other packets
 if(es->p == NULL)
 {
 es->p = p; //pTx->next;

 /* send back received data */
 tcp_echoserver_send(tpcb, es);
 }

 

try adding the Modbus entry function before passing the pointer to buffer structure to handle "more data received form client "

/* first data chunk in p->payload */ the first packet recieved
 es->state = ES_RECEIVED_SERVER;

 //Modbus entry point
 pTx = ModbusIxFace(p);

 /* store reference to incoming pbuf (chain) */
 es->p = pTx;
 
 /* initialize LwIP tcp_sent callback function */
 tcp_sent(tpcb, tcp_echoserver_sent);

 /* send back the received data (echo) */
 tcp_echoserver_send(tpcb, es);

 ret_err = ERR_OK;
 }
 else if (es->state == ES_RECEIVED_SERVER)
 {
 /* more data received from client and previous data has been already sent*/other packets
 if(es->p == NULL)
 {
 //Modbus entry point
 pTx = ModbusIxFace(p);
 /* store reference to incoming pbuf (chain) */
 es->p = pTx;
 /* send back received data */
 tcp_echoserver_send(tpcb, es);
 }

this is my suggestion after seeing you code and what I got from your description of its behavior. I'm still clawless to the way the ModbusIxFace() function constructs Modbus packets.
Regards

[VSomasekhar]

I have implemented the suggested code change. Now I'm getting more number of responses, around 8-12. After that, the server-client connection is breaking. The server continues to run, but there is no ping response and Modbus response from the Board.

These are the requested details regarding the Modbus processing. The ModbusIxFace(..) function is implemented as follows:

 

struct pbuf * ModbusIxFace(struct pbuf *p)

{

MBDEFS_BOOL mbHandle;

struct pbuf *pTx = pbuf_alloc(Layer, TX_BUFFER_SIZE, bufType);

mbHandle = HandleModbusRequest(p->payload);

memcpy(pTx->payload, (void *)(QUERY_MESG_HEADER *)MBSlave_SendBuffer, sizeof(MBSlave_SendBuffer));

//pIx = p;

return pTx;

}

 

The HandleModbusRequest(p->payload) function receives the Modbus request packet in payload of

the packet buffer p. The payload is typecasted into the Modbus packet structure, and the internal

fields are accessed. As per the Modbus protocol specification, the field values are processed, and

the response is built into the byte array, as given below:

MBSlave_SendBuffer[TX_BUFFER_SIZE]

This byte array is copied into the payload of the received packet buffer, pTx, as shown in the

above function.

Although one response packet is sufficient for one request, the response has to be there for

every request. Please help in this regard.

[VSomasekhar]

One small correction please. pTx is a new packet buffer and not the received packet buffer. 

[VSomasekhar]

Today's observations: The ModPoll tool continuously sends the configured Modbus requests. With the suggested source code changes, the expected Modbus response is sent for the first 9 requests. I've repeated and confirmed that this is the case. Why is it 9? After 9 responses, it is falling back to echo server mode. I can see the logic in the suggested code changes, and I think it should work. Or maybe the state changes have to be managed more suitably? Or a different Server altogether need to be installed? My initial understanding was that by properly setting the Response, it should be possible to change it to a standard server. I did this attempt, as I couldn't find a better option. I need help in this regard. Many thanks in advance. 

[STea]

Hello @VSomasekhar ,

I think that the server stops responding because you are running out of memory as you allocate a packet each time you try to send a response and then this packet should be free afterwards.
you need also to set the pointer back to NULL after sending the response to make sure you are not holding the received buffer value in your send buffer.

ideally you should implement proper buffer management for this to work properly and ensure that you have sufficient space in the pool for handling data in LWIP.
Regards

 

[VSomasekhar]

Hi,

Thanks for the suggestion. I've created a packet buffer in ModbusIxFace(struct pbuf *p), using pbuf_alloc(). I've freed it in tcp_echoserver_recv(..), using pbuf_free(), where:

es->state == ES_CLOSING

But now the server is responding 4-5 times only and then stopping responses. The file is attached. Please check and suggest. I've tried the same at other locations also, where pbuf_free(p) was called and where: es->state = ES_CLOSING_SERVER;

In the program, I've created only one packet buffer, and tried to free it. But it doesn't seem to improve - hardly 4-5 responses. 

[VSomasekhar]

Thanks again for your help. Today I came to know that our customer is going to order 3000-5000 boards very soon. We need to complete this project asap to keep this business. I am ready to provide any more data required, which doesn't violate my employer's IP. I have attached the Modbus request and response traces in that attachment. Please check and suggest any possible way to correct the server. 

As suggested by you, I have included the pbuf_free() function, as given in the tcp raw server file. But if that place is not suitable, please suggest a different place.

If not please suggest any other solution, including an appropriate server program, with LWIP or Raw socket server. 

1. Please confirm if this attempt to use the Echo Server works out or not. Initially I thought it should work. I couldn't find any other suitable server program. If available, please let me know.

2. This server is definitely sending continuous responses in the echo server mode, although I didn't find any "infinite loops" in the server program. Please help me understand how this can happen.

3. If there is any other server program available for this purpose, please help me with it. 

4. My employer's survival is dependent on this business. I request you to please help.

4. Thanks a lot for all the help. 

[STea]

Hello @VSomasekhar ,

I'll take a look at this and come back to you as soon as possible.
Regards

 

[VSomasekhar]

I thought it would be useful to attach the main.c file also for more clarity on this issue. Please find the application entry point and the relevant code flow in the attached main file. The tcpRawServer() function called in the main function is defined as: 

int tcpRawServer()

{

int Ret = 0;

//lwip socket based tcp echo server entry point

tcp_echoserver_init(portnum);

return Ret;

}

tcp_echoserver_init(..) is already known to you.

Just for your reference, this echo server is available at:

https://www.st.com/en/embedded-software/stsw-stm32070.html

 

[STea]

Hello @VSomasekhar ,

can you try to set a new project based on the example found in STM32CubeH7/Projects/NUCLEO-H723ZG/Applications/LwIP/LwIP_TCP_Echo_Server/Src/main.c at master · STMicroelectronics/STM32CubeH7 (github.com) because the main you are using is quite an old example in which we use old drivers.
Maybe try modifying the project From GitHub and tell me what you get afterwords.

you can take the memory management and dynamic allocation and chaining of pbufs on this example changing the echo part with you function construction messages to the es structure.

try calling you function for constructing Modbus packet before any call to tcp_echoserver_send() :

-in tcp_echoserver_recv(Handles incoming data, manages connection state, and sends data back to the client)

-in tcp_echoserver_poll(Periodically checks and handles pending operations, such as sending remaining data or closing the connection)

-in tcp_echoserver_sent(Handles the acknowledgment of sent data, sends any remaining data, and closes the connection if necessary)

Regards

[VSomasekhar]

I have downloaded the required files and will create a new project in the STM32CubeIDE now. I'll include the raw socket server files, do the changes in the tcp_echoserver_recv(..) function, add the requires Modbus functionality and test it. 

I need one clarification please. When the es structure is created, it is put into ES_ACCEPTED state in the

tcp_echoserver_accept(..) function. It is moved to ES_RECEIVE state in the tcp_echoserver_recv(..) function. If the received packet buffer is NULL, it is moved to ES_CLOSING state in the same function. 

 

The server is expected to run indefinitely, in a loop. From the above description, tcp_echoserver_recv(..) is supposed to be in a loop. From the echo server functionality, which runs indefinitely as observed, clearly, there is a loop. But where is the loop in the source code? I couldn't find it, please help. In other words, how does the function tcp_echoserver_recv(..) run in a loop?

 

In the standard BSD sockets or LWIP sockets programs, recv() and send() functions will be in an infinite loop. But in the case of raw sockets, tcp_echoserver_recv(..) has to run forever. What is it that makes it run forever?

[STea]

Hello @VSomasekhar ,

The loop is periodically called in the main file  

while (1)
 {
 /* Read a received packet from the Ethernet buffers and send it
 to the lwIP for handling */
 ethernetif_input(&gnetif);

 /* Handle timeouts */
 sys_check_timeouts();

#if LWIP_NETIF_LINK_CALLBACK
 Ethernet_Link_Periodic_Handle(&gnetif);
#endif

#if LWIP_DHCP
 DHCP_Periodic_Handle(&gnetif);
#endif
 }

The received messages will be then passed to the LWIP to handle it by trigging thee receive callbacks in Lwip when receiving a packet.
Regards

[STea]

Hello @VSomasekhar ,

based on the informations you previously shared i Don't see why you are seeing such behavior.

maybe try to overwrite the receive and send callback maybe they are causing this echo responses .
Regards 

[STea]

Hello @VSomasekhar ,

Any updates on this issue?
Regards