LwIP and a TFTP client

[Will5]

I've run into an interesting problem and curious if anyone knows a solution.  I am adding TFTP support to an amateur radio project (soon to be published as open source).  All the other code is based on LWIPs netconn API and I want the TFTP code to use netconn as well.  Almost all the embedded LWIP examples I can find are for a TFTP server, not a client and all seem to be the native API too.  Undeterred I'm off writing my own, but I've run into what seems a fundamental incompatibility between the TFTP protocol and how I understand basic IP comms work!

The weird part about TFTP is that while the client sends a UDP packet from it's "upper" port to the well know TFTP port on the server (69), for a read request, by spec the TFTP server responds not from its well-known port, but a new random upper port!  With LWIP (and this seems normal and correct), if you set up a connection to another device and it's port, only replies back from that port come back to you.  A packet coming back from a different port number would be considered another connection.  LwIP (I feel correctly) ignores the packet coming back from the TFTP server and of course things don't go well after that.

Just to make sure I understood the issue, I was able to force my particular TFTP server to respond on a specific port and then I cheated and over-wrote the netconn structs pcb.udp->remote_port value to the same thing and comms work.  That isn't a real solution, but just a check to make sure I wasn't off on a tangent. The unfortunate bit is I clearly have no idea ahead of time what the new port value is going to be ahead of that first data packet.

Going full raw mode would mess up all my other code working OK using netconn, so kinda stuck.  There does seem to be an option called NETCONN_RAW in the netconn_new() call and when looking at some other netconn calls they mention the portid being ignored if in "raw mode".  However, when I do searches on NETCONN_RAW even Google has trouble finding anything!  I tried following it in the LWIP source and quickly got lost.

Has anyone run into this and found an elegant way around this?  Not looking for code drops, just pointers if anyone has them.

Thanks in advance.

will

[Guillaume K]

It seems TFTP works like you describe.

As per the TFTP RFC 1350 chapter 4 "Initial Connection Protocol": (TID means "Transport Identifier")

In order to create a connection, each end of the connection chooses a
 TID for itself, to be used for the duration of that connection. The
 TID's chosen for a connection should be randomly chosen, so that the
 probability that the same number is chosen twice in immediate
 succession is very low. Every packet has associated with it the two
 TID's of the ends of the connection, the source TID and the
 destination TID. These TID's are handed to the supporting UDP (or
 other datagram protocol) as the source and destination ports. A
 requesting host chooses its source TID as described above, and sends
 its initial request to the known TID 69 decimal (105 octal) on the
 serving host. The response to the request, under normal operation,
 uses a TID chosen by the server as its source TID and the TID chosen
 for the previous message by the requestor as its destination TID.
 The two chosen TID's are then used for the remainder of the transfer.

 

 

UDP is connectionless. It's not TCP. Each packet is sent individually. There's no guarantee to receive an answer from same port as request was sent to.

So if LwIP requires to receive an answer from port 69 then it may be wrong. One has to check what function is used exactly and what it expects.

 

 

[Will5]

Thanks Guillaume:  Yes, I read (and re-read) the RFC several times and found a few tutorials on it and this is indeed how it works.  Just very weird it really does violate how IP connections are supposed to work!  The server is always supposed to answer back to the client from the port it got the request from.  That is how "the internet" works and (correctly) how LWIP works if you are using netconn.

Separately, I am going to ask some of my Unix/Linux gurus *why" TFTP works this way.  I'm sure it wasn't by accident, but now I've got to find a way to handle it.

Even going down the native/raw API path isn't clean.  Should I just assume any packet coming back from that server from any port must be a reply to a TFTP request?  No, I could have other connections with that server.

I'm sure there is a good answer, just haven't found it yet.  I just hope it doesn't require doing something very non-standard with LwIP to make it work.

BTW, after posting this I got a link to a similar thread I'm also participating in:

https://community.st.com/t5/stm32-mcus-embedded-software/stm32h7-as-tftp-client/m-p/795803#M62616

will

[Guillaume K]

That's not a rule at IP level. It's a rule at TCP level. And TFTP uses UDP, not TCP.

 

 

[Will5]

Guillaume:  OK, I thought it was a rule for UDP as well as TCP.  If so, then I'm hopeful LwIP has a way of handling it and I just haven't figured it out yet.  The part I struggle with is that if you change ports like TFTP does, what information is available for LwIP (or even an app if in raw mode) to know how to handle a packet coming in that has a new port number?  You could have multiple connections to that server, so nothing about the server info (IP address or MAC address) is unique, so maybe somehow rely on the client port number?  As it is a client, at least it has a unique upper port number.  Just for more context, here is the issue reduced to pseudo-code-ish and ignoring all the error checking to show the sequence:

/* Port setup */

Context->TFTPconn = netconn_new(NETCONN_UDP);

netconn_bind(Context->TFTPconn, IP_ADDR_ANY, 0) ; /* Not sure this matters */

netconn_connect(Context->TFTPconn,&(Context->TFTP_Server_IP),TFTP_PORT);

netconn_set_recvtimeout(Context->TFTPconn,TFTP_TIMEOUT) ;

 

/* Sending the TFTP RRQ packet */

tx_buf = netbuf_new();

netbuf_alloc(tx_buf,len);

pbuf_take(tx_buf->p, (const void *)packet,len);

netconn_sendto(Context->TFTPconn,tx_buf,&(Context->TFTP_Server_IP),SrvrTID);

netbuf_delete(tx_buf);

 

/* Waiting for the first packet back */

err = netconn_recv(Context->TFTPconn, &buf) ; /* Times out! */

 

Hopefully that gives an accurate context. If I cheat and force the TFTP server to only respond on port 30000 and then add the code Context->TFTPconn->pcb.udp->remote_port=30000 right before the netconn_recv() call I get the packet and all is good, so while not a viable solution, it confirms it is a case of LwIP rejecting the packet from the TFTP server because it is not coming from the expected port. It looks like I just need to figure out how to get LwIPs netconn API to accept any source port for my target port? If you look at the LWIP netconn API docs they hint at it saying the portid value is ignore in "raw mode", but for netconn API calls. There is a parameter called NETCON_RAW for netconn_new(), but again I can't find any docs or examples on how to use it.

 

I can't be the first person wanting to implement a TFTP client using netconn, but I certainly can't find any examples. I'll certainly publish the answer when I find it! (BTW, sorry about formatting, something went weird.)

 

will

[Will5]

I realize I didn't close out this topic officially.  After much additional searching there does not appear to be any way to address this with the netconn API.  The only option is to move to the raw API.  Ick.  I've been living with the workaround for now.

A bit later I ran into another problem that netconn doesn't handle well.  TCP timeouts.  There doesn't appear to be any way to do anything faster than the very long timeout of around 20 minuets.  Where ever I look I get a similar answer, no support using netconn, you must move to the raw API.  That is enough motivation to put that on the to do list.

It will be a big change and I can live with the limitations for now,so it is now a case of when, not if I move to the raw API.

will