Stm32H723 Ethernet DMA tail pointer

[chris1seto]

Hi,

I understand this has been discussed before, but can we please get an official explanation of how the dma tail pointer works in the Stm32H7 eth mac ip block? It's extremely difficult to write software to support the eth mac when the docs and example drivers are questionable and the actual functioning of the silicon is simply left to guess by a lack of official information.

 

Issue: 2974/3357 RM0468 Rev 3 clearly says that the tail pointer is "an offset from the base", ie, it is not an absolute address or an index of a descriptor (from 0 to max descriptors). This completely disagrees with the cube driver which sets this as an absolute address: https://github.com/STMicroelectronics/stm32h7xx_hal_driver/blob/a7ac5eac41b0c73072401566237ba202f07251cf/Src/stm32h7xx_hal_eth.c#L1218

 

So, can we get an official no-further-questions answer for what kind of address is supposed to go in the tail pointer?

 

Thanks,

[LCE]

Too long ago that I wrote my that driver...

From my source code I find that:

- at descriptor init, the tail pointer registers are set to the last descriptor

- RX: when re-allocating free RX descriptors to POOL pbufs, the RX tail pointer register is set to 0

- TX: when starting TX in low level output, the TX tail pointer register is set to 0 (after clearing TBU ETHERNET DMA flag)

 

[chris1seto]

@LCE Yeah, I've seen that the older version of the cube driver does that. I think that effectively disables the DMA suspension because then the address matching simply can never happen, though I am not sure about that. My question is: If the cube drivers effectively disable this suspension mechanism, what is the point of the dma tail pointer?

[LCE]

Yeah, no point there - it seems...

> I think that effectively disables the DMA suspension because then the address matching simply can never happen

I'd say the same. And with enough descriptors and quick enough handling, it should never get to the point that the descriptor ring gets "filled up".

 

I'm completely out of the ETH part since it's working perfectly for over a year now, so I can't quite remember what exactly is going on there.

 

[chris1seto]

A couple of other questions: If it's the case that the tail pointer isn't necessary if relying on the OWN bits, what's the point of setting it to the last descriptor at init? Why not just set it to zero from the beginning?

One other related question: Lets say that DMA is not processing descriptors because Current Descriptor Pointer == Descriptor Tail Pointer; If a packet comes in, is an RBU generated just the same as it would be if there was a shortage of DMA owned descriptors?

ST, can we please get an official answer on how the tail pointer works?

 

Thanks!

[LCE]

I just checked my code, and I must have had the same thoughts as you, because I found in descriptor init:

	ETH->DMACTDTPR = (uint32_t)&DmaTxDescriptors[ETH_TX_DESC_CNT - 1];
	//ETH->DMACTDTPR = 0;

 Same for TX.

I just changed that, ethernet still working the same.

Wasn't that something related to using the descriptors as a ring structure? So in that case the tail registers are only used as a TX trigger / RX free signal I think.

[Pavel A.]

 Is the ETH of STM32MP1 similar? If yes, I'd look at the Linux driver for MP1.

[chris1seto]

I believe I have finally figured out the purpose of the DMA tail pointer: https://www.mattkeeter.com/blog/2023-10-31-dma/

 

I believe this could mean that this post from @Piranha ( https://community.st.com/t5/stm32-mcus-embedded-software/how-does-ethernet-rx-descriptor-wrapping-work-on-stm32h7/m-p/239135/highlight/true#M15786 ) doesn't fully describe the danger in relying on the OWN bit because the tail pointer protects the eth dma from reading descriptors before they are fully rebuilt by software.

 

@Imen.D Can we please get a an official response on how the tail pointer works and how it should be used? Additionally, is it an absolute address as is implied in the HAL software or an offset as is said in the documentation?

 

---

The fix

Okay, that's enough fun for now.

Fortunately, it is possible to use the hardware safely!

The DMA configuration registers also includes the "Channel Rx descriptor tail pointer register" (ETH_DMACRXDTPR), which – as it turns out – must be used to protect descriptors when being written.

• The hardware peripheral stops processing descriptors when it reaches the one indicated by the tail pointer
• User code should update the descriptor, then update the tail pointer
• Updating the tail pointer triggers the hardware to re-read the current descriptor

In this configuration, the OWN bit still marks whether a descriptor is owned by hardware or user code, but the tail pointer acts as the primary barrier to prevent mismatched reads. Dan implemented this fix in our system, including an exhaustive explanation of why it works.

---

 

[LCE]

Thanks for these links, VERY interesting!

I might have had or seen the same problem when I wrote my own driver, that's why I defined my own descriptors and added some flags, for descriptor "protection" - and PTP usage (WT = wait).

---

 

typedef struct
{
	__IO uint32_t DESC0;			/* TX: Header or Buffer 1 Address[31:0] */
									/* RX: Buffer 1 Address[31:0] */
	__IO uint32_t DESC1;			/* TX: Buffer 2 Address [31:0] or Buffer 1 Address[63:32] */
									/* RX: Reserved */
	__IO uint32_t DESC2;			/* TX: control & buffer / header length */
									/* RX: Buffer 2 Address[31:0] */
	__IO uint32_t DESC3;			/* TX: OWN (+ CTXT) + control, Payload / Frame length*/
									/* RX: OWN (+ CTXT) + control */
	/* custom non-descriptor parts */
	uint32_t 	Buf1Addr;			/* backup buffer 1 address */
	uint32_t 	pNext;				/* pointer to next descriptor */
	uint32_t	Flags; 				/* flag info */
 struct pbuf *pPbuf;				/* pointer to lwIP packet buffer pbuf */
} EthDmaDescr_t;

/* +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ */
/* descriptor FLAGS */
#define ETH_DESCR_FLAG_TX_IN_PREP			(uint32_t)0x00000001
#define ETH_DESCR_FLAG_TX_DMA_OWN			(uint32_t)0x00000002
#define ETH_DESCR_FLAG_TX_WT_PTP			(uint32_t)0x00000004
#define ETH_DESCR_FLAG_RX_WT_PTP			(uint32_t)0x00040000

 

[chris1seto]

Hey @LCE , I had another question.

 

How does DMA know how big each descriptor is? I noticed you added additional fields to the descriptor struct, and that the ST driver contains a few extra of its own. How does it know the length of those extra fields so it can index the descriptor list? I searched all through the driver looking for the size of the descriptor struct to be considered, but I couldn't find anything.

 

Thanks,

[STea]

Hello @chris1seto ,

"Issue: 2974/3357 RM0468 Rev 3 clearly says that the tail pointer is "an offset from the base", ie, it is not an absolute address or an index of a descriptor (from 0 to max descriptors). This completely disagrees with the cube driver which sets this as an absolute."
the answer is:

The absolute address needs to be programmed in tail pointer register.

sorry for the confusion induced by this description in the reference manual.

Regards  

 

[chris1seto]

Hi @STea Thank you for conforming that.

 

I believe I may have found a bug in the HAL driver. Please see the following ticket: https://github.com/STMicroelectronics/stm32h7xx_hal_driver/issues/65

[STea]

Hello @chris1seto ,

I will proceed by closing this thread as it is actively discussed in your other thread.

Regards

[LCE]

It's well hidden in ETH->DMACCR.

/* DMACCR: Channel control register
 *	DSL		= xxx: Descriptor Skip Length,
 *				-> depends on "extra data" in descriptor,
 *				for now 2x 32bit vars
 *	PBLX8	= 0: NOT 8x PBL
 * 	MSS[13:0]: Maximum Segment Size, only used when TSE set for
 *				packet segmenting in HW
 */
	u32TempReg = (uint32_t)TCP_MSS;

/* DSL depends on descriptor size */
	uint32_t u32Dsl = ETH_DESCR_ARR_SIZE - ETH_DESCR_BASE_ARR_SIZE;
	if( u32Dsl > 7 )
	{
		#if DEBUG_ETHNETIF
			uart_printf("# ERR: EthDmaInit() Descriptor Skip Length = %lu\n\r", u32Dsl);
		#endif 	/* DEBUG_ETHNETIF */
		u32Dsl = 0;
	}
	u32TempReg |= (uint32_t)(u32Dsl << ETH_DMACCR_DSL_Pos);
	ETH->DMACCR = u32TempReg;

with:
#define	ETH_DESCR_BASE_ARR_SIZE		4
#define	ETH_DESCR_ARR_SIZE		(uint32_t)(sizeof(DmaTxDescriptors[0]) / 4)

Could be more elegant concerning the defines, maybe was a takeover from my F7 driver, which has different ETH / MAC.

[chris1seto]

That's what I suspected, I see it now:

 

https://github.com/STMicroelectronics/stm32h7xx_hal_driver/blob/a7ac5eac41b0c73072401566237ba202f07251cf/Src/stm32h7xx_hal_eth.c#L384

 

I wasn't expecting it to be in bits, so I didn't catch it at first.