UART how the overflow works ?

[Xenon02]

Hello ! 

I've been wondering, how UART works in receiving side ? I have like maybe 4/5 questions ? 

-> What if I have a RX buffer that is full ? The flag is on and I didn't read the data. Then the new data comes after sometime, will it overwrite my data and still the flag will be on ? 

-> Is the data in the RX buffer loaded bit after bit or is it loaded whole word (8 bits) in one go and not bit by bit ? 

-> How do I know when I type HAL_UART_Transmit and after that the peripheral will be sending data and after transmit command I have the command HAL_UART_Receive, so how do I know it was fast enough to read the whole data without loosing any new or old data ? Like it was to late although the commands were one after another.

-> What if I used IT or DMA, they are non blocking, and questions like above like what if I type Transmit and Receive will I make it in time ? or what if I didn't read data and the buffer sets the flag and new data comes. 

These questions came out of curiosity because I read something about how UARTs looks like conceptually but I wondered how it looks like in practice. Because there are many types of commands in UART and wondered "what if" stuff. Also I know that RX works all the time and not only when I type HAL_UART_Receive so I wondered how it works when I have full data or when I type HAL_UART_Receive after HAL_UART_Transmit, will it be fast enough to read full buffor because this Receive commands reads full buffors and not bit by bit data. And what if I type HAL_UART_Receive after some time .. Yea I didn't know how to check it so maybe asking here won't be a problem ;> Sorry if it is. 

[TDK]

-> What if I have a RX buffer that is full ? The flag is on and I didn't read the data. Then the new data comes after sometime, will it overwrite my data and still the flag will be on ? 

The data is not overwritten. The new byte is lost and the ORE flag is set. The ORE flag must be cleared before the peripheral starts working correctly again.

-> Is the data in the RX buffer loaded bit after bit or is it loaded whole word (8 bits) in one go and not bit by bit ? 

It is loaded one word at a time. Typically one word is one byte, but it can be 9 bits, for instance.

-> How do I know when I type HAL_UART_Transmit and after that the peripheral will be sending data and after transmit command I have the command HAL_UART_Receive, so how do I know it was fast enough to read the whole data without loosing any new or old data ? Like it was to late although the commands were one after another.

You need to know your system. Does data start coming in the middle of a HAL_UART_Transmit call? If so, you'll need to use HAL_UART_Receive_IT or HAL_UART_Receive_DMA and call it prior to the transmit call so that data is received.

-> What if I used IT or DMA, they are non blocking, and questions like above like what if I type Transmit and Receive will I make it in time ? or what if I didn't read data and the buffer sets the flag and new data comes. 

If you use IT, you need to read the byte in the interrupt handler. If you don't, OVR will be set. With DMA, it is automatically transferred into memory and you can read it at a later time.

 

You're free to ask questions here, but these points are covered in detail in the reference manual as well.

 

The OREDIS bit can also disable this error flag being set, but data will still be lost.

[Xenon02]

Holy guacamolly.

So there are 2 types of error of the same thing for IT it is OVR and without IT is ORE ? 

I've got 4 more questions (of course I will flag your reply as a solution) :

-> So there are 2 registers in UART for RX ? One that takes bit by bit and the other register that takes the whole data at once ? This makes sense I think because instead of having time before 1 bit comes I have time before a whole word comes if it makes any sense (before 1 bit vs before 1 word comes to read the actual data). 

-> So when ORE flag or OVR flag is set then Receiver perihperal doesn't take any new data (the register that takes bit after bit won't take any data so RX is disconnected) ? 

-> What if I let's say want to read 50 characters but the transmiter sends only 3 characters ? Like I send a command, and I want to receive 50 characters (just by mistake), and the transmiter only sends 3 characters like OK\0, I read the data OK\0 but the transmiter doesn't send next data but somehow I received 47 characters of "\0". Does it mean I read all the time the old data ? Weird it must be because it restarts the RX flag so it is all the time waiting for new data hmmmm. 

-> I tried to read from manual but like in the picture you've sent, it says that it has shift register and RDR register I didn't much understand why there were two registers and it says that during the ORE the shift register is overwritten and that data is lost hmmm so the RX is not disconnected and the data is still received by the shifter register but not by the RX register ? There are also so many flags that I at some point get lost in what is what and what should I think of and what not. 

[TDK]

-> So there are 2 registers in UART for RX ? One that takes bit by bit and the other register that takes the whole data at once ? This makes sense I think because instead of having time before 1 bit comes I have time before a whole word comes if it makes any sense (before 1 bit vs before 1 word comes to read the actual data). 

It's ORE in both cases. Same flag, same bit, just a typo in my response. (The overrun flag is called OVR on other peripherals, not sure why it's called ORE here instead.)

-> So when ORE flag or OVR flag is set then Receiver perihperal doesn't take any new data (the register that takes bit after bit won't take any data so RX is disconnected) ? 

Effectively, yes. The actual behavior is a little more complicated. There is a shift register, and that keeps getting updated, but the data never makes it into the DR register. The RX pin is never disconnected.

-> What if I let's say want to read 50 characters but the transmiter sends only 3 characters ? Like I send a command, and I want to receive 50 characters (just by mistake), and the transmiter only sends 3 characters like OK\0, I read the data OK\0 but the transmiter doesn't send next data but somehow I received 47 characters of "\0". Does it mean I read all the time the old data ? Weird it must be because it restarts the RX flag so it is all the time waiting for new data hmmmm. 

If the transmitter only sends 3 characters, that is all you will read. You won't "somehow" read additional 0 characters. You can use the IDLE event to determine when transmission has stopped in the event of variable-length transmission. For example, this is what HAL_UARTEx_ReceiveToIdle_DMA does.

-> I tried to read from manual but like in the picture you've sent, it says that it has shift register and RDR register I didn't much understand why there were two registers and it says that during the ORE the shift register is overwritten and that data is lost hmmm so the RX is not disconnected and the data is still received by the shifter register but not by the RX register ? There are also so many flags that I at some point get lost in what is what and what should I think of and what not. 

You don't have to understand the low-level hardware architecture to start. It sounds like you plan to use HAL. There are UART examples in the CubeMX repository, that's a good place to start. Consider connecting RX and TX (i.e. loopback mode) and verifying you can receive what you're sending.

You do of course need some level of understanding. One common mistake people make here is not understanding that receive needs to be active (i.e. HAL_UART_Receive* needs to be called) before the characters are sent.

If you create the correct code, ORE should never become set.

[Tesla DeLorean]

Peripheral registers are not "memory" but a window into logic and state-machines.

There is a shift register in the background, once it is full it gets moved to the receive data register, if that is empty.

Most STM32 will generate an interrupt for each byte received, and you have around one byte time to recover it before the next potentially arrives. The HAL clouds this a bit because you tell it how large the buffer expectation is. In a more natural implementation you'd receive a byte at a time, and process via a buffer or stateful analysis, so you can understand where the data is complete. For example an AT command spanning a few bytes and terminated by a carriage return / line feed pair ( <CR><LF> )

I'd imagine there are some college level texts explaining how UARTs work and how peripherals are designed / built.

[Xenon02]

>I'd imagine there are some college level texts explaining how UARTs work and how peripherals are designed / built.

I've read how basic uart works from some websites but they were simple that didn't consider whether we read it in time or not. I don't know if I can send links to these websites so I just mention them for now. 

>Peripheral registers are not "memory" but a window into logic and state-machines. 

Window ? I imagined it that there is shift register which just updates all the time and DR register that takes the full data from shift register all at once. So like two step to update RX register so that it updates every 8 bits and not every 1 bit so there is time to take data. 

>For example an AT command spanning a few bytes and terminated by a carriage return / line feed pair ( <CR><LF> )

I just didn't understand why I got so many "\0" when I only requested it to send me back "OK\0". But accidently requested 50 bytes from RX, but of course it sends 3 bytes which is "OK\0". So I wondered if RX just spitted ERROR and stopped doing UART_Receive and just filled the actual array with "\0" or what. 

[Pavel A.]

Let's put it into proportion. USB, CAN, Ethernet are much more complex than the UART. If UART confuses you... too bad. Cheer up.

 

[Xenon02]

I mean I know it looks lame ... I also know it is just simple two commands, I was just wondering how it always gets information in time, usually I think nobody thinks about it but I just wondered that's all so these questions came up. 
Like didn't you think like when I use a command to transmit how much time do I have to get data ? or will the command after transmit which is receive will be read and interpreted on time ? 

I know there are more complex systems and sorry if my confusions seemed disappointing to the point where simple UART made me confused. I mean those speeds made me wonder how my code just makes it in time everytime :D Because I've learned that timing is very important in logic gate systems. So I tried to understand how I just got it in time like 1ms or smaller for bigger baund rate it is just unimagible for me that using two commands one after another I was able to send transmit and read the receive hal command, interpret it and read the RX buffer while the transmit was sending the data during that time where code was reading receive hal command, interpreting it etc.

So fast (i know there are faster just saying). 
Your comment didn't cheered me up ;> But I understand how it looks like. 
I apologize and sorry for my questions and how they look like ;> 

[Pavel A.]

But @TDK already has answered your question. Please read his reply again. Think what if the device on the other end of the UART does not wait at all and echoes back each sent character?  Unfortunately the HAL library for UART is not quite usable for many real life cases. You'll have to roll your own bi-directional communication, look for suitable examples.

 

[TDK]

When only 3 characters are received, HAL_UART_Receive doesn't do anything with the other 47 bytes. They have the same value they had before the function was called.

I wouldn't spend time worry about how exactly ORE affects the peripheral. It is an error condition--you shouldn't be hitting it. The reference manual does explain it in detail, which I've explained and screenshotted in my first reply.

Receive must be called before data on the line is transmitted. There isn't a buffer or anything that stores it until the receive function is called.

[Xenon02]

You know what thinking about it, maybe my post was to long.
But still this answer bugs me : "Receive must be called before data on the line is transmitted. There isn't a buffer or anything that stores it until the receive function is called.".

Hmmm You mean that HAL_UART_Receive must be called before data appears in transmit line (so it is the same as calling Receive function before data appears in RX line), makes not much sense, because After calling HAL_UART_Transmit it can send data right after ending calling the Transmit function and then we call Receive function like here : 

Like there are two starting HAL_UART_Receive, the red one is ideal that it if fast enough after transmit function that it starts waiting for RX flag in the starting bit of RX but what if function is slower and the call of HAL_UART_Receive end waiting in the second bit out of 8 bit of the receive data ? Dunno how fast this function is interpreted, probably the red one is ideal for the speed 9600, because it is 1ms and the function to read (HAL_UART_Receive) takes couple ns so we have like less than 1 ms spared time before the full data comes I guess, but for faster baund speed dunno. 

But like I said, I don't know why Receive must be called before data appears when I don't know if it fully appeared after HAL_UART_Transmit or not, usually in simple examples people call HAL_UART_Receive after HAL_UART_Transmit, but maybe data already appeared in RX line while calling HAL_UART_Receive so it doesn't make sense + there is this DR register to hold this 8 bit word for some time. 

Sorry and thanks.

 

[Pavel A.]

---

@Xenon02 wrote:

Receive must be called before data is transmited in line ? 
Usually I see this examples : 

HAL_UART_Transmit(); 

HAL_UART_Receive();

What you wrote above applies to so called half duplex mode, when only one side can send and the other side must listen. There indeed are systems that work in this mode (think RS-485).

But often the UART is used in full duplex mode, when both TX and RX directions work independently at the same time. In this mode a device (such as a modem) can echo back received characters immediately after receiving them. The red vertical line on your picture that separates receive from transmit simply does not exist. They overlap. Will your code snippet work in this case? (Surprise! it won't). Therefore you have to start at least the receive process "in background" so that it won't  wait for transmit. Again as Mr. Kostka wrote, the functions to use are non-blocking  HAL_UART_Receive_IT or HAL_UART_Receive_DMA - but the latter has complications (the timeout thing, TL;DR) so eventually people write their own interrupt driven receive code. If this still does not make it clearer... just have a good nap, drink more water and search for more full-duplex examples.

 

[Xenon02]

> "What you wrote above applies to so called half duplex mode, when only one side can send and the other side must listen. There indeed are systems that work in this mode (think RS-485).

But often the UART is used in full duplex mode, when both TX and RX directions work independently at the same time. "

I used this same command setup in my Bluetooth, and in other uart devices as well. 
And It was set as full duplex, I saw a half duplex simple wire but maybe it was this exactly. 

I also used two different lines for TX and RX because I used two wires to connect RX and TX so I tried to show how both lines looks like. 

But if both TX and RX are sending at the same time then yea my code won't work because the assumption is that after sending the TX command the device will send data after receiving the full command, so after receiving full data it sends back data in the other lane. That's why there are two lanes. 

My questions were partly answered, I wondered just that if it worked like in my diagram which I believe some work like that. At least the Bluetooth worked like that I guess. Because I wrote the same code like in the examples I gave. 

I didn't know how to ask the question. Because it might be just to specific. I wondered if just the function is fast enough to be before the RX flag is ON, like in the picture. 

I just interpreted that maybe this HAL_UART_Receive in that diagram picture, the call itself is fast enough to wait for RX flag before 1ms passes for 9600 baund rate because the function maybe takes like ns to be executed so 1ms - some nano secons = couple of micro seconds, still some time to spare. For higher baund rate I dunno maybe the function is still fast enough to be ready for RX flag before the whole data is sent to the DR register. 

Still I don't know if my question was asked correctly. Because I feel like they hit the question but not specifically it. 

Thank you Pavel for the answer. I'll take a nap and drink water but still I am new to it so you know ;> 
I think a bit ahead of time analyzing it deeply. Simple concept of UART is simple but details can make me think and the detail was whether the function call when it works as half-duplex it sends data by Transmit function, after it sends full data the Receive function is called (HAL_UART_Receive) and I wondered how fast this function is executed/called dunno how to say it, how long it takes for this function to be ready to ready RX flag (it does other stuff before checking the RX flag so it must be prepared by that time some data is already being stored in shift register and after some time it is sent to the DR register which then sets the RX flag but will the function will be ready by that time ? As I mentioned while the code is being proccessed the data is comming from the device, so will it be fast enough the function to get to the point to wait and check the RX flag. 

I gave some assumptions and maybe they are true for slow baud rate like 9600 but for faster, I dunno.