Ring Buffer vs Array Buffer

Go To Last Post
58 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yo Ho HO!

 

I am working on a commercial application where I am going to receive a few simple commands in ASCII and the application will send back various status information from a couple of sensors.

 

I have always used a simple receive buffer such as:

 

uint8_t receive_buffer[buffer_size];

and then in my USART ISR I have a simple pluck the byte from UDRx, and put it in the buffer and increment the write-to counter by one.  Should the received byte be the \r or \n then a flag is set to indicate the EOL. and the main code then reacts to the buffer contents when it sees the EOL flag.

 

Simple enough, and this would work just fine in this application.

 

So, heres the deal.  When I use CodeVisons wizard it always sets up a circular buffer.  I always convert it to my simple array type of buffer as the circular buffer always confuses me.  I have read teh wikipedia on circular buffers and have been reading the online articles about their implementation and my takeaway is this:

 

1) They can be either fill and wait, or fill and overwrite

2) They have separate write and read indexes.

3) They are good for constant stream data.

4) They store the most recent data when used in overwrite mode.

 

I am sure there are more that I havent come up with.

 

So that being said, for simple intermittent command/react data, do I need to use the circular buffer?  The commands are not coming in constantly, more like every 10 seconds to one minute.

 

Does it even matter which one I use?  Since I am sensing an EOL termination, the Wizard generated ring buffer is essentially being used as an array type buffer anyway correct?

 

Should I reset the write and read indexes after I finish reading the data out of the buffer,>>like a regular array type buffer<< or should I simply leave them be and just write/fetch from the next available location?

 

JIm

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You can use whatever you want.   Just be aware that buffers can overrun.    You can never guarantee that the NL will appear.

 

A circular buffer means that you can set an OVERRUN flag.   And choose whether to discard old bytes or ignore the new bytes.

Your simple array can also set an OVERRUN flag.    Discarding old bytes means you have to shift the whole buffer with every new byte.   Ignoring new bytes is easy.

 

If you intend to read "sentences" you simply use CV gets(sentence, max).

And it will fill your sentence[] from the circular buffer.

 

If you are a masochist you can even use scanf() or sscanf()

 

I would stick with the circular buffer.   It is difficult to handle your simple array[].    Especially if a new byte arrives while you are shifting your buffer around.

 

David.

Last Edited: Tue. Jan 21, 2020 - 04:28 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If the inbound data is always just repeats of the same thing .. a few characters and then an EOL .. then, no, you probably don't need a ring buffer. 

 

If you ever used a 16550 on a PC in the old days of serial ports you'll remember that the 16550 was just a "souped up" version of the original old Intel 8250 where the main thing it added was a 16 byte FIFO (ie ring buffer). (*)

 

Well in AVR land an RXC interrupt and a ring buffer is pretty much doing the same - it's just upgrading a "UART with no buffer" (well OK - 1 byte!) to a "UART with a buffer" that is the 16 or 32 bytes or whatever you allow for in the ring.

 

Loads of old PC programs got on just fine with an 8250 but the 16550 just took a bit of the workload off the programmer as you could now "get on with other stuff" safe in the knowledge that as long as you never let it get it more than 16 bytes behind then you'd always find the bytes that arrived while you were busy in the buffer.

So if you write a program that can always guarantee to see the EOL, process the string, clear the array and be ready for the next character arrival then you don't need to have an interrupt buffer to catch characters while you are still processing the "last string"

 

If there is a concern then do let it do the buffering, then extract bytes from the buffer into a second char[] array, when you pull out an EOL then begin to process that array and if anything else arrives while you are dong that it will be held in the ring until next time.

 

(*) "IBM PC" (the original 501) had the 8250 then the "IBM AT" came along with its fancy new 80286 CPU and also these souped up 16550 UARTs

Last Edited: Tue. Jan 21, 2020 - 04:40 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Should I reset the write and read indexes after I finish reading the data out of the buffer

 

Wow, seems like a lot of work.

 

The nice about an interrupt driven, ring buffer, in Bascom, is that it takes one line of code to set it up, with a parameter for how large you want the buffer to be, and then one can read a flag to see if there is any data within the buffer or not.

 

If there is data there are two read options, one which reads and removes the data from the buffer, which is often what one wants, and one which lets one see what the next character to read will be, but it doesn't actually read and remove it from the buffer.

 

All of the head & tail pointers, buffer full flags, etc., are handled behind the scene.  Truth be known, one can access them if they have a special reason for doing so, but in general the point of using it is that it is efficient, bug free, and easy to set up and use.

 

One would, I guess, use the hidden pointers to read the buffer to spot a CR / LF character within the buffer, if one didn't want to read them out of the buffer until the EOL was received.

 

JC     

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The simple array is both simple & fine for most intermittent use.   The circular has an advantage in that commands can overlap;  you can be decoding one (from point a to b in the buffer) , while another (or more) is coming in.  So you can have several commands queued up & can decode them at your leisure.  I do a scan/rescan looking for any CR to "split out" a string to process.  Sometimes I also fill those spots with zeros, to ensure they don't get processed twice & to more easily monitor what is still pending, if I dump the buffer contents.

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

DocJC wrote:
One would, I guess, use the hidden pointers to read the buffer to spot a CR / LF character within the buffer,
Ring buffers (like that buffer in the 16550) should be a "hidden thing". yes, when implemented in software (and presumably even when implemented in silicon?) there are separate read/write pointers and other "internal stuff" (sometimes a "number of bytes in buffer" counter) but the user should never be aware of these (presumably as in Bascom?). You just have a single "UART_getchar()" and it delivers you the next character. The fact that it was received 3 days ago and there's another 11 characters "behind it" still waiting in the buffer or that the buffer read and write pointers are at locations 12 and 6 or the count is 11 are all hidden from the "consumer".

 

In fact too often we see threads here were someone is implementing buffers and then they go and do stuff like "if (buffer[read_index] == '\n')" or whatever - but that is misusing the buffer stuff. They literally should only ever be doing stuff like "if (buffer_read_next() == '\n') ..." and not messing with the "internals".

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Agreed!  100%

But it looked like Jim was trying to hold all of the incoming data within the ring buffer's buffer, and not allocate additional memory for a "secondary" buffer.

Hence I was thinking about how one could still generate the EOL flag he is used to using.

 

I think, in fact, that it might be reasonable to set up a "large" FIFO buffer, and have a very small ring buffer to handle the USART module.

On the Rx Data ISR one would read the ring buffer, stuff the byte in the FIFIO, and set the EOL flag if indicated.

 

IIRC Bascom lets one set up a FIFO with another one line command.

 

So, if one uses either the interrupt driven ring buffer, or the ring buffer with a secondary FIFO, the user never has to process, (or even be aware of the existence of), any of the pointers.

 

JC

 

Edit: Typo

 

BTW, this constitutes my token "contribution" to the General Programming sub-forum for the year 2020!

As a non-programmer I'll now head back under my rock, where I belong!

 

JC

Last Edited: Tue. Jan 21, 2020 - 05:06 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

DocJC wrote:
But it looked like Jim was trying to hold all of the incoming data within the ring buffer's buffer,

Thats what I am/was intending to do.

 

DocJC wrote:
So, if one uses either the interrupt driven ring buffer, or the ring buffer with a secondary FIFO, the user never has to process any of the pointers.

Which is sort of what I was hinting here:

jgmdesign wrote:
......or should I simply leave them be and just write/fetch from the next available location?

 

In other words, when the RX_ISR 'sees' the EOL and sets the flag, the MAIN code simply reads the buffer using the RX index and resets the flag as the write index simply keeps writing to the next available location and so on.

 

Jim

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

DocJC wrote:
trying to hold all of the incoming data within the ring buffer's buffer, and not allocate additional memory for a "secondary" buffer.
which is the classic error we see here when others implement rings. They don't want to "waste" RAM on a "foreground buffer" but try to make do with the "internal" buffer the FIFO is using by fiddling around inside it.

 

Like a 16550 16 bytes should hopefully be all a FIFO ever needs - then you can define string[40] or whatever in the foreground to pull the buffered bytes to and do the processing on. The 16 or whatever for the ring just needs to be enough to ensure that during the "longest time away" (while doing other processing) there'll never be more than that many bytes arriving that need to be held for a bit.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I use both types in the same application(s) on the same serial port(s).

 

The ring buffer goes in the ISR as, if it's a power-of-2 sized, it is efficient to manage. It's there to buffer the USART output to ensure I don't miss any characters. Effectively I'm extending the depth of the USART's FIFO.

 

The array buffer is used to decode the data as you might want to do more to the index value than increment or decrement it.

#1 Hardware Problem? https://www.avrfreaks.net/forum/...

#2 Hardware Problem? Read AVR042.

#3 All grounds are not created equal

#4 Have you proved your chip is running at xxMHz?

#5 "If you think you need floating point to solve the problem then you don't understand the problem. If you really do need floating point then you have a problem you do not understand."

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I've used both. For me, the determinant for choosing one style over the other is timing. In particular, will the processing on each message ALWAYS be complete before something new begins. If the latter is possible, then the complicated Dance of Indices for a simple linear buffer easily gets tossed out in comparison to a ring. Or, if you start thinking about shifting buffer contents to avoid overflows.

 

By the way, the index management for a ring buffer is not so terrible. It seems overly complex at first but once you figure out what is happening, it is straight forward. I implement write_ring() and read_ring() functions that manage the indices. Then, it just works.

 

Jim

 

Until Black Lives Matter, we do not have "All Lives Matter"!

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'd use a straight buffer, but that's mostly because ring buffers are a bit less fun in assembler.  I like your "set flag at EOL, says command has arrived in full, then read'n'parse" idea.  Straightforward, simple, works.

 

With a ring buffer, you can also get buffer underrun errors, where you're trying to read bytes out faster than they're being put in.  Tell the "Wizard" to go jump in a lake (If Merlin, he should be all over that! :)

 

Put how long the commands can be into the spec, set the array size to the spec, and if spec is violated, throw errors.  Done deal.  S.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:

You just have a single "UART_getchar()" and it delivers you the next character.

I'm with cliff here, use a small (8, 16 or 32) char ring buffer that is hidden from main and just process the command/data into an index buffer if needed for your line of data / command parsing needs.

 

Flyover Jim

 

(Possum Lodge oath) Quando omni flunkus, moritati.

"I thought growing old would take longer"

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

As an aside, just how long and complicated are these ASCII queries for 'a couple of sensors'?  Are you trying to parse them like a 70's text adventure, "Please tell me the temperature of the living room", or more like:

 

a) Send living room temperature

b) Send dining room temperature

c) Switch between C and F

d) Switch between F and K

e) Send status

f) Send something irrelevant

g) Send the Gettysburg Address

h) &c.

 

Then you don't need a buffer at all, just parse the data register (and you can easily get more than 62 commands just with a typical keyboard.  You can have 254 with a front-end to encode your command (I'd keep $00 as a 'no command', if I were you...).

 

IMHO, reading and parsing buffers for me is always much faster than loitering around on the serial lines (thus the data underrun problem).  Just how fast is your serial bus, anyhow?  Reading the one byte (or character(!)) is even faster.  S.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Scroungre wrote:
Just how fast is your serial bus, anyhow? 

RS-232 - 115.2k

 

Scroungre wrote:
As an aside, just how long and complicated are these ASCII queries for 'a couple of sensors'? 

The 'command' is STATUS\r.

 

THe return is ascii encoded characters for two temperature probes, and a fan speed sensor along with a single digital input.  R E A L simple.

 

JIm

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

So set the 'command rec'd' flag on the first S, and ignore the rest.  Won't work if the 'S' is in italic, though...  wink  S.

 

Edited to add:

Unless it's only half-duplex, in which case you probably should wait for the EOL flag as well, just for 'Bus Idle' notification.  Meanwhile, why have a buffer at all?  Keep it in UDR and have the interrupt just set a couple of flags.  Am I prematurely optimizing?  S.

Last Edited: Tue. Jan 21, 2020 - 07:57 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Unless you are really short of RAM, I would use a standard rx interrupt driven ring buf as well.

Once you have a working setup you can reuse it as a 'black box' that just works.

Main code just queries to see if any chars are available and reads the next one from the ring buf when available into a separate 'command' array, do with that whatever you want.

The code required for ring buf is minimal eg. a common technique is all you need is a read and a write  index, which only need to be uint8_t,  you get the number of chars avaiable from write - read (requires that you always keep one space empty so you are wasting one byte of RAM, who cares) and then no interrupt blocking is needed.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'd use a straight buffer, but that's mostly because ring buffers are a bit less fun in assembler.

Straight buffer is OK, ring just has a variable beginning (rather than 0), the beginning just chases after the ending.   It's very tame coding in assembly & then you decode your commands.  The same in C, perhaps slightly slower.  Use real commands, not "A"  B" "C" ---that's for the birds.  The decoder is basically a dictionary lookup, with a few fancy features added.

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

Last Edited: Tue. Jan 21, 2020 - 09:36 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

MrKendo wrote:

The code required for ring buf is minimal eg. a common technique is all you need is a read and a write  index, which only need to be uint8_t,  you get the number of chars avaiable from write - read (requires that you always keep one space empty so you are wasting one byte of RAM, who cares) and then no interrupt blocking is needed.

It's not necessary to keep a space empty in the ring buffer. Let the read and write indexes grow larger than the size of the buffer (up to twice as big), and then mask each index only when reading from or writing to the buffer, rather than keeping the indexes themselves masked.

 

Here's a quick (untested) example:

typedef uint8_t index_t; // change this to uint16_t if bufsize >= 256

const index_t bufsize = 16; // must be a power of 2
uint8_t buf[bufsize];
index_t ridx, widx;

// Return how many bytes are in the buffer, between 0 and bufsize (inclusive).
index_t size(void) { return widx - ridx; }

// Add a byte to the buffer. Return false if the byte cannot be added because the buffer is full.
bool put(uint8_t c) {
    if (size() == bufsize) return false; // full
    buf[widx++ % bufsize] = c;
    return true; // success
}

// Return the next byte and remove it from the buffer, or -1 if the buffer is empty.
int get(void) {
    if (size() == 0) return -1; // empty
    return buf[ridx++ % bufsize];
}

// Return the next byte without removing it from the buffer, or -1 if the buffer is empty.
int peek(void) {
    if (size() == 0) return -1; // empty
    return buf[ridx % bufsize];
}
Last Edited: Wed. Jan 22, 2020 - 05:01 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

christop wrote:

Here's a quick (untested) example:

Lacks atomic accesses.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

joeymorin wrote:

christop wrote:

Here's a quick (untested) example:

Lacks atomic accesses.

Of course. Think of it as pseudo-code for the purpose of illustrating the concept of using larger indexes, rather than actual working code. I left out details like atomic access for simplicity and clarity.

 

(I think the only thing that needs to change is to add a "volatile" keyword before "index_t ridx, widx;". If index_t is changed to uint16_t, the index variables also must be read atomically in the "size" function and modified atomically in the "put" and "get" functions. Though if I were to write an actual ring buffer I'd also stuff it in a class, but as it's just a simple example I didn't bother to do that.)

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Go on.   ISR()s are like bootloaders.    They must be 100% perfect.

 

There are plenty of proven examples.   And plenty of textbooks.

 

It seems wiser to start with a functional round wheel than invent your own square wheel.

You will make swifter progress by making small improvements to the circular wheel than developing one that does not even rotate.

 

Things like Circular Buffers,  FIFO, LIFO, ... have been well known since the beginning of Computer Science in the 1950s.

 

David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

christop wrote:
It's not necessary to keep a space empty in the ring buffer. Let the read and write indexes grow larger than the size of the buffer (up to twice as big), and then mask each index only when reading from or writing to the buffer, rather than keeping the indexes themselves masked.

 

Ah yes, that's a neat way of doing it. I've seen that before somewhere.

Let the indices wrap at some wider value eg. just wrap modulo 256 as a uint8_t, and do the modulo buffer_size only when accessing the buffer.

Then as long as 256 is an exact multiple of your buffer size it works.

So this definitely restricts buffer size to a power of 2 but that's rarely an issue.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

MrKendo wrote:

So this definitely restricts buffer size to a power of 2 but that's rarely an issue.

Yes and no.  A non-power-of-two size is still possible, with bounds checking.  The bound for the buffer accesses is the buffer size.  The bound for the index is some integer multiple (greater than 1) of the buffer size.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It is not worth the trouble to do power of 2. For less than a handful of extra instructions (usually less than imagined) you can use any size buffer you want.

 

Restricting yourself to a power of 2 buffer, means your buffer sizes take large steps when you get into greater than single digit sizes and you may be forced to use a larger buffer than you needed (you needed 40 but have to use 64). You also have to enforce power of 2 which is another headache. There is mostly no need for power of 2 which came from a time when you had to count instructions and shave off bytes in your small code space. It seems to live on, but not quite sure why. I guess anything that looks clever is thought to be a good thing.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

joeymorin wrote:

MrKendo wrote:

 

So this definitely restricts buffer size to a power of 2 but that's rarely an issue.

 

Yes and no.  A non-power-of-two size is still possible, with bounds checking.  The bound for the buffer accesses is the buffer size.  The bound for the index is some integer multiple (greater than 1) of the buffer size.

OK, I was thinking specifically of where the bound for the index is simply when the unsigned type wraps, so modulo 256 for uint8_t.

Since 256 is a power of 2, the thing that it's an exact multiple of would have to be a power of 2.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 

It is not worth the trouble to do power of 2.

Power of 2 is not necessary; however, it is easier, shorter, and faster, since you just allow (or force) a wraparound over a certain number of bits.  Let a byte rollover, or mask off to the desired quantity (ex: 5 bits for 32 locations). 

 

There is mostly no need for power of 2 which came from a time when you had to count instructions and shave off bytes in your small code space.

True!   

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hmmm....Good stuff!

 

I think I am going to go with the ring and see how it works out.  For something this simple I can just kick a flag in the ISR when it sees the \r and the MAIN then can tickle the read index to get the command and go from there.

 

Thanks for the ideas and thoughts.  Keep it going!

 

JIm

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

jgmdesign wrote:

I think I am going to go with the ring and see how it works out.  For something this simple I can just kick a flag in the ISR when it sees the \r and the MAIN then can tickle the read index to get the command and go from there.

 

 

Probably not a good idea. The circular buffer is mainly for decoupling the incoming data rate from the rate in which you read the data. Let the circular buffer do its job then have your main line code extract the chars from it and form a command string. You then process the command string. Rinse and repeat.

 

I'll use a flat buffer for protocols - especially half duplex. The isr code manages the low level packet assembly and sets a flag when it has a complete packet. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
Let the circular buffer do its job then have your main line code extract the chars from it and form a command string. You then process the command string. Rinse and repeat.

 

THis is where I am getting confused.  the data coming in is not a continuous stream.  Its between 10 seconds and 60 seconds between commands.  The rest of the time the device is just sitting there.  So setting a flag and simply increasing the read index as I extract the command until it rolls over and the index self-resets should not be a big issue.  Its not like I have a lot going on.

 

Jim

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It’s an architectural decision. There are many ways you can skin a cat - some are more effective than others.
So whilst you can do what you’ve suggested, convention would suggest you keep the circular buffer as a circular buffer and not with added magic. Besides, at some point you need to empty the buffer - what happens if the buffer overflows then you get an end of line? What you retrieve may not be what you want.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Besides, at some point you need to empty the buffer -

You can set it so that if it is full (the head has come around & hit the tail-1), nothing new is accepted.  This must happen anyhow, since it means the incoming has been happening faster than the processing (either due to a sudden blitz of commands, or just slow response in general). 

 

the data coming in is not a continuous stream.  Its between 10 seconds and 60 seconds between commands. 

If so, a linear buffer will prob be just as good (unless you take 2 minutes to process a command!).    The circ buffer allows you to queue up 2 ,3, 4 or more commands which might be sent out in a sudden burst, or pile up due to some critical process that must momentarily remain uninterrupted. 

 

The circ buffer only requires a handful of lines of code so you can always put one in place quick & easy later.   Instead of a CR flag, use a CR counter to count how many commands are pending in the buffer & count down as they are processed. 

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If it really is simple and a single command (?), you can do it all in the isr-

https://godbolt.org/z/hDrmHt

Start adding more commands, and it soon becomes a bad idea.

 

Probably the main advantage to something real simple, is you can see in a few lines of code all the possibilities, and convince yourself you have left no way to mess up- continuous stream of bytes incoming- check, framing/overrun errors- check, upper/lowercase combos- check (if wanted). If the requests come in faster than the data is sent, no big deal as the isr only sets a flag. You just have a little state machine in disguise.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It might be that in your case you don't really need a circular buffer, but the point is, it won't hurt unless you realy need to optiimise things down to absolute minimum of code and RAM.

If you have a 'ready made' circular bufer implementation then why not use it.

Assuming the existence of circular buffer and a couple of functions, one called uart_rx_available (or whatever) which returns the number of unread bytes sitting in the buffer, and a uart_getchar (or whatever) which returns the next unread byte, I would use a pattern something like the following.

 

#define CMD_BUF_SIZE 20
/* +1 so there's always room to stick a null on the end */
static char cmd_buf[CMD_BUF_SIZE + 1];
static uint8_t cmd_index;

static void process_command(const char *buf)
{
    /* ignore if buf empty eg. if you send "abc\r\n", gets called once
     * with "abc\0" when the \r is received, then gets called again with
     * "\0" when the \n is received. */
    if (buf[0] == '\0')
    {
        return;
    }

    if (strcmp(buf, "STATUS") == 0)
    {
        /* received a status comand */
    }
    /* etc. */
}

int main(void)
{
    /* init */

    while (1)
    {
        while (uart_rx_available())
        {
            char c = uart_getchar();
            if (c == '\r' || c == '\n')
            {
                /* so cmd_buf can be treated as null terminated string */
                cmd_buf[cmd_index] = '\0';
                process_command(cmd_buf);
                cmd_index = 0;
            }
            else if (cmd_index < CMD_BUF_SIZE)
            {
                cmd_buf[cmd_index++] = c;
            }
        }
    }
    return 0;
}

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

 

 

avrcandies wrote:
nothing new is accepted.  This must happen anyhow,
Well actually there are two strategies you can operate. If the buffer really does fill completely and further characters start to arrive before the current ones are consumed you have two options:

 

1) do as avrcandies says and simply reject the arriving characters

 

or

 

2) start to discard the oldest ones held to make room for the new ones arriving.

 

In either case some data is lost. You have to decide what it is that is lost. If you are hitting this scenario (the foreground code does not get back to consume inbound characters before the buffer fills) then it almost certainly means either

 

a) the buffer needs to be bigger

 

or

 

b) the foreground processing code needs to be rescheduled so it gets more opportunity to process buffered characters so the buffer cannot never become totally full.

 

jgmdesign wrote:
For something this simple I can just kick a flag in the ISR when it sees the \r and the MAIN then can tickle the read index
which suggests you haven't quite grasped the advice above. The fact is that you don't have "watch for specific character code" in the ISR and you never "tickle the index". In fact your consumer code shouldn't even know where the buffer is or have access to the read/write pointers at all (something that's easily achieved in C++!). Your only external knowledge of the buffer should be APIs provided for it such as:

 

buffer.WriteCharIn()

buffer.ReadCharOut()

buffer.checkIsFull()

 

there may also be other helper interfaces such as

 

buffer.getCountInBuffer()

 

I always think that Dean's implementation is about the right level of complexity - the right amount of functions but simple/quick/efficient code behind the scenes:

 

http://fourwalledcubicle.com/files/LUFA/Doc/151115/html/group___group___ring_buff.html

 

 

it all lives in a single .h file (yup, just a .h with inlinable code):

 

http://www.fourwalledcubicle.com/files/LightweightRingBuff.h

 

I'm guessing the ring buffer that is auto-generated by Codevision presumably offers something quite similar? In use most of the time (in the above example code) you probably only ever use 3 functions: RingBuffer_Insert() in the ISR to put characters in. RingBuffer_GetCount() to check whether there's anything available at the point you want to read data and RingBuffer_Remove() to actually pick up the buffered characters. Only after you have retrieved them would you then do things like testing to see if the one you just got is '\n' if that suggests "action now needed" on the previously retrieved characters (that you hold in a foreground array).

 

EDIT: OK so I see that CV provides:

 

 

To be honest I think it exposes too much information here about the internal working of the buffers. However in the absence of some kind of isAvailable() test I guess you could use rx_counter to check whether there's anything ready to pick up if you don't want to wait. After that it seems your main interface to the buffer are going to be using the stdio.h standard functions getchar(), gets(), scanf(). In fact perhaps you just want to use gets() and be done with it?

 

Last Edited: Thu. Jan 23, 2020 - 10:43 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Since you have scanf/printf available, you could maybe just use them and forget about all buffer things-

 

https://godbolt.org/z/GDbBFs

 

scanf can be a strange cat, so may have to verify some things like it actually is adhering to the string length restriction. Some testing would verify- like a whole page of text thrown at it with a few valid commands inside. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The fact is that you don't have "watch for specific character code" in the ISR

I tend to do a little of both.  Watching for a CR is good if you need almost the fastest possible reaction time (or to flag the main priority so it doesn't decide to get busy computing pi to 300 digits when there is a command to work on).

However, typically waiting a ms or two is no big deal.  In some programs, I just occasionally scan the buffer from main (maybe 25 times a sec), looking to see if there are any CR's present.  

 

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

This discussion highlights the change in the approach to embedded coding. Back in the asm days, the code was highly optimised to save rom and ram space. This got translated in C. As time has moved on, we have microcontrollers with a lot more performance, flash and ram, so the general approach has shifted to writing more generic code, code that is testable and is better structured. 

 

Generally it was electronic engineers that did the embedded code as they understood the hardware. In the meantime, software engineering was progressing and the electronic guys were still making the same old mistakes. Nowadays, many universities have double degrees in electronic and software engineering - this is driving the new generation of embedded system people. I would implore all to have a read of 'Code Complete' - especially the early versions.

 

Whilst the skills in crafting tight, fast code are still useful, nowadays, if I'm having to count bytes and cycles, then I'd be questioning the design choices - having said that, I've applied for a job where they bit bash a number of protocols, so the requirement is for someone who understands hardware, bytes and cycles! Personally, I would've leaned towards a small fpga - these are cheap then coupled that with a mid range micro vs a high end one to do the lot.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
Nowadays, many universities have double degrees in electronic and software engineering
I did that 39 years ago ;-)

 

(the course I did had the fancy title "Microelectronics and Microprocessor Engineering" but the previous year the exact same course (that did not get many takers using the old name) was just called "Computing and Electronics")

 

On completion I promptly forgot all the electronics bits!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

39 years ago you would have written an interrupt-driven UART handler.

Except that it was in ASM for Z80.   Or for 6502 with 6551.

 

You still thought it important that characters were never missed.   buffers never overflowed.   errors were flagged.

 

Whether you use or ASM or C the task is exactly the same.   And quite honestly,   the code size and performance is about the same too.

 

I presume that the first Teletypes on the first MiniComputers also had interrupt-driven UART.   They certainly had the same requirement i.e. functional serial.    i.e. this was all resolved before East Coast Jim was born.

Before 6551 or 16550 chips were made,   the "UART" would be constructed from discrete logic.

 

David.

Last Edited: Fri. Jan 24, 2020 - 09:50 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You make it sound like a Z80 had a UART , it don't

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You make it sound like a Z80 had a UART , it don't

I'm just glad it don't require one of them bipolar supplies like those other chips needed.

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

From mem

avrcandies wrote:

You make it sound like a Z80 had a UART , it don't

I'm just glad it don't require one of them bipolar supplies like those other chips needed.

From memory the Z80 family had a horrible peripheral chip called Z80SIO

 

The 6502 family had 6550 or 6551.

 

The early UART chips were not very good.

Hence Cliff's comments about 8250 and 16550

 

The first singe-chip MicroControllers had UARTs.   Which meant you could discard the external memory bus.

 

Some modern Cortex-M3 and most Cortex-M4 have on-chip UART buffers.    You only need a small buffer to make a dramatic improvement in performance.

 

David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

david.prentice wrote:

The first singe-chip MicroControllers had UARTs.   Which meant you could discard the external memory bus.

 

My first-ever use of an AVR was a 90S1200 used mostly as a UART (with a few party tricks) for a load of CPLDs.  S.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The 8051 had has a UART.

 

I have never used its predecessor ( ? 8049 )

I don't know whether that chip had a UART.

 

The Mitsubishi 740 chips had a UART.

 

The first PICs did not have UART.

The first AVRs did not have UART.  e.g. AT90S1200 did it in software

 

My first AVR was AT90S2313 which did have a UART.

 

David.

Last Edited: Sat. Jan 25, 2020 - 10:39 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

My first SBC designs were 8080 and 8085 based.

 

I rummaged through by basement and located this book, first published in 1980.

I have the 4th addition, published 2 years later, in 1982.

I build several Z80 boards based upon the included schematic.

The first chapter or two were building the board, the remainder of the book was projects, all coded in Asm.

 

JC

 

 

 

BTW, the original design didn't include a UART

 

JC

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

DocJC wrote:
My first SBC designs were 8080 and 8085 based.

 

8085!!  AHHHHH....the good old days!  I miss that part dearly.

 

 

JIm

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

8085!!  AHHHHH....the good old days!

Yes, the first 5V only micro.  I remember interfacing one to bubble memory for Ford...Bubble memory gonna take over & wipe out other types.

 

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

<cough> 6502 was available a year before the 8085 and was 5v only; it was preceded by the 6501 and 6800 and I think both of those were 5v only, too...

 

Neil

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 

<cough> 6502 was available a year before the 8085

Why, I think you are right, it appears it was the first 8080-style 5V part:   The "5" in the part number highlighted the fact that the 8085 uses a single +5-volt (V) power supply by using depletion-mode transistors, rather than requiring the +5 V, −5 V and +12 V supplies needed by the 8080.   I need to look more carefully at the subtitle\.   Look at those transistor counts!

 

 

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

Last Edited: Sat. Jan 25, 2020 - 10:03 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ring buffers are simple and very powerful

 

The RingBuff code is an excellent data structure.

 

There are lighter ways to make ring buffers.
Simple macros can do an excellent job.

 

#define RB_SIZE    (1<<8)                                                    // working with a power of two is convenient

#define RB_MASK   (RB_SIZE-1)

struct rb

{

    unsigned head;                                                                 // head and tail need to be in separately addressable memory elements

    unsigned tail;                                                                   // head and tail are controlled in coherently separate threads, like ISR and upper driver

    unsigned buf[RB_SIZE];                                                    // note buf can be characters, pointers or complex structures

}

 

#define RB_EMPTY(p)  (p->head == p->tail)                            // report is the ring buffer is empty

#define RB_COUNT(p) (p->head - p->tail)                               // return the number of elements in the ring buffer

#define RB_FREE(p) (RM_MASK - p->head - p->tail)                // return the number of elements in the ring buffer

#define RB_PUT(p, x)  p->buf[p->head++] = x; p->head &= RB_MASK

#define RB_GET(p, x)  x = p->buf[p->tail++]; p->tail &= RM_MASK

 

This covers a great many cases and is not type-dependent and is very lightweight, often presenting less overhead than a function call.

 

 

The size and type of element in the ring buffer needs to be tuned for the application.

In your case, there are bytes.  Looking at the code, make an estimate of the maximum number of bytes that would be expected in the ring buffer.

It could be the longest message on the egress side or the longest latency in byte times on the ingress side.

 

The ring buffer, FIFO in the ISR is generally small, maybe 16 bytes.  One side, ISR, gets head and the other side, application, tail.  Each side can manipulate it's index into the data according to the rules implemented in get and put.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

david.prentice wrote:

The first AVRs did not have UART.  e.g. AT90S1200 did it in software. My first AVR was AT90S2313 which did have a UART.

 

Being slightly pedantic here but the 90S1200 and 90S2313 (along with the S4414 and S8515) were the first AVRs. It was just the 1200 which omitted the UART.

#1 Hardware Problem? https://www.avrfreaks.net/forum/...

#2 Hardware Problem? Read AVR042.

#3 All grounds are not created equal

#4 Have you proved your chip is running at xxMHz?

#5 "If you think you need floating point to solve the problem then you don't understand the problem. If you really do need floating point then you have a problem you do not understand."

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I assumed that a Google "AVR history" or "microcontroller history" might give a timeline.   I did not find a good table.

MCS-48 1976 e.g. 8048

MCS-51 1980 e.g. 8051

740        1984 e.g. M50740

PIC        1993 e.g. PIC16C84

AVR       1997 e.g. AT90S1200

 

So AVR was fairly late to the game.    But obviously very successful.

 

1980 was a long time ago. But 8051 had an on-chip UART.   And it was "better" than the dedicated 8080, 6800, 6502 Serial Peripheral chips available in 1980.   (but obviously cost more)

 

David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Trivet wrote:
This covers a great many cases and is not type-dependent and is very lightweight, often presenting less overhead than a function call.
Dean's version is "static inline", there's no "CALL overhead".

 

Personally I would go with something like:

template<typename T, int size> class ringBuffer {
public:
    ringBuffer() : mWriteIdx(0), mReadIdx(0), mCount(0), mElements(size) {
        mBuff = new T[size];
    }

    ~ringBuffer() {
        delete(mBuff);
    }

    void write(T item) {
        mBuff[mWriteIdx] = item;
        mWriteIdx++;
        if (mWriteIdx == mElements) {
            mWriteIdx = 0;
        }
        mCount++;
        if (mCount > mElements) {
            // buffer is full so stop counting as it's simply overwriting old entries now
            mCount = mElements;
        }
    }

    T read(BOOL & OK) {
        T retItem;
        if (mCount == 0) {
            OK = false;
            return retItem;
        }
        retItem = mBuff[mReadIdx];
        mReadIdx++;
        if (mReadIdx == mElements) {
            mReadIdx = 0;
        }
        mCount--;
        OK = true;
        return retItem;
    }

    T peek() {
        return mBuff[mReadIdx];
    }

    int getNumItems() {
        return mCount;
    }

    bool isAvailable() {
        return (mCount > 0);
    }

    void reset() {
        mWriteIdx = 0;
        mReadIdx = 0;
        mCount = 0;
        memset(mBuff, 0, mElements * sizeof(T));
    }


private:
    T * mBuff;
    int mWriteIdx;
    int mReadIdx;
    int mCount;
    int mElements;
};

You use it like:

typedef struct {
    float f;
    long l;
    char c;
    short s;
} test_type;

private:
    ringBuffer<char, 30> testBuff;
    ringBuffer<test_type, 10> structBuff;
    test_type aStruct;
    testBuff.reset();
    testBuff.write('A');
    testBuff.write('Q');
    testBuff.write('Z');
    if (testBuff.getNumItems() > 2) {
        BOOL OK;
        (void)testBuff.read(OK);
    }

    structBuff.reset();
    aStruct.frameNum = 137;
    aStruct.c = 'W';
    aStruct.f = 3.14F;
    aStruct.l = 0xBABEFACE;
    aStruct.s = 1234;
    structBuff.write(aStruct);
    aStruct.frameNum = 139;
    aStruct.c = 'Y';
    aStruct.f = 2.71F;
    aStruct.l = 0xACEB1ADE;
    aStruct.s = 4321;
    structBuff.write(aStruct);

As shown it is in theory just as good at buffering complex struct types as it is for "char" or whatever.

 

Oh, it does need you to be using C++ not C though ;-)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0


david.prentice wrote:

AVR       1997 e.g. AT90S1200

 

#1 Hardware Problem? https://www.avrfreaks.net/forum/...

#2 Hardware Problem? Read AVR042.

#3 All grounds are not created equal

#4 Have you proved your chip is running at xxMHz?

#5 "If you think you need floating point to solve the problem then you don't understand the problem. If you really do need floating point then you have a problem you do not understand."

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I did not get an AVR until 2004 (AT90S2313)

I did not get an 8051 until 2004 (AT89S8252)

 

I am relatively late to the game.

 

It surprised me that there is 17 years between 8051 and AVR.

 

In Circuit Serial Programming for AVR Flash was the major feature.    Until then you had to have a OTP chips.

Or an expensive UV-erasable chip with a window.

 

8051 C compilers were pretty good.   But AVR was specifically designed for C compilers.

 

David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Atmel were ahead of the pack when it came to flash and micros. Their flash based 8051 devices were very popular. The AVR took it to a new level of performance along with adcs.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Trivet wrote:
#define RB_COUNT(p) (p->head - p->tail)                               // return the number of elements in the ring buffer

  It should be

#define RB_COUNT(p) (p->head - p->tail) & RB_MASK ?

 

  Having the pointers exposed allows you know where exactly your data will end up so you can use memcpy to write / read a whole string at once for example.

 

  For the case of this thread I would use two fixed arrays in a ping-pong manner.