LightWeightRingBuff.h

Go To Last Post
102 posts / 0 new

Pages

Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

http://www.fourwalledcubicle.com/files/LightweightRingBuff.h

Is it possible to add this header file to my project in order to use it ? Is it a correct way ?
The case is that I am not use to add a header file that is also implement the functions it declares within the header file.

Shouldn't I separate this file to a source - LightWeightRingBuff.c and to a header - LightWeightRingBuff.h ?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Just put it in the same directory as your other .c and .h files in the project and then in the one that is to use use it just:

#include "LightWeightRingBuff.h"

That's all you have to do.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yes, I did it and it's working. My question is : Is it a correct way of programming ? Shouldn't I need to separate it to header and source file inspite of pushing everything in the header ?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Each and every function is "static inline" : they are defined -and what they are supposed to do is in the header file (not in a c file : in this case, they would have been  extern ). -> from reading the source, I bet it is sufficient (and this header file  is short enough not to be cutted into smaller places). Edited : you wrote it works...

Having "static inline " functions is a better way than preprocessor macros , but it is almost the same functionality (with less annoying things) .

They can be called **only** from a .c file, according to http://www.greenend.org.uk/rjk/t... (the only way to achieve it is to put them into ... a .h file or to cut and paste them, which would be worse more dangerous than preprocessor macros).

Last Edited: Tue. Oct 13, 2015 - 10:48 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

No, it was a choice on Dean's part but he's chosen to implement it as .h only. This is because all the code is in "static inline" functions. Just accept his choice, #include the file and invoke the functions.

 

The difference here (compared to "normal" C code) is that his functions will not be CALL'd. Instead, at each place where you "invoke" a function the body of it will be placed inline into your own code with no CALL/RET overhead. This works because the body of each of his functions are relatively small and it's also unlikely that you will be invoking any of the functions many times (in which case a single, call-able copy would have made more sense).

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ok thanks!

another question :

Is there any usage example ( a code) of using those functions in this header file as part of a usart driven interrupt ? Can someone give me an useful link ?

for filling the FIFO in the ISR and consuming the ring buffer in the Main

 

thanks

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

For consuming, you look whether there are data to eat and .... eat a character

char getChar(RingBuff_t* const myBuffer);

    if ( RingBuffer_IsEmpty( myBuffer) 
        return -1 ; // arduino way

    else // is not that necessary 
        return RingBuffer_Remove(myBuffer); // uint8_ should be converted to chars, your serial line in another thread never receives -1/255

}

 

Last Edited: Tue. Oct 13, 2015 - 11:09 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

OK, thats "eating" is right after the ISR that is filling the FIFO. How do I fill the FIFO ?

 

I'v wrote this code for the filling :

 

//Interrupt Service Routine (ISR) - only serviced whenever the USART hardware module receives a byte of data in the rx pin
ISR(USART1_RX_vect)
{
//code to be executed when the USART receives a command string here

	while (!RingBuffer_IsFull(&buf))
	{
		RingBuffer_Insert(&buf,c);
	}
}

but I'm not getting in the ring buffer the full response. 

for example, if I send AT\r\n, I need to receive \r\n\nOK but I get just the \r\n\n\n\n\n\n\n....

why's that ?! does anyone has a clue ?

Last Edited: Tue. Oct 13, 2015 - 12:28 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Is there any usage example ( a code) of using those functions in this header file as part of a usart driven interrupt ? Can someone give me an useful link ?

I take it you have read the user manual:

 

http://www.fourwalledcubicle.com...

 

Did the example there not show you everything you need to know?

 

The fact is that you are going to have a uart.c that contains both interrupt and non-interrupt code. Globally within the file you will have something like:

RingBuffer_t Buffer;
uint8_t      BufferData[128];

then in the UART_init() or similar you will make a call to:

RingBuffer_InitBuffer(&Buffer, BufferData, sizeof(BufferData));

thta same routine will configure the UART and arrange to get RXC interrupts or whatever. Back in main() after the call to UART_init() you will have an sei(). From that point onwards if a character is received it will generate an RXC interrupt. So back in the uart.c you will have a handler along the lines of:

ISR(UART_RXC_vect) {
    RingBuffer_Insert(&Buffer, UDR);
}

the same file will also have some kind of:

unsigned char UART_getchar(void) {
    return RingBuffer_Remove(&Buffer);
}

and that is the basics of the whole thing. However it is not quite THAT simple you have to concern yourself with what happens when you are inserting characters in the interrupt and there is no room. This is where the functions like GetFreeCount() come into their own. If you are in the ISR() and you have received a character in UDR but RingBuffer_GetFreeCount() says that there is no free space you have a dilemma. The only thing you can really do is lose the character. There's no point waiting in the ISR() for space in the buffer to be freed because nothing else can occur when inside the ISR so you will just have to say something like:

ISR(UART_RXC_vect) {
    if (RingBuffer_GetFreeCount(&Buffer) > 0) {
        RingBuffer_Insert(&Buffer, UDR);
    }
    else {
        unsigned dummy_read = UDR;
    }
}

In fact it's not as simple as this either. The fact is that the received character might have some error like "Framing error" or "Data over run error" so you can't just read UDR you should also check those flags too and only insert valid characters into the ringbuffer, discard anything with error but you might want to have some way to report to the foreground that characters with errors have been received.

 

Similarly there is the code that extracts from the buffer. You cannot really just return RingBuffer_Remove() - suppose there isn't actually anything in the buffer when that function is called. So you need to check for that with something like:

unsigned char UART_getchar(void) {
    while(RingBuffer_IsEmpty(&Buffer)); // wait until we have something to return
    return RingBuffer_Remove(&Buffer);
}

Now this function becomes "blocking". It will be "stuck" inside this function until a character eventually arrives. You may not always want this so you may also need to implement something like:

bool UART_CharacterAvailable(void) {
    return !RingBuffer_IsEmpty(&Buffer);
}

The alternative is to do what dbrion has showed - you return an error code (-1) if getchar() is called and nothing is available, otherwise you return the next Remove() character.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Your ISR should be very short (I would avoid "real" -not inlined-  function calls "get_char_uart" instead of c=UDR, loops...)

 

you might look whether there is room in the buffer (via RingBuffer_IsFull) , and then you put the received char in the buffer (via RingBuffer_Insert).

 

If your buffer is full, maybe you should light a LED (it means you are very unlucky, your buffer is too short, your clock too slow)... or do nothing...

 

"eating/consuming" does not occur **after** data have been produced, but when there are enough data and when main function has time.... (maybe you / your main should check wheter you have received more than xxx chars, without reading them, via RingBuffer_GetCount )

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:

However it is not quite THAT simple you have to concern yourself with what happens when you are inserting characters in the interrupt and there is no room. This is where the functions like GetFreeCount() come into their own. If you are in the ISR() and you have received a character in UDR but RingBuffer_GetFreeCount() says that there is no free space you have a dilemma. The only thing you can really do is lose the character. There's no point waiting in the ISR() for space in the buffer to be freed because nothing else can occur when inside the ISR so you will just have to say something like:

ISR(UART_RXC_vect) {
    if (RingBuffer_GetFreeCount(&Buffer) > 0) {
        RingBuffer_Insert(&Buffer, UDR);
    }
    else {
        unsigned dummy_read = UDR;
    }
}

 

 

I will be glad if you would elaborate a little about the yellow markered line. 

what is this dummy_read variable ? why not to enlarge my buffer so there won't be any chance for a full buffer in any time it would be ?

 

Last Edited: Sun. Oct 18, 2015 - 09:29 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

kobidon wrote:

why not to enlarge my buffer so there won't be any chance for a full buffer in any time it would be ?

 

You can do that for sure. But how do you know when it's big enough?

#1 This forum helps those that help themselves

#2 All grounds are not created equal

#3 How have you proved that your chip is running at xxMHz?

#4 "If you think you need floating point to solve the problem then you don't understand the problem. If you really do need floating point then you have a problem you do not understand." - Heater's ex-boss

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Brian Fairchild wrote:

 

You can do that for sure. But how do you know when it's big enough?

 

I thought that I could estimate averagely the size of the incoming messages but now I am not so sure.

So can you explain me what is this dummy_read variable ? How and where can I store the prone to be lost characters so I won't eventually loose them ?

Last Edited: Sun. Oct 18, 2015 - 09:18 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I thought that I could estimate averagely the size of the incoming messages but now I am not so sure.

 

Perhaps. But "average" is disjunct from "maximum". It often pays off to code defensively. Especially when the effect of not doing so, should the extreme case actually happen, the results are catastrophic.

 

Cliff simply coded defensively. He's been around long enough to know that you stand a better chance of surviving if you expect the worst from everyone else, but behave as good as you possibly can yourself.

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

Last Edited: Sun. Oct 18, 2015 - 09:45 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

JohanEkdahl wrote:

 

Perhaps. But "average" is disjunct from "maximum". It often pays off to code defensively. Especially when the effect of not doing so, should the extreme case actually happen, the results are catastrophic.

 

Cliff simply coded defensively. He's been around long enough to know that you stand a better chance of surviving if you expect the worst from everyone else, but behave as good as you possibly can yourself.

 

You'r right! So I will take the worst case.

What about that variable ? I didn't understand how he is saving the situation of loosing characters

 

Last Edited: Sun. Oct 18, 2015 - 09:54 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

He's just saving the integrity of the ring buffer by not overrunning it. And the character has to be pulled out of the USART. He simply throws it away.

 

On a higher level you will have to guard the integrity of the message protocol. Cliff's solution for not crashing the buffer does nothing about that. Regardless of if the ring buffer gets thrashed or not there will be a problem on a higher level with e.g. bogus messages (missing characters, extra characters, pure static/garbage etc).

 

In general, each problem should be dealt with separately on the appropriate level. The message protocol has (or should have) content to recover from garbage and synchronization errors. E.g. a clearly defined way of determining start and/or end of a message (a simple thing would be a newline). If you can not afford to lose any message you have to build in provisions for acknowledgement and/or re-sending messages.

 

Depending on the situation this can be (fairly) trivial to quite complex. Need high throughput? Have loooong messages? Etc...

 

Studying how e.g. TCP/IP has divided the responsibilities into different layers can be enlightening. Don't try do dig too deep into the details for TCP/IP in this matter, they will not help you. IP e.g. does routing, not very applicable for a point-to-point connection like "serial"/UART/RS-232. It is the principle that IP does the low(ish) level transfer or relatively raw data, and that TCP does the "all in good order - nothing lost" bit that is important.

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm encountering a weird problem

I try to find a string in the ring buffer with strstr() function but unfortunately I'm not succeeding. Can anybody help me find my code mistake ?

 

I declared a pointer to the ring buffer that way :

 

volatile RingBuff_t *b=&buf;

then I'm asking :

if (strstr((char*)b->Buffer,"OK") == NULL)

 but I always get null eventhough I see in the watch windows the ascii numbers 79 and 75 one after the other which represent the word "OK"

 

what am I doing wrong ?

 

P.S this is the declaration of the strstr function located in string.h header file : http://www.tutorialspoint.com/ansi_c/c_strstr.htm

 

thanks!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It's not a ring buffer if you access the buffer directly. You need to 'pull' the chars out and form a null terminated string. Then do the comparison.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

kobidon wrote:
then I'm asking : if (strstr((char*)b->Buffer,"OK") == NULL)

You can't do that with a ring buffer. Remember that, conceptually, "the end of the buffer is connected to the front of it". That what makes it a "ring".

 

It might well be that, after the ring buffer has been used, you can have a string "Kobidon" represented like this in the ring buffer:

 


+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
| i | d | o | n |   |   |   |   |   |   |   |   |   | K | o | b |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
                  ^                                   ^
                  |                                   |
                  in                                  out

 

As you can see, it will not be possible to do a simple strstr() on that, regardless of if you pass it the beginning of the physical buffer, or if you pass it the out-pointer.

 

What you need to do is read up on how a ring buffer actually works. IIRC the Wikipedia article is excellent.

 

Then you can write a function, to complement Deans ring-buffer-implementation, that pulls out a copy of the current contents of the ring buffer into a "normal string" and then do the strstr using that. Since the strstr might come out with "no match" I assume you don't want to "consume" the contents of the ring buffer. Thus your complementing function really must do a COPY of the contents. That means you can not use RingBuffer_Remove() to get the contents into your ordinary string.

 

You might want to write a RingBuffer_Peek() to copy (without "consuming") a character from the ring buffer. That could then be used to build a copy of the ring-buffer contents ass an ordinary string. But since such a function can not touch the Out pointer but still needs "state" (i.e. the next call to RingBuffer_Peek() should get the next-next character etc) you need to have a copy of the Out pointer also - holding the state of RingBuffer_Peek. And you need a way to reset this copy of the Out pointer before the next time you want to do strstr. No, it's not trivial. OTOH, it's not that complicated either.

 

If that was over your head, then you really need to understand how the ring buffer works before proceeding.

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

JohanEkdahl wrote:

What you need to do is read up on how a ring buffer actually works. IIRC the Wikipedia article is excellent.

 

Then you can write a function, to complement Deans ring-buffer-implementation, that pulls out a copy of the current contents of the ring buffer into a "normal string" and then do the strstr using that. Since the strstr might come out with "no match" I assume you don't want to "consume" the contents of the ring buffer. Thus your complementing function really must do a COPY of the contents. That means you can not use RingBuffer_Remove() to get the contents into your ordinary string.

 

 

 

1. I'v already read it but I will sharpen my knowledge in this subject by reading it once again slowly. 

 

Anyway, I implemented this function :

 

void ReadString(char *BufferData)
{
        int n=0;

	while (RingBuffer_GetCount(&buf)>0)
			{
				BufferData[n]=UART_getchar();
				n++;
			}
}

which consumes the contents of the ring buffer (I don't know why it's not good, I don't care consuming its content cause I just wait and need to check if an 'OK' response received and I don't need the whole content of the ring buffer).

 

2. 

JohanEkdahl wrote:

 

You might want to write a RingBuffer_Peek() to copy (without "consuming") a character from the ring buffer. That could then be used to build a copy of the ring-buffer contents ass an ordinary string. But since such a function can not touch the Out pointer but still needs "state" (i.e. the next call to RingBuffer_Peek() should get the next-next character etc) you need to have a copy of the Out pointer also - holding the state of RingBuffer_Peek. And you need a way to reset this copy of the Out pointer before the next time you want to do strstr. No, it's not trivial. OTOH, it's not that complicated either.

 

If that was over your head, then you really need to understand how the ring buffer works before proceeding.

 

An initially try :

static inline RingBuff_Data_t RingBuffer_Peek(RingBuff_t* const Buffer)
{
	RingBuff_Data_t* Out1 = Buffer->Out;
	RingBuff_Data_t Data = *Buffer->Out1;
	RingBuff_Data_t* Out1 = Buffer->Out++;

	return Data;
}

 

3. I will be glad to understand what this lines under RingBuffer_Remove() means :

 

    if (++Buffer->Out == &Buffer->Buffer[BUFFER_SIZE])
              Buffer->Out = Buffer->Buffer;

 

I thought maybe -  if the address of next cell (of the data currently reading occured from) equals the address of the cell of the end of the array of data then...

 

 

 

Last Edited: Sun. Oct 25, 2015 - 02:17 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

kobidon wrote:
I will be glad to understand what this lines under RingBuffer_Remove() means

That is the code that advances the Out pointer (++Buffer->Out) , and handles the wrap-around if the end of the linear buffer has been reached (the if ... tests if we've reached the end of the linear buffer, and the  Buffer->Out = Buffer->Buffer sets the Out pointer back to the beginning of the linear buffer if it has).

 

The confusing thing with that code is that both the variable for the whole ring-buffer thing (the struct) and a member in that struct (the linear buffer, an array) is named Buffer.

 

If the struct was called RingBuffer and the linear array that is it's member was called LinearBuffer it would go

 


    if (++RingBuffer->Out == &RingBuffer->LinearBuffer[BUFFER_SIZE])
              RingBuffer->Out = RingBuffer->LinearBuffer;

Makes more sense now?

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

Last Edited: Sun. Oct 25, 2015 - 04:24 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yes, It's pretty clear now. thanks!

But I want to ask a question. I'm testing my code and I see a strange thing I cant explain.

I am concatenating a string with this command : 

 

sprintf(ConnectionString, "%s%s%s%s%s","AT+CWJAP=",ssid,",",pwd,"\r\n");

 

the string should be sent like this : AT+CWJAP="ssid","pwd"\r\n

 

then I send through UART the ConnectionString string with this command : 

 

c_printf_uart(ConnectionString);

 

the response I expect to be is : AT+CWJAP="ssid","pwd"\r\n\r\r\nOK

 

but if I check the buf.buffer field its not giving me this response but AT+CWJAP="ssid","pwd"\r\n\r\r\n WITHOUT THE OK 

 

my ISR code is : 

 

//Interrupt Service Routine (ISR) - only serviced whenever the USART hardware module receives a byte of data in the rx pin
ISR(USART1_RX_vect)
{
    //code to be executed when the rx pin of the USART receives a char
    RingBuffer_Insert(&buf,UDR_N);
}

and I'm declaring on : volatile RingBuff_t buf where I implement the ISR.

 

why's that ?

Last Edited: Mon. Oct 26, 2015 - 12:25 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I didn't understand why not to really consume the data from the ring buffer into the new linear buffer ? 

Anyway, I just looking for the OK response and all the other data is not relevant to me.

Last Edited: Tue. Oct 27, 2015 - 09:05 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

How long do you wait for the OK response?
Have you looked at Arduino examples to see what they do?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Cliff simply coded defensively. He's been around long enough to know that you stand a better chance of surviving if you expect the worst from everyone else, but behave as good as you possibly can yourself.

Hello. I'm back from my short holiday (maiden voyage of the Norwegian Eclipse from Hamburg to Southampton ;-) so let me say a bit about the yellow bit in:

ISR(UART_RXC_vect) {
    if (RingBuffer_GetFreeCount(&Buffer) > 0) {
        RingBuffer_Insert(&Buffer, UDR);
    }
    else {
        unsigned dummy_read = UDR;
    }
}

At first I was just writing the ISR example to illustrate the need/use for GetFreeCount(). But then I looked to the actual implementation of Insert() (isn't life easier with namespace:: rather than having to prefix everything with "Ringbuffer_"?). In Insert() Dean does this:

 

		static inline void RingBuffer_Insert(RingBuffer_t* Buffer, const uint8_t Data)
		{
			GCC_FORCE_POINTER_ACCESS(Buffer);

			*Buffer->In = Data;

			if (++Buffer->In == Buffer->End)
			  Buffer->In = Buffer->Start;

			uint_reg_t CurrentGlobalInt = GetGlobalInterruptMask();
			GlobalInterruptDisable();

			Buffer->Count++;

			SetGlobalInterruptMask(CurrentGlobalInt);
		}

That will always insert a character (with wrapping of the pointer if required) regardless of whether the buffer is "full" or not. So my code was an attempt to mitigate any problem this might cause. Now when doing buffering you always face a dilemma of how to handle the over-run situation. You have a choice to make - either you discard the oldest character you received and use its place in the buffer (just go ahead and Insert() without checking) or you throw away the recently arrived character on the basis that you might want to process the older stuff instead. I chose the latter (and it illustrated getFreeCount()). Now the AVR clears the RXC flag when UDR is read so that is the reason why I included a dummy read of UDR to simply discard the recently arrived character. If I didn't do the read the code would return form the ISR() but as RXC is still set, after one opcode it would re-enter the ISR() again.

 

When using buffers the idea is that you always size the buffer to be bigger than the "worst possible case". A buffer is just that, a holding place to allow you to "catch up" if something blocks the ability to handle received data for a while. On the one hand you have the unstoppable process of characters arriving and on the other you have your process code (in the foreground) that is consuning the arrived/buffered characters. You should estimate your buffer sizes so you know you will always finish processing long before the buffer completely fills up. If you employ some kind of data stalling process like Xon/Xoff or CTS/RTS you might monitor a "high water mark" in the buffer and if it gets near full you tell the other end to "shut up" to prevent the buffer filling up completely.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

JohanEkdahl wrote:

 

Then you can write a function, to complement Deans ring-buffer-implementation, that pulls out a copy of the current contents of the ring buffer into a "normal string" and then do the strstr using that. Since the strstr might come out with "no match" I assume you don't want to "consume" the contents of the ring buffer. Thus your complementing function really must do a COPY of the contents. That means you can not use RingBuffer_Remove() to get the contents into your ordinary string.

 

You might want to write a RingBuffer_Peek() to copy (without "consuming") a character from the ring buffer. That could then be used to build a copy of the ring-buffer contents ass an ordinary string. But since such a function can not touch the Out pointer but still needs "state" (i.e. the next call to RingBuffer_Peek() should get the next-next character etc) you need to have a copy of the Out pointer also - holding the state of RingBuffer_Peek. And you need a way to reset this copy of the Out pointer before the next time you want to do strstr. No, it's not trivial. OTOH, it's not that complicated either.

 

If that was over your head, then you really need to understand how the ring buffer works before proceeding.

 

If every time that I want to compare something with the current content of the ring-buffer I will have to copy it from the ring-buffer to a linear one and then do the comparison so what is the advantage of using a ring buffer ? lets put the received characters directly into a linear buffer instead of a ring-buffer, do the comparison with strstr(), and initiate it after every comparison (for example).... In my opinion it seems inefficient from a complexitya and run time point of view.

I would be glad to hear what do you think about that point.

 

For now, what I actually did is that I build a function int contain(char *str) which "travels" from the Out pointer of the ring-buffer to the In pointer (if necessary) and return 1 if the string sending to the function located in the ring-buffer and 0 otherwise. 

Here is it, and I will be happy to hear what do you think about the implementation :

 

int contain(char *str)
{
	char* curr=str;
	int seen=0; // 
	RingBuff_Data_t d;

	while ((buf.Out!=buf.In) && (seen != strlen(str)))
	{
		d=RingBuffer_Remove(&buf);
		if (d==*curr)
		{
			curr++;
		}
		else
		{
			curr=str;
			seen=0;
		}
	}
	if (seen == strlen(str))
		return 1;
	else
		return 0;
}

 

 

Last Edited: Thu. Nov 26, 2015 - 08:00 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The problem arises when e.g. you have a line-oriented communication protocol. In the example below I use <EOL> to denote a line ending character.

 

Suppose you have incoming data where every line is on the form of a command (one of a few well known strings) and optionally some additional data. Let's for the sake of having a good example assume that the commands are about doing stuff with a LCD display connected to your AVR. ;-)

 

The valid commands could e.g.

 

DISP text to display<EOL>

CLR<EOL>

GOTO line column<EOL>

 

The reception is asynchronous to the process of handling the data. That is, while you are still dealing with one chunk of data the next command might be on its way in.

 

One problem is that while you are still analyzing one command, complete with the <EOL> but still in the buffer the next command might be on it's way in. Where are you to place this? Not at the beginning of the linear buffer because there the command you're analyzing is still placed.

 

You could argue that two buffers could be used. But this falls as soon as the handling of data is so slow that a third command is on its way in while you're still dealing with the first.

 

The elegant solution is to imagine the buffer to be endless. You have two pointers into the endless buffer, one for the position where the next datum should do in and one for where the next datum to be extracted is.

 

Of-course, in reality we have a limited amount of RAM so the endless buffer is not practically feasible. We solve the practical problem by allocating a limited buffer and bend the end of it to meet the front of it.

 

In essence, the ring buffer is an elegant solution to a complex problem. With a well formed interface it will behave like the endless linear buffer. Yes, it might be slightly inefficient, adding the overhead of handling the wrap from the end to the front of the linear buffer it relies on. But this is a small cost to pay for getting something that is fairly generic, and will give you less complex code in other places and will give you much less maintenance problems in the long run. I.e. the primary optimization is in cost of code production.

 

Now, regarding the possible copying out from the ring buffer this might or might not be necessary. It depends on if any analysis done on the data is destructive or not. It depends on if you want to apply ordinary string handling functions - obviously you cant do that with data as it is physically arranged in the ring buffer.

 

Example: We have received a command to place the cursor at line 3 character 12. In a linear buffer it would look like below. In the figure @ denotes the null string termination character - you've replaced the <EOL> with that upon insertion into the buffer.

 

Index     0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
Contents  G  O  T  O     3     1  2  @  ?  ?  ?  ?  ?  ?  ?

 

You could pass this buffer to e.g. strtok() and everything would work out just fine.

 

In a ring buffer the current start of the contents are not guaranteed to be at index 0. The received command might be placed like below (using ^ and v for marking the in- and out-pointers positions):

 

Index     0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
Contents  1  2  @  ?  ?  ?  ?  ?  ?  ?  ?  G  O  T  O     3
                   ^                       v

 

Obviously you cant call strtok() using the out-pointer as the parameter for the string buffer it expects. Of-course strok() will run off the far edge of the physical array looking for the string terminator. strtok() does not know how to traverse a circular buffer. It knows nothing about the wrap necessary when the end of the buffer is reached. (Actually it does not even know about the end of the buffer at all. It just keeps on running until it hits a null string terminator...).

 

It will be the same for many other string-handling functions. The reasonable thing to do is to have one function that gets a complete line (if there is one) into a linear buffer, and then use that buffer for any standard string handling functions.

 

So... An ordinary ring buffer, and if the problem demands it you write one extra function called getLine() that extract a complete line (if there is one) into a linear buffer. You now have a beautiful abstraction of the problem of synchronous data handling and reception. It's isolated. It's clean. Once it works you will likely never need to fix it. Well written a change of the size of the circular buffer should not demand any other textual code changes apart from e.g. a simple change in a #define.

 

I don't have thee time right now to go back in this thread to study the details of your problem, and recall why I suggested the copying. You will need to think for yourself, perhaps helped by what I've written in this post. Keep this alive to the week-end and I might find the time.

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

Last Edited: Thu. Nov 26, 2015 - 09:49 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

then do the comparison so what is the advantage of using a ring buffer ?

When you programmed something like a PC (well, in the old days anyway, before USB) the UARTS were 16550's which included a 16 byte FIFO in the electronics so your software could just service the UART every now and again because if a few characters arrived before you got the chance to interact with it they would be buffered in the 16 byte buffer that the chip contained.

 

The AVR UARTs are not as advanced as this. They can only hold one character while the next one is being received and you have to pull that character out of UDR as soon as you can because, if you don't, the next character being received. Once it has fully arrived will over-write the one there. So you use a receive interrupt and a buffer in your AVR software to effectively implement what something like a 16550 had in the hardware.

 

THAT is why you use a ring buffer. You just set the buffer size to be large enough to give you enough time between foreground servicing. So if you think you can pull bytes from the buffer every few milliseconds you may only need 8/16 bytes of buffer. But if it could be 10's or 100's of ms between your opportunity to service the buffer then you would set it to be 32/64/128 bytes long (binary multiples make the wrap around easier!).

 

So don't think if it as "two buffers". The ring buffer is purely there to achieve in software something that is missing from the hardware - it's purely "buying time" - that's all.

 

Oh and another argument for both the ring buffer and a foreground buffer is when it wraps so the ring buffer contains "o world...hell". It's quite tricky in this to see that you just received "hello world" but the read pointer is positioned on the 'h' and if you now read the characters in turn to your own foreground buffer you will get "hello world". You can now strcmp(fore_buffer, "hello world") and get a positive result.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
in the old days anyway, before USB) the UARTS were 16550's

Oh, you youngsters! More or less my first close-to-the-metal programming was writing a driver for a MS-DOS program for 8250. It eventually evolved into the 16450 which in turn evolved into the 16550. I was on a PC (running MS-DOS 5 or 6 ... yay!) that actually had at least a 16450, but we had to be 8250-compatible.

 

IIRC the 8250 had one shift-in register and then the latch register. Basta. I'm not sure , but I think we ran it polled. OTOH, I suspect that this was the time when 9600 bps was relatively fast. (The PC was probably based on a DX2 processor running at 66 MHz...)

 

And when you tell young people that ... they won't believe you! ;-)

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yup, I remember that too.

 

Remember that I worked for Europe's leading manufacturer of IBM PC clones. The IBM PC used 8250s. The IBM PC AT moved on to 16550. The "AT"="Advanced Technology" presumably included the use of UARTs with buffers ;-)

 

But at least even an 8250 had CTS/RTS and DTR/DSR support - I remember when I first came to embedded micros like AVR and found that the UARTs didn't even have control line support it was a bit of a disappointment. Of course just like you use RXC+ringbuffer to implement the missing FIFO you can "fake" those things in software too if you need them.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I suppose I saw the 8250s RTS/CTS and  DTR/DSR support as a question of driving those lines with enough "ooomph" (be it voltage or current).

 

Was there any "coupled logic" in the 8250 coughing if you tried to transmit w/o CTS being asserted? Can't recall..

 

Somewhere in the attic, in a stash of papers and binders, I have the data sheet on the 8250. I won't dig for that copy, and I know what crap I will get for the first 100 hits Googling for "8250 data sheet"...

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

O.K, I understand it. thanks!

I would like to consult with you about a problem I encountered using Interrupt and a ring-buffer together.

I'm using this Interrupt code and like that I insert chars into the ring buffer :

//Interrupt Service Routine (ISR) - only serviced whenever the USART hardware module receives a byte of data in the rx pin
ISR(USART1_RX_vect)
{
	//code to be executed when the rx pin of the USART receives a char
	RingBuffer_Insert(&buf,UDR_N);
}

i.e , every time I get a char , I call the RingBuffer_Insert().

In addition, I'm searching after the 'OK' key word, repeatedly, by this function (and its destructive as you can see) :

int contain(char *str)
{
	char* curr=str;
	int seen=0;
	RingBuff_Data_t d;

	while ((buf.Out!=buf.In) && (seen != strlen(str)))
	{
		d=RingBuffer_Remove(&buf);
		if (d==*curr)
		{
			curr++;
			seen++;
		}
		else
		{
			curr=str;
			seen=0;
		}
	}
	if (seen == strlen(str))
		return 1;
	else
		return 0;
}

Unfortunately, I saw that the out pointer being incremented while the the interrupt occurs and in some cases I miss the OK word.

So I thought maybe to wait until the whole sentence being inserted into the Ring buffer and just after that the Contain() will take place and the out pointer will begin incremented (by the RingBuffer_Remove()). I'm affraid that the delay caused from this will make the program inefficient. 

How would you solve it ?

 

Last Edited: Sun. Nov 29, 2015 - 12:42 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think most people would just assemble the whole line (terminated by cr and/or lf) then test for keywords.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yes but by that I'm making a delay in the program and I don't want this (what is the point of using an interrupt if the program is waiting for the whole line to being received ? the program needs to continue working in the background and can't hold the program..  By that way, How can I wait for the whole sentence being inserted and then start to check and remove... ? )

The second solution I thought was to enlarge the Ring-Buffer size to a size much large than a typical sentence (like 100 for example)...

Last Edited: Sun. Nov 29, 2015 - 01:28 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm trying to remember the number of the uart we used to use with 8080s... Unsuccessfully trying to remember.

If you don't know my whole story, keep your mouth shut.

If you know my whole story, you're an accomplice. Keep your mouth shut. 

Last Edited: Sun. Nov 29, 2015 - 01:39 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Torby wrote:

I'm trying to remember the number of the uart we used to use with 8080s... Unsuccessfully trying to remember.


See posts #28, #29, & #30.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You must never let the two "threads of execution" access the ring buffer at the same time. The relevant case here is the case of the process pulling things out of the buffer being interrupted by the ISR when a new character comes in on the USART. The solution is to temporarily disallow interrupts to occur while the process of pulling out data is active. This means

 

a) only the actual part of code pulling data out of the buffer, and

 

b) that this must be as short and fast as possible so that you do not get into a data overrun condition on the USART.

 

This is anotheer argument for having a function that pulls data out of the ring buffer, w/o destroying it, into a linear buffer. Only during this pulling out do you disallow the interrupts. The analysis of the data in the linear buffer takes place with interrupts enabled.

 

An alternative is, as you say, to wait for a whole "sentence" - I interpret that as a complete line. You could take on the extra cost in the ISR to detect the EOL arriving, and raise a flag that signals that the ring buffer holds an entire line. The pulling out then can be destructive since you know that you will have a complete line in the linear buffer. Pulling from the ring buffer to the linear will still need to be done with interrupts disabled, so make it fast.

 

Your code does not make much sense to me, and I am not inclined to analyze it in detail. It seems to me that you are trying to program two levels of abstraction into one piece of code. I argue that the problem is easier solved, and the code becomes much more clear, if you separate this into the fitting levels of abstraction.

 

The ring buffer solves the problem of data arriving at the USART asynchronously. I suggest it could also detect the EOL being seen - and it would be quite handy to replace it with a null character upon insertion into the ring buffer (if the protocol situation will allow this - i.e. the null character isn't bearing any other meaning in the communication protocol).

 

Next you write the line extraction function, which becomes fairly trivial.

 

Finally your main loop waits for the signqal that a whole line has been see. Upon that condition you call the function that extracts the line into a linear buffer, and then do your analysis using that linear buffer. This analysis is now nicely decoupled from the problem of handling the reception of data on the USART which has two nice properties

 

a) it knows nothing about the physical reception, and it should not since that is at a lower level of abstraction. The code becomes clearer!

 

b) it can have reasonably long running time since it runs with interrupts enabled so there will be no data overrun condition occurring.

 

Very sketchy (playing with my previous example):

 

volatile int eolnSeen;

ISR(something) {
    char c = UDR;
    if (c == EOL) {
        c = null;
        eolsSeen++;
    }
    RingBuffer_Insert(&ringBuffer, c);
}

voi RingBuffer_GetLine(char * linearBuffer) {
    // Disable interrupts
    cli();
    // Get the line
    int i = 0;
    do {
        char c = RingBuffer_Extract(&ringBuffer);        
        linearBuffer[i] = c;
    } while (c);
    // Mark it as gotten
    elonsSeen--;
    // Enable interrupts
    sei();
}

int main() {
    char linearBuffer[16];
    / Init stuff...
    
    while (1) {
        if (eolnsSeen) {
            // At least one complete line in the ring buffer
            RingBuffer_GetLine(linearBuffer);
            // Analyze and handle
            if (strstr(linearBuffer, "CLR")) {
                // Handle CLR being received
            } else if (strstr(linearBuffer, "DISPLAY")) {
                // Handle data to be ddisplayed
            } else if (strstr(linearBuffer, "GOTO")) {
                // Handle cursor positioning
            }
        }
    }
}

Now, that is VERY sketchy. I have not passed the ring buffer around as a parameter - you might (or might not) want to do that. I would prefer to build the eolnsSeen into the ring buffer itself - but I'll leave that to you as an exercise ;-) I'm sure I did some other mistakes. But what I am trying to present is an approach, a way of thinking (in levels of abstraction), a way to make different levels of abstraction as independent of each other as possible.

 

Think about it: If you move on and change communications channle to e.g. USB, then you do need to replace the whole communications code, but you should not need to replace any of the "analysis and action" code. If you keep the USART communications channel for a different project (say, controlling a motor) then the USART comms will be identical but the command set ("FWD", "BACK", "STOP") will be different. You'd keep the USART communications layer but replace the "analysis and action" code.

 

kobidon wrote:
I'm affraid that the delay caused from this will make the program inefficient.

Others might not share my opinion, but I argue that you should never optimize until it is necessary. Do you know that it will be inefficient? No? Then don't optimize. (My escape hatch here is "this principle may never be an excuse for writing obviously inefficient code" ;-) , but the point is that if you don't know what is inefficient, and even if it is inefficient then it makes no sense optimizing it.

 

Optimizing is one of the two things that is hardest with developing software (the other one is debugging). It is hard exactly because it is hard to find out if something is inefficient, and if so what part of it is inefficient, and when you know that understand what you need to do about it.

 

Optimizing just because you fear it will be inefficient is like taking a drug before you've become ill. (And the paralell to my escape hatch above is: But do eat healthy, exercise, sleep and take your vitamins.)

 

That became quite an essay.. HTH!

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

(what is the point of using an interrupt if the program is waiting for the whole line to being received ?

Then I fear that you may not have followed my explanation of why we use interrupts/ring buffers above. It's not for "speed". It's simply to buffer characters that arrive (asynchronously!) so that nothing that's inbound gets lost. The interrupt/buffer is just a "holding tank" to make sure that if the code is busy doing other stuff when an unstoppable stream of characters heads in the direction of the AVR that it is able to receive and hold them until such time your processing is ready to work on what has arrived. This should have no effect on the actual processing time. Obviously you cannot detect you have reeceived the whole word "cheese" until the third 'e' has arrived so it makes little difference how quickly you process the preceding characters. Now you could operate a state machine watching for 'c' then 'h' and so on so that if you might receive "cheese" or "chess" you will know when the 4th character arrives. But you kind of have to ask yourself whether you need this complication and for the thing to be that responsive. Is it not enough to wait until you have seen '., ' ', '/n' or whatever at the end of the word to now trigger a complete test for "cheese" or "chess"?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Actually, the ring buffer serves two purposes IMO:

 

1) The purpose pointed out by Cliff (clawson): It is a buffer in it original meaning. It "distributes load over time", it "dampens sudden impacts" (of data in our case).

 

2) It serves as a temporary storing place until we have a meaningful amount of data to work with on a higher level. In my example above this amount is a complete line.

 

I concur with clawson on his important point: Interrupts have little or nothing to do with optimization. Interrupts is a construct/mechanism to cope with asynchronous (un-predictable in time) events. (That also explains it's name. When the asynchronous event occurs it interrupts the normal flow of execution.)

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Some questions/concerns regarding what you have wrote :

 

JohanEkdahl wrote:

 

An alternative is, as you say, to wait for a whole "sentence" - I interpret that as a complete line. You could take on the extra cost in the ISR to detect the EOL arriving, and raise a flag that signals that the ring buffer holds an entire line.

In my case every response ends with a '\r\n' (I guess that this is the EOL you meant) but the problem is that before that there are another \r\n sequences so I am afraid it won't detect the real EOL. An example for a typical response : blablabla\r\r\n\r\nOK\r\n

1. What is your advice for that ? How can I really detect the real EOL ?

2. The EOL is not a char but a string \r\n so I can't just compare it to the UDR (the incoming char)

 

JohanEkdahl wrote:

 

The pulling out then can be destructive since you know that you will have a complete line in the linear buffer. Pulling from the ring buffer to the linear will still need to be done with interrupts disabled, so make it fast. Your code does not make much sense to me, and I am not inclined to analyze it in detail. It seems to me that you are trying to program two levels of abstraction into one piece of code. I argue that the problem is easier solved, and the code becomes much more clear, if you separate this into the fitting levels of abstraction.

 

 3. I prefer not to use a linear buffer because I can get manage without it. My working method is to insert characters to the ring-buffer using the ISR and simultaneously check if there is a sequence of OK characters by using the Ring_buffer Remove() [ I'm moving along the ring-buffer from the Out pointer to the In pointer until I met the OK sequence].

So if you can stick to this way (using just the RingBuffer, for the beginning) I will be glad.

 

JohanEkdahl wrote:

Pulling from the ring buffer to the linear will still need to be done with interrupts disabled, so make it fast.

 

 

4. Also, what if when I turn-off the interrupt by the cli() ,you suggested, and right after that I get a character ? it will be lost.. Am I wrong ??

 

5. another last question : I know that an interrupt must occur as fast as possible so I'm afraid to check for the EOL inside the ISR. What does people mean when they say that an ISR should occur "as fast as possible" ? what are the limitations ? 

 

Last Edited: Mon. Nov 30, 2015 - 08:22 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

To extract a char from the ring buffer takes microseconds vs 1ms for a char at 9600 baud, thus it is unlikely you will lose a char by temporarily disabling interrupts.
You really are making a simple task hard for yourself. By using interrupts and a circular buffer removes the hard real time requirement. You only need to ensure you can read all the chars faster than them coming in. Considering you can process the chars 1000 times faster than what they're coming in at, there shouldn't be a problem. Next you grab a character at a time from the circ buffer and assemble a line. If there's no chars, then go do something else but make sure you come back in a reasonable time. Eventually you'll assemble a line. Check for ok or whatever. Rinse and repeat. That's how many systems are implemented.
I built a system that had two gps receivers, a uhf data link and a wifi data link. The main loop would check each buffer in turn, extract the chars to build a message. If there was a complete message, it would process the message otherwise it would try the next circ buffer and repeat.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

kobidon wrote:
 what if when I turn-off the interrupt by the cli() ,you suggested, and right after that I get a character ? it will be lost.. Am I wrong ??
 

Yes, you're wrong. The received character will lie waiting in the USART and the moment you enable interrupts again the interrupt will occur, the ISR will run and the character will be put into the ring buffer.

 

The problem arises if you have interrupts disabled long enough that a second char will arrive. Now you have the data-overrun problem I talked about above (disturbing that you didn't ask, when you obviously didn't understand, BTW...). So the trick is to have interrupts disabled no longer than the time to receive one char.

 

kobidon wrote:
The EOL is not a char but a string \r\n so I can't just compare it to the UDR (the incoming char)
 

Decide on one of the chars to signal the EOL. Just throw away the other. There might be practical implication going one way or the other, but I'd try using \r to detect EOL and simply throwing any \n away when it arrives. YMMV, depending on the data/protoccol you have.

 

Re the limitation on how the time an ISR may block, it all depends. The principles, as far as I am concerned:

 

- Just do "the necessary" stuff in the ISR. E.g. buffer data and set a flag. Everything else happens in the main thread of execution.

 

- Do ballpark estimates of how close you are to "interrupt contention". The example with the USART ISR above is a school book case.

 

- Learn to recognise the symptoms of the system going into "interrupt contention". Deal with it when it happens.

 

kobidon wrote:
 I prefer not to use a linear buffer because I can get manage without it.
 

Yes, you can get by without it. My point is that life will probably be easier with it. You are fearing the initial extra effort while not seeing the long-term gain.

 

It is entirely up to you.

 

kobidon wrote:
An example for a typical response : blablabla\r\r\n\r\nOK\r\n What is your advice for that ? How can I really detect the real EOL ?
 

Now you have a complex syntax, in need of complex parsing methods. This argues even more for a layered design. You need to tokenize the input - i.e. convert the basic stream of chars into a higher level stream of tokens. You the parse that strem of tokens - the parser probably being implemented in a "state machine" (Google that, and search this site - we have a tutorial on the subject in the Tutorials forum.

 

All in all - your problem, as you describe it, calls for a well-structured, layered, abstracted, solution. You are not even interested in introducing the most obvious layering/abstraction so I have little hope you will attack the more complex syntax you introduced above with a structured approach.

 

I've argued all I can, and for the time being more arguments from me is obviously a waste of time, both yours and mine.

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

Last Edited: Mon. Nov 30, 2015 - 09:47 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

blablabla\r\r\n\r\nOK\r\n How can I really detect the real EOL ?

As always I would use '\n' as the delimiter and probably simply filter out (ie just ignore) any \r so I would see that as:

blablabla

OK

(and yes that does include a blank line between the two as you appear to have a bonus "\r\n" sequence in the middle).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I will try in parrallel to the existing code this way you all suggest and I will be back with conclusions.

 

In regarding to :

clawson wrote:

blablabla\r\r\n\r\nOK\r\n How can I really detect the real EOL ?

As always I would use '\n' as the delimiter and probably simply filter out (ie just ignore) any \r so I would see that as:

blablabla

OK

(and yes that does include a blank line between the two as you appear to have a bonus "\r\n" sequence in the middle).

 

So what do you suggest me is to refer the '\n' as the EOL and raise a flag at every '\n' so that chasing after the string in the ring-buffer will begin then ?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

kobidon wrote:
So what do you suggest me is to refer the '\n' as the EOL and raise a flag at every '\n' so that chasing after the string in the ring-buffer will begin then ?

See my sketchy code.

 

Not a flag, but an integer variable indicating how many seen but unhandled EOLs there are. Yes, most (all?) of the time it will be wiggling between 0 and 1, but if you want to be prepared for e.g. several empty lines coming in while you're doing something else then counting instead of flagging will make things easier and less bug-prone.

 

 

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

Last Edited: Tue. Dec 1, 2015 - 09:10 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

So what do you suggest me is to refer the '\n' as the EOL and raise a flag at every '\n' so that chasing after the string in the ring-buffer will begin then ?

Various systems tend to send one of "\r\n", "\n\r" or "\n" at the end of line. (Windows typically "\r\n", Linux "\n") so you can be pretty sure that all line endings have '\n' in them somewhere but there may or may not be '\r' so the easiest technique, as I said above, is simply to ignore \r all together and act on \n.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

JohanEkdahl wrote:

The solution is to temporarily disallow interrupts to occur while the process of pulling out data is active. This means

 

a) only the actual part of code pulling data out of the buffer, and

 

b) that this must be as short and fast as possible so that you do not get into a data overrun condition on the USART.

 

This is anotheer argument for having a function that pulls data out of the ring buffer, w/o destroying it, into a linear buffer. Only during this pulling out do you disallow the interrupts. The analysis of the data in the linear buffer takes place with interrupts enabled.

 

An alternative is, as you say, to wait for a whole "sentence" - I interpret that as a complete line. You could take on the extra cost in the ISR to detect the EOL arriving, and raise a flag that signals that the ring buffer holds an entire line. The pulling out then can be destructive since you know that you will have a complete line in the linear buffer. Pulling from the ring buffer to the linear will still need to be done with interrupts disabled, so make it fast.

 

 

I'm not sure I agree with your proposal to disable interrupts ( by cli() ) while the process of pulling out data is active.

Think of a case that I'm pulling data from the Ring-Buffer into the Linear buffer (after an EolSeen is no 0 anymore ), what if a new data from the module comes in this time or still continue to flow into the MCU ? it will be lost cause the ISR is not firing (disabled) and hence the RingBuffer_Insert will not occur.

 

I will be glad for an explanation.

thanks!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 

If you don't understand "atomicity" it's time to hit your computer science text books again!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It is critically important that shared variables and arrays be read and modified atomically.  That's why you must briefly disable interrupts when accessing the ring buffer.

 

The USART can buffer 3 frames (less one bit) before dropping the oldest unread frame.  At 9600 baud 8N1, a frame takes 1.04 ms.  3 frames less one bit will take 3.12 ms.  At 8 MHz, that's over 24,000 cpu cycles.  Even at 1 MHz, that's over 3,000 cycles.  It won't take a tiny fraction of that to copy the ring buffer for analysis.

 

 

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

kobidon wrote:
it will be lost cause the ISR is not firing (disabled) and hence the RingBuffer_Insert will not occur.

Wrong.

 

1. Note that I said to have interrupts disabled for as short a time as possible.

2. I've already told you above (in post #42) that the USART will happily receive one char while interrupts are disabled and fire the interrupt for it when they are enabled again.

3. The short time will likely be on the order of 1/100th or 1/1000nd of the time it take a character to arrive at the USART.

 

kobidon wrote:
I will be glad for an explanation.

I've already explained all this in several posts above. If you aren't reading them then that it not very respectful if you ask me, and writing them was a total waste of time. I'm definitively out.

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

This thread had become rather drawn out and we don't seem to be converging on a solution. This task is a common problem that has common solutions. There is zillion examples of code on the web where it has been done. Even the arduino code shoves the chars in a circular buffer, extracts them into a line, then decodes. Simple to understand, implement abd debug. So where is the problem?

Pages