Copying strings in ATTiny416 unexpectedly painful

Go To Last Post
65 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Function USART_send_word_HEX_DBG below fails to work.

- Using AS7 and ATTINY416 nano board.

 

1. Specifically, the content copied into string outstr (ID'd as "OFFENDING CODE") by strncpy is all 0xff (which looks suspiciously like eeprom content) into the dest string.  Function strncpy_F copies all 0's.

2. In size_t len = strlen(desc); strlen returns 0; 

 

  Commented code remains because it shows what has been tried.  The one test call from main() appears below the function.  Toolchain is shown at the very bottom.

 

What the usart emits confirms that the values for the desc part of the string are 0xff, and strncpy copies into the correct location in the dest string.  If I strncpy a different string created within the function, it works as expected.  Only when I strncpy from the parameter does it fail.

 

Stack containing string outstr after the strncpy (previously initialized to 0's):

 

Debug display for the location of the string contained in parameter desc (seems like a "feature" of the debugger to me):

 

The debugger tells me the address of the incoming parameter desc is 0x8ff5 (confirmed by r[22:23]) which is in flash but I cannot read that content in the debugger.  PROGMEM is too full to use sprintf (the project is nearly done and need to debug one remaining difficult IR bitstream).

 

My questions please:

1.  Where in mem is the const string in the caller's arg (and should it be visible to me in the debugger)?  Am I missing some mem map translation?

2.  How can I know (or control) where such const strings are stored without debugging?  I cannot find rules.  Const strings created as function args like this can slip from notice pretty easily.

3. The language spec says a definition like char str[10] = "";  initializes the array, but in this compiler it does not.  How can I find out where the compiler does not meet the language spec?  Does GCC have a spec or a compliance statement like microsoft publishes?

 

I have possible undesirable workarounds to try but I really would like this to work as-written.

 

All advice and commentary is always gratefully received.

 

Jeff

 

//void USART_send_word_HEX_DBG(uint16_t value, const char *desc)
void USART_send_word_HEX_DBG(uint16_t value, const char desc[10])
{
    // "xxxx: tttttttttt\0"
    //  where xxxx = value, tttttttttt = desc
    
    size_t len = strlen(desc);        // <- OFFENDING CODE
    //char outstr[17] = "";
    char outstr[17];
    memset(outstr, 0, 17);
    
    CharToHex((char)(value >> 8),     &outstr[0]);
    CharToHex((char)(value & 0x00ff), &outstr[2]);
    
    outstr[4] = ':'; outstr[5] = ' ';
    strncpy(&outstr[6], desc, len > 10 ? 10 : len);        // <- OFFENDING CODE
    //strcat(outstr, desc);                                     // <- OFFENDING CODE
    //strcpy_P(&outstr[6], desc);                        // <- OFFENDING CODE (copies 0's)

    USART_write_string(outstr, true);
}

 

int main(void)
{
...
    USART_send_word_HEX_DBG(0xa5a5, "test word");
...

}

 

Last Edited: Mon. Mar 18, 2019 - 01:46 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi,

I think, this could help you

https://teslabs.com/openplayer/d...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ned Zeppelin wrote:
1.  Where in mem is the const string in the caller's arg (and should it be visible to me in the debugger)?  Am I missing some mem map translation?
By default that string is stored in flash (beyond the end of the code with other .data). During startup the CRT copies .data in flash to .data in RAM. The address of the RAM string is passed into the routine.
Ned Zeppelin wrote:
2.  How can I know (or control) where such const strings are stored without debugging?  I cannot find rules.  Const strings created as function args like this can slip from notice pretty easily.
It should not matter where they are stored as this is all "hidden" - you don't need to know - however you should be aware that it is in RAM - which is a waste for "const" data. Do you may want to explore the use of PSTR() to locate the string to flash alone. The receiving code would then dereference the pointer as a flash, not RAM address (so LPM not LDS). You can now do that easily with "const __flash"
Ned Zeppelin wrote:
3. The language spec says a definition like char str[10] = "";  initializes the array, but in this compiler it does not.  How can I find out where the compiler does not meet the language spec?  Does GCC have a spec or a compliance statement like microsoft publishes?
That is true for globals - in that case it would be in .BSS and the CRT wipes the .BSS with 0x00 at start up. However it is NOT true for stack frame automatics.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

To see what was happening I built this:

#include <string.h>
#include <stdbool.h>
#include <avr/io.h>

void CharToHex(uint8_t n, char * dst) {
	char hex[] = "0123456789ABCDEF";
	*dst = hex[n >> 4];
	*(dst + 1) = hex[n & 0x0F];
}

void USART_write_string(char * str, bool foo) {
	
}

__attribute__((noinline)) void USART_send_word_HEX_DBG(uint16_t value, const char desc[10])
{
	// "xxxx: tttttttttt\0"
	//  where xxxx = value, tttttttttt = desc
	
	size_t len = strlen(desc);        // <- OFFENDING CODE
	//char outstr[17] = "";
	char outstr[17];
	memset(outstr, 0, 17);
	
	CharToHex((char)(value >> 8),     &outstr[0]);
	CharToHex((char)(value & 0x00ff), &outstr[2]);
	
	outstr[4] = ':'; outstr[5] = ' ';
	strncpy(&outstr[6], desc, len > 10 ? 10 : len);        // <- OFFENDING CODE
	//strcat(outstr, desc);                                     // <- OFFENDING CODE
	//strcpy_P(&outstr[6], desc);                        // <- OFFENDING CODE (copies 0's)

	USART_write_string(outstr, true);
}



int main(void)
{
	USART_send_word_HEX_DBG(0xa5a5, "test word");
}

But then I noticed you said Tiny416. From other threads it seems like there are quite a few problems with debugging these new AVR-0 and AVR-1 Xtiny devices so I'm guessing that if you are not seeing the "right thing" it's probably a bug in the debugger for those chips.

 

But anyway, in main() what I see is that the CALL is made as:

	USART_send_word_HEX_DBG(0xa5a5, "test word");
0000008A 67.e5                LDI R22,0x57		Load immediate 
0000008B 71.e8                LDI R23,0x81		Load immediate 
0000008C 85.ea                LDI R24,0xA5		Load immediate 
0000008D 95.ea                LDI R25,0xA5		Load immediate 
0000008E c1.df                RCALL PC-0x003E		Relative call subroutine 

This is to be expected. GCC puts the first parameter of a call in R25:R24, then the next in R23:R22 and so on. You can see the 0xA5A5 in R25:R24. So 0x8157 is the address of the ("test word") string in RAM. Sure enough. In the memory display I see:

 

So 0x8157 in RAM does have "test word" as one might hope.

 

If I follow it into the called function the first thing it does is the strlen(). I see it do this...

 

 

So after the strlen R17:R16 contains 0x0009 which is the strlen. So this is all working as expected.

 

So what exactly do you think is wrong here with the parameter passing ??

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Actually I modified it so there was an "output" and ran it to here:

 

 

The memory window is showing "str". AFAICS that has worked exactly right hasn't it? The string is "A5A5: test word" which is surely what you were hoping for. So everything about this works doesn't it?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

AS always.  It is handy to provide the full buildable code.    e.g. ZIP up your AS7 project and attach the ZIP.

 

I would expect the Compiler to be correct.   And the Simulator to be correct.    And AS7 to work ok.

 

Of course the world is not always perfect.    If you have found a bug,   fellow readers can build and replicate.   

This makes a tremendous difference to any Bug Report.

 

However most times it is a User problem.    Readers can show you how to fix it.

 

Not everyone has a Tiny416.   I have a XMINI-Tiny817.    This can run the Tiny416 code in real life.    (And test the Simulator too)

 

David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I was just going to add a post where I showed the CRT doing the .data copy but:

00000034 <__ctors_end>:
  34:	11 24       	eor	r1, r1
  36:	1f be       	out	0x3f, r1	; 63
  38:	cf ef       	ldi	r28, 0xFF	; 255
  3a:	cd bf       	out	0x3d, r28	; 61
  3c:	df e3       	ldi	r29, 0x3F	; 63
  3e:	de bf       	out	0x3e, r29	; 62
  40:	8c d0       	rcall	.+280    	; 0x15a <main>

It does not exist!

 

Yup, confirmed:

 

 

So 0x8nnn is a flash address - it does not need to need to do the usual flash->RAM .data copy loop in the CRT because 0x8nnn is directly visible.

 

I think this is because the compiler for Xtiny is being very clever indeed. Because Xtiny have "flat memory" (non-Harvard if you like) they make both RAM and ROM visible in a single memory space. So it looks like the build system is smart enough to know that "test word" can remain in ROM and the 0x8157 used above is a flash address.

 

Crikey - these new chips (and the build tools) are clever in this sense!!

Last Edited: Mon. Mar 18, 2019 - 10:59 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thank you.  Aye, they're clever, and I'm amazed and happy with what I can pack into an 85-cent chip (decode two fast rotary encoders, debounce two switches, detect ambient light changes, decode IR signals from two different devices, abstract it all and shoot it out out SPI, along with blinkies and a trace uart for dev).  but I think the debugger is not keeping up - it is riddled with bugs, as is the IDE.  No need to digress into that just here.

 

On topic:  Sure, it makes sense to put const vars in progmem.  But the debugger does not display the content of progmem, so I cannot confirm const string pointers.  As in the posted code, strncpy_P fails by copying all 0's to the destination (unexpected).  It would help if I could see the actual progmem source string in the debugger.

 

I've been graciously handed some training wheels, which I'm bolting back on.  I have some work to do, starting with a little appl to work through these string handling issues after having been edified.  I will return here when I fully understand fixes, and will supply a simple test appl if necessary.

 

And BTW, you being in Essex, near The Wash, you might enjoy this book a lot.  I sure did.  I hope you can make the time.

 

Best wishes,

Jeff Lloyd

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi David,

 

Understood, all.  What I needed most was just the clue about how progmem is handled so differently by the compiler.  I'm quite inured to working around insufficiencies in dev tools, over many years.  I think I have plenty of info now to resolve the issue.  If not, I will supply a buildable sample.  But AS7 does not work ok.  I have found a number of bugs in the debugger/IDE (one pretty serious, and reproducible by others), and which I will submit once I figure out how to do that and when I get past my immediate programming problem.  We all just work around minor bugs, which are found in all IDEs.  Compilers just about always work, and expecting full language compliance in an MCU compiler is perhaps asking a bit too much, but it does help to know the exceptions, especially if your practice it to code to the language spec and not to observed behavior.

 

I do expect and believe this compiler to be correct, and I always look at my own code first.  In this case the issue really now narrows to why strncpy_P does not copy src string (in progmem) into the dest string (on the stack).  Again, I will make a test project to work through these things.

 

Thank you kindly for your help,

Jeff Lloyd

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Like I said above these new Xtiny are different to most previous AVR because they can LD from flash (no need for LPM) so you shouldn't need strncpy_P or anything like that because plain old strncpy() can do a flash to RAM copy. So forget PROGMEM and __flash etc. It looks like const (or implied const) is all you need

Last Edited: Mon. Mar 18, 2019 - 04:40 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

YIKES!  Had I known anyone was going to go to such lengths to analyze my problem I would have supplied the problem in a small test appl.  I really do not want to post the actual project.  I could have done the work myself, but I presumed that someone would immediately recognize my error without much analysis.

 

I think I got my mind right on const-ness in GCC, so my immediate plan is to just make a little test project and work through the issues.  I am still puzzled, however, by why strncpy_P copies 0's into the dest string instead of copying the actual progmem src string (which I cannot see in the debugger).  I hope to know a lot more today, or as soon as I can get back after it.

 

BTW, I consider it the responsibility of the compiler to honor the requirements of const-ness, no matter where const values are stored.

 

It will be hard to ID which of the several contributions is THE solution.  The recommended article by Dean Camera is gold, even if 12 years old and perhaps not current in some of the specifics.  I'll return to this thread soon when I really resolve it all.  Obviously, we want the solution to be useful for future users but the topic is messy.

 

Thank you SO MUCH for doing all the legwork!

 

Jeff Lloyd

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If anything this new architecture massively simplifies things. You can just write C like K&R intended. Apply "const" when it's appropriate and everything falls into place because these new chips are "hiding" their Harvardness and looking very VonNeumann indeed. So all the clunky workarounds that were needed to run a VonNeumann compiler (GCC is) on a Harvard architecture (AVR is) go out of the window. Bottom line you are overthinking things. These new chips have been (presumably deliberately?) designed to get on better with VonNeumann dev tools.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Does LPM work on OP's device?

If not and strncpy_P uses it, that could be a problem.

Iluvatar is the better part of Valar.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The datasheet lists LPM, so I suppose it works, though it's kind of redundant.

This reminds me of an experiment I wanted to do, though. Since the base address for LPM is at 0x8000 unified address, what happens if I read an address beyond the limit of the unified address space using LPM?

Wraparound to 0x8000 probably.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

(update)

No useful project to post yet:  The non-working functions at issue work ok when copied from my appl into a small new START project.  So as to post a complete project, I pared away everything but the offending code from a copy of my project, but then the uart-related functions worked in the minimized project.  This of course does not bode well at all.  strlen_P(str) returns 0 and strncpy_P copies 0's, BUT I do find the expected const string in the hex file.

 

I have one question you might be able to help with, if you please:

You posted a screen snip showing memory content in progmem.  This made me quite jealous:  I cannot display anything within progmem.  e.g. within function USART_send_HEX_DBG(uint16_t value, const char desc[10]). the debugger tells me that the address of parameter desc is 0x81d6.  Here is what I see surrounding that address:

 

 

Is there something I must do to enable the IDE to display progmem content?  I can see no content anywhere within progmem, ever, in any ATTINY project.  Symbol  desc appears in the locals window, but showing only its address and referring registers (it does not expand to show content).  In a watch window, desc[n] cannot show the content.   Quickwatch shows no content for the symbol.  Inconveniently, clicking the symbol in the locals window sometimes causes the IDE to wreck and restart, so I stopped trying THAT.

 

I'm using the nano's onboard debugger.  Progmem is 96.5% full in the real appl, but this will relax when I strip out the uart trace code.

 

I still have more things to try.

 

Anything you might be able to offer will be great...

 

Jeff

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks - I might need to use LPM just to see if I can access the parameter address.  I pared back my project to only the offending code and it works ok (using strncpy_P).  Progmem in the full project is 96.5% full, which should not be a problem, but...

 

My only alternative may be to reconstruct the project piece-by-piece to see where it fails, which will NOT be easy.

 

Jeff

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Your code does not make a lot of sense! I was going to look at the disassembly and step the Asm in the simulator for your call to strncpy_P() so I enabled that in the code but you are using:

strcpy_P(&outstr[6], desc); 

But the whole point of strncpy_P() is that it is used in the form:

strcpy_P(to_this_string_in_RAM, from_this_fixed_text_in_flash_that_has_to_be_LPMd);

So the second parameter has to be the address of some text in flash. But you use "desc" which is the parameter passed into the USART function. There is nothing on the pointer to say "it's in flash". What I see is:

 

 

 

The first &outstr[6] parameter will be in R25:R24 so it is 0x3FEA (not surprisingly a stack address). So as hoped it is the highlighted byte here:

 

 

The second parameters will be in R23:R22 for "desc" so it is 0x8171 which has the 0x8000 offset so is an address of flash data mapped into the RAM space, not the flash address (which would be the same with 0x8000 subtracted that is 0x0171).

 

Even if I apply the __flash modifier to "desc" then it is still 0x8171 so I guess this is telling us "don't use stncpy_P()". That kind of makes sense. There's no point copying from flash space something that is already in "RAM space".

 

As I keep saying, on these new architecture AVR there basically is never any point/need to use LPM at all because everything is visible to LD/LDS.

 

The only proviso is that if you want something that does not change ("const") to locate in flash alone you must use the const modifier. This puts it in .ro.data and the linker has the sense to do the 0x8000 offset thing for .ro.data

 

This chip design is clearly "cleverer" than traditional tiny/mega.

 

If you really, really do want to LPM something then subtract 0x8000 from the pointer to it. Maybe:

strcpy_P(&outstr[6], desc - 0x8000);

When I do that I get:

data 0x3FE4  41 35 41 35 3a 20 74 65 73 74 20 77 6f 72 64 00 00 3f ff 00 00 00 00 00 00 a5 00 21 ?? ?? ?? ?? ??  A5A5: test word..?ÿ......¥.!.....

So it seems that as long as it has the -0x8000 then LPM works OK. But life is a whole lot simpler with:

strncpy(&outstr[6], desc, len > 10 ? 10 : len);

which is doing a flash->RAM copy anyway (using LD not LPM)

Last Edited: Tue. Mar 19, 2019 - 10:01 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

PS OK so if you do what to use the (redundant) strcpy_P() this achieves it:

#include <string.h>
#include <stdbool.h>
#include <avr/pgmspace.h>
#include <avr/io.h>

void CharToHex(uint8_t n, char * dst) {
	char hex[] = "0123456789ABCDEF";
	*dst = hex[n >> 4];
	*(dst + 1) = hex[n & 0x0F];
}

void USART_write_string(char * str, bool foo) {
	uint8_t i;
	for (i=0; i< strlen(str); i++) {
		PORTB_OUT = str[i];
	}
}

__attribute__((noinline)) void USART_send_word_HEX_DBG(uint16_t value, const __flash char desc[10])
{
	// "xxxx: tttttttttt\0"
	//  where xxxx = value, tttttttttt = desc

	size_t len = strlen(desc);        // <- OFFENDING CODE
	//char outstr[17] = "";
	char outstr[17];
	memset(outstr, 0, 17);

	CharToHex((char)(value >> 8),     &outstr[0]);
	CharToHex((char)(value & 0x00ff), &outstr[2]);

	outstr[4] = ':'; outstr[5] = ' ';
	//strncpy(&outstr[6], desc, len > 10 ? 10 : len);        // <- OFFENDING CODE
	//strcat(outstr, desc);                                     // <- OFFENDING CODE
	strcpy_P(&outstr[6], desc);                        // <- OFFENDING CODE (copies 0's)

	USART_write_string(outstr, true);
}

int main(void)
{
	USART_send_word_HEX_DBG(0xa5a5, PSTR("test word"));
}

I used the usual PSTR() macro to ensure the literal string is in flash (except that for 416 is was going to be anyway) then I told the interface to USART_send_word_HEX_DBG() that the second parameter was "__flash".

 

Now the R23:R22 passed to strncpy_P() is 0x0034. The data at 0x0034 in PROGMEM is:

 

 

0x74 is at 0x0034.

 

The same is, of course, mapped to:

 

 

so it's at 0x8034 too

Last Edited: Tue. Mar 19, 2019 - 10:16 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well, there is a LOT that doesn't make sense, and today it's not a lot better.  However much sense it makes, the code works.  Looking for symptoms that do make sense, I discovered the weirdness seen in the screen snips below, which show two breakpoints in the same function call.

 

- Please have a look at the address of the 2nd parameter "desc", coming in via R[22:23] in the first screen snip.  It's the CCP register.  I confirmed the contents of R[22:23] (0x0034), and the value of that address (just for fun), which is CCP, which is 0.  Still, the function works, with strncpy_P working as expected for a progmem value.  Then, when I step down to line 114 (2nd screen snip), both function parameters have vanished from the locals window.

 

- I take the length of parameter "desc" in line 102, but I don't use it because I can't verify it because I have never, ever, been able to see the value of automatic variable "len" anywhere in the IDE.

 

All this shows the IDE is displaying incorrect and incomplete information.  This IDE is just NOT WORKING correctly.  I changed the handling of the desc parameter a few times based on the address of this parameter as seen in the debugger, and now I cannot trust it at all.  This must be a BUG in the IDE.  Apparently, the AS7 buglist is not public 8(

 

Aside from the above, string initialization is not working now, which is new and different.  The snip below shows how I must initialize a string - char by char.  Yesterday, it was fine.

    //char outstr[11] = "c=23, v=89\0";    // doesn't work (always worked before today)
    //char *outstr = "c=23, v=89\0";    // doesn't work   

    char outstr[11]; // "c=23, v=89\0";   
    memset(outstr, 0, 11);

   // initialize the hard way:
    outstr[0] = 'c'; outstr[1] = '='; outstr[4] = ',';  // "c=  ,"
    outstr[5] = ' '; outstr[6] = 'v'; outstr[7] = '=';  // " v="

 

The actual appl is attached.

 

Again, I thank you for your considerable time investment and your help...

 

Caller for below snips:   USART_send_word_HEX_DBG(SPI_msg_ambient_light.message_content.message.command_value, PSTR("light lvl"));

 

execute to line 114: