String in struct in progmem

Go To Last Post
12 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hello!

Sorry for starting up yet another program memory thread.
I ran into a problem when trying to implement a command interpreter (a shell if you like).
I need to have a dictionary with the commands and the functions that should be called when invoking them. The command names should be stored in program memory to minimize RAM usage, but the structure itself could be in RAM too.
I ran into a problem defining the structure, I tried something like:

typedef void (*cmd_funcptr)(uint8_t*);

typedef struct 
{
  uint8_t* cmd;
  cmd_funcptr func;
} cmd_desc;
cmd_desc cmd_table[] = 
{
  {PSTR("addcard"), &cmd_addcard},
  {PSTR("delcard"), &cmd_delcard},
  {PSTR("list"), &cmd_list},
  {PSTR("gettime"), &cmd_gettime},
  {PSTR("settime"), &cmd_settime},
  {PSTR("adjtime"), &cmd_adjtime},
  {PSTR("help"), &cmd_help},
  {PSTR("auth"), &cmd_auth},
  {PSTR("log"), &cmd_log},
  {PSTR("open"), &cmd_open},
  {PSTR("lock"), &cmd_lock},

};

This didn't compile, it complained about:
"commands.c:17: error: braced-group within expression allowed only inside a function"

I saw some tutorials that used a separate variable to define the strings, and then use these pointers in the structure, but this seems clumsy.

Declaring a variable for each command is not very flexible and inconvinient. If one of the command names are modified, or a new command needs to be added, or one deleted, it can be a mess to modify.

Is there really no way to implement this with anonymous variables, something like what I was trying to do up there?

Regards,

axos88

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

axos88 wrote:
I saw some tutorials that used a separate variable to define the strings, and then use these pointers in the structure, but this seems clumsy.
Your best option, I think, is to write a simple generator program (using awk, Perl, etc.) that processes a description language and outputs the definitions that you need. It is a simple matter to add another rule to your makefile to automatically regenerate the data definitions whenever your description file changes.

I've done this type of thing on numerous occasions over the years. Whenever you have a situation where manual editing is tedious or error-prone, your first thought (assuming that re-designing won't eliminate those attributes) should be of creating a tool to automate the process.

Don Kinzer
ZBasic Microcontrollers
http://www.zbasic.net

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
This didn't compile, it complained about:
"commands.c:17: error: braced-group within expression allowed only inside a function"

Hmmm... what is that message trying to tell us, exactly? I'm a bit confused by it. It seems to be complaining about the {PSTR(),&foo}, groups. The braces around those groups are optional, even if it makes things harder to read. Have you tried simply leaving them out?

I also note that while PSTR yields a pointer that has been cast to a const char *, you are putting it into a uint8_t* -- the two should be compatible, but are not identical -- maybe the compiler is getting annoyed over that??

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Here's a sleazy technique that sometimes helps in situations like yours. Suppose you have a file containing the names of your functions in the
form of a bunch of macro invocations, like so:

// FunctionList.src - Names of functions
//
  AddFunc(addcard)
  AddFunc(delcard)
  AddFunc(list)
  AddFunc(gettime)
  AddFunc(settime)
  AddFunc(adjtime)
  AddFunc(help)
  AddFunc(auth)
  AddFunc(log)
  AddFunc(open)
  AddFunc(lock)

And then suppose your source code did something like this:

// SomeModule.cpp - a module that builds a table of string pointers
//
#define AddFunc(naym) const prog_char naym ## _str[] = #naym;

#include "FunctionList.src"

#undef AddFunc
#define AddFunc(naym) { naym ## _str, &cmd_ ##naym },

cmd_desc cmd_table[] PROGMEM =
{
   #include "FunctionList.src"
};

It might be possible to define the macro such that it could work in "one pass", so that you wouldn't need to resort to the hack of #including the same file twice. It might be possible for such a macro to generate inline gas assembler that ".pushsection"s into a "function strings" section, emits a labeled function name as an .asciz construct, then changes sections to a "function table" section, where it emitted the value of the label (an ".int" directive??? would help if the manual were comprehensible), followed by the value of the associated function address. The command table you're looking to build would then be effectively the beginning of that "function table" section. I suspect, though, that you'd also have to secure the help of one of the three people on the planet who can understand gnu's linker enough to add the right linker control magic (not me, though; sadly, having been exposed at a tender age to several linkers which were powerful, capable, and comprehensibly documented, I've no hope of grasping ld)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

dbc wrote:
I'm a bit confused by it. It seems to be complaining about the {PSTR(),&foo}, groups. The braces around those groups are optional, even if it makes things harder to read. Have you tried simply leaving them out?
It is caused by the PSTR macro. The error message relates to the braces inside of the macro. PSTR as an initialization value simply don't work.

Stefan Ernst

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yeah, the define is:

 # define PSTR(s) (__extension__({static char __c[] __attribute__((__progmem__)) = (s); &__c[0];}))

axos88

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I kinda get how this is supposed to work, but can you tell me what the ## and the # mean?

Levenkay wrote:
Here's a sleazy technique that sometimes helps in situations like yours. Suppose you have a file containing the names of your functions in the
form of a bunch of macro invocations, like so:
// FunctionList.src - Names of functions
//
  AddFunc(addcard)
  AddFunc(delcard)
  AddFunc(list)
  AddFunc(gettime)
  AddFunc(settime)
  AddFunc(adjtime)
  AddFunc(help)
  AddFunc(auth)
  AddFunc(log)
  AddFunc(open)
  AddFunc(lock)

And then suppose your source code did something like this:

// SomeModule.cpp - a module that builds a table of string pointers
//
#define AddFunc(naym) const prog_char naym ## _str[] = #naym;

#include "FunctionList.src"

#undef AddFunc
#define AddFunc(naym) { naym ## _str, &cmd_ ##naym },

cmd_desc cmd_table[] PROGMEM =
{
   #include "FunctionList.src"
};

It might be possible to define the macro such that it could work in "one pass", so that you wouldn't need to resort to the hack of #including the same file twice. It might be possible for such a macro to generate inline gas assembler that ".pushsection"s into a "function strings" section, emits a labeled function name as an .asciz construct, then changes sections to a "function table" section, where it emitted the value of the label (an ".int" directive??? would help if the manual were comprehensible), followed by the value of the associated function address. The command table you're looking to build would then be effectively the beginning of that "function table" section. I suspect, though, that you'd also have to secure the help of one of the three people on the planet who can understand gnu's linker enough to add the right linker control magic (not me, though; sadly, having been exposed at a tender age to several linkers which were powerful, capable, and comprehensibly documented, I've no hope of grasping ld)

axos88

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

can you tell me what the ## and the # mean?

They are the pre-processor stringizing and concatenation operators, respectively.

http://gcc.gnu.org/onlinedocs/cp...
http://gcc.gnu.org/onlinedocs/cp...

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It strikes me that writing this in assembler is just about the worlds simplest .S file....

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Okay, found a kindof workaround.
As I had program memory to spare, and I knew that all my command lengths are below 12 characters,
I just modified my structure to have char a[12] instead of char*. This way the whole string is part of the structure, rather than just the pointer to it.
This way I could do:

cmd_desc cmd_table[] PROGMEM =
{
{ "cmd1", &cmd1 },
{ "cmd2", &cmd2 },
{ "cmd3a", &cmd3 },
{ "cmd3b", &cmd3 },
{ "cmd4", &cmd4 }
};

And it is now stored in program memory.

Regards,

axos88

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well, the strings were going to be stored there anyway, and this way you get back the two bytes per string that were going to the pointer. So the cost really isn't much.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The following is ripped directly from my command parser .c and .h files.

Things you should know: My commands are in the form:

token_token_token_token [param [param [param [param]]]]

for example: get_status
             send_command 27

so I have split the commands into recognizing the four token pieces. Also, I arbitrarily decided that no more than 4 characters of a token would be needed to identify the token. If the token is less than for 4 characters ("set", "get", ...) I pad the token in the parser table with spaces.

Each command can have up to 4 parameters, of a type specified in the command table.

I know this is far more than you asked for, but I hope it will help.

Stu

/* cmdPT_x - the type of parameter a command requires or returns
 *          --- maximum types: 16 (0-15)
 */

#define cmdPT_NONE 	 0		// no parameter
#define cmdPT_BOOL	 1		// boolean
#define cmdPT_INT	 2		// integer
#define cmdPT_OINT   3		// OPTIONAL integer param - only optional or no options can follow
#define cmdPT_LONG   4		// long integer (receiver responsible for freeing space)
#define cmdPT_OLNG   5		// long integer (receiver responsible for freeing space)
#define cmdPT_UINT	 6		// unsigned integer
#define cmdPT_HEX	 7		// returned unsigned integer, print in hex format
#define cmdPT_LHEX	 8		// returned unsigned long, print in hex format, release long
#define cmdPT_FLT	 9		// floating point number
#define cmdPT_OFLT	10		// OPTIONAL floating point number - only optional or no options can follow
#define cmdPT_STR	11		// string in SRAM (receiver responsible for freeing space)
#define cmdPT_PSTR	12		// string in Program Flash
#define cmdPT_PLIST 13		// Parameter List Extension (pointer to another CmdPacket)
#define cmdPT_IARR	14		// Integer array (pointer to CmdIntArray structure)

/* cmdPP_x - parameter procesing options (usually output related) */

#define cmdPP_RPT	0x01	// Input:  This command was also the previous command
#define cmdPP_COMMA	0x02	// Output: use comma separators instead of spaces
#define cmdPP_ASEOT	0x04	// Output: Asynch processor only: add EOT after CRLF

typedef CmdError (*CmdProc) ( CmdPacket* pCmdPkt );

/* CmdTableType - one entry for the command table */

typedef struct _CmdTableEntry {
	uint32_t	token_list;			// up to 4 8-bit cmd tokens specify the command
	uint16_t	ptypes;				// # of params and type of each (up to 4)
	CmdProc		command;			// pointer to the command procedure
} CmdTableEntry;

// ---------------------------------------------------------------------------
// A wee lesson in AVR support
//
//	We want to store our tables in flash - duh!.  In normal C we would use:
//
//		const   = ;
//
//  Unfortunately, string tables are a little more gnarly.  This would be the
//	simple and obvious method:
//
//		const char* array[] PROGMEM = {
//			"Foo",
//			"Bar"
//		};
//
//	where PROGMEM is and AVR macro telling the linker to put things in the
//  "program memory" (i.e., flash).  Too bad this doesn't work.  Instead,
//	we must use the following clumsy code:
//
//		const char foo[] PROGMEM = "Foo";
//		const char bar[] PROGMEM = "Bar";
//
//		PGM_P array[2] PROGMEM = {
//			foo,
//			bar
//		};
//
//	where PGM_P is another macro defining pointers into the flash memory.
//	
//	YUCK!  Well, to do this without making this file totally unreadable, we'll
//	define some macros to do the heavy lifting.
// ---------------------------------------------------------------------------

// ---------------------------------------------------------------------------
// CmdTokens -  Valid command tokens
//
//	These are the valid tokens within a given command.  EXACTLY 4 letters are
//	compared; any differences after matching the token below will be ignored.
//	Thus, we may have more than one string matching to the same token (e.g., 
//	"err" and "error").  If the token is less than 4 characters, the token 
//	must be space padded.
//
//  Implementor's note:  This is clumsy -- a better way would be to generate
//  a parsing table, and I still may do that, but this will work.  Rule: First
//  make it work, THEN make it fast.
//
//  Another note:  When you want to add another token to the list, please
//	pick the next available number, add the new name to the CmdTokens table,
//  AND REMEMBER TO INCREMENT THE SIZE OF CmdTokens!!!
// ---------------------------------------------------------------------------

#define TOKEN_TABLE(t) ctt_ ## t

#define TOKEN(t,s) \
	const char TOKEN_TABLE(t) [] PROGMEM = s; 

/* the following are in alphabetical order -- the token table is NOT */

TOKEN( ACCEL,   "acce" )   // accel
TOKEN( ADCNV,   "adc " )
TOKEN( AGC,     "agc " )
TOKEN( ALL,     "all " )
. . .
TOKEN( WRITE,    "writ" )
TOKEN( XSPEED,   "xspe" )   // xspeed
TOKEN( ZERO,     "zero" )


/* DO NOT change the order of the following table!!  You can add to the end */

#define LAST_CMD_TOKEN 158

const PGM_P CmdTokens[ LAST_CMD_TOKEN + 1 ] PROGMEM = {

    /*  0 */ TOKEN_TABLE( GET ),		
    /*  1 */ TOKEN_TABLE( SET ),		
    /*  2 */ TOKEN_TABLE( STATUS ),
    /*  3 */ TOKEN_TABLE( INIT ),	
    /*  4 */ TOKEN_TABLE( ERR ),
. . .
   /* 157 */ TOKEN_TABLE( BOUND ),
   /* 158 */ TOKEN_TABLE( SYNC ),
};

//  The following defines make the command table below easier to read.
//  Make sure that the numbers match the order in the token table
//  above!

#define CT_GET       0
#define CT_SET       1
#define CT_STATUS    2
#define CT_INIT      3
. . .
#define CT_TRAN    156
#define CT_BOUND   157
#define CT_SYNC    158


#define CT_FF      0xFF   // "terminating" entry


// ---------------------------------------------------------------------------
// CmdTable -	Command table
//
//	This gives all of the recognized commands.  Note that "recognized" does
//  NOT equate to "implemented"!
//
//  The token list lists the parsed token numbers from the CmdTokens table
//  above.  Each list must have four elements, so if the command has less
//  than that, the list is filled with 0xFF.
//
//  The parameter field lists the type of parameter expected by the command.
//
//  Finally, the command index is used in the parser to call the procedure 
//  that implements the command.
//
//  Note that we have an unimplemented command procedure for commands that 
//  are recognized but not implemented.
//
//	The order in the list should be arranged to find most frequently used
//  commands first, but we have just generated this from the command summary.
//
//  As above, we have the annoying AVR problems to deal with.  Again, we will
//  use the macros to create the items in flash and an array of pointers to 
//  the items, also in flash.
// ---------------------------------------------------------------------------

#define COMMAND_TABLE(t1,t2,t3,t4) cl_ ## t1 ## _ ## t2 ## _ ## t3 ## _ ## t4

#define PARAMS(p1,p2,p3,p4) ((uint16_t)(p1 << 12) | (uint16_t)(p2 << 8) | (uint16_t)(p3 << 4) | (uint16_t)(p4))


#define COMMAND(t1,t2,t3,t4,p,c) \
	const CmdTableEntry COMMAND_TABLE(t1,t2,t3,t4) PROGMEM = { \
			( ((uint32_t)CT_ ## t1 << 24) | ((uint32_t)CT_ ## t2 << 16) | ((uint32_t)CT_ ## t3 << 8) | ((uint32_t)CT_ ## t4) ), \
			(p), (c) };


//	Command	Tokens					Parameters													Command	Procedure			   

/* General Commands */
COMMAND( GET, STATUS, FF, FF,		PARAMS( cmdPT_OLNG, cmdPT_OINT, cmdPT_NONE, cmdPT_NONE),	CmdGetStatus			) 	// get_status
COMMAND( GET, ERROR, TEXT, FF,		PARAMS( cmdPT_INT,  cmdPT_NONE, cmdPT_NONE, cmdPT_NONE),	CmdGetErrorText			)	// get_error_text
COMMAND( GET, LAST, ERROR, CODE,	PARAMS( cmdPT_OINT, cmdPT_NONE, cmdPT_NONE, cmdPT_NONE),	CmdGetLastErrorCode		) 	// get_last_error_code
. . .


// The most important part of this table is that it include one entry of
// each command above.  There are not any particular ordering issues here.

#ifdef USE_TASKLIST
#define SIZEOF_CMD_TABLE 303
#else
#define SIZEOF_CMD_TABLE 302
#endif

const CmdTableEntry* CmdTable[SIZEOF_CMD_TABLE] PROGMEM = {
	/* General Commands */
	& COMMAND_TABLE( GET, STATUS, FF, FF ),
	& COMMAND_TABLE( GET, ERROR, TEXT, FF ),
	& COMMAND_TABLE( GET, LAST, ERROR, CODE ),
. . .
};

Engineering seems to boil down to: Cheap. Fast. Good. Choose two. Sometimes choose only one.

Newbie? Be sure to read the thread Newbie? Start here!