How to distinguish between SRAM and PROGMEM at run time?

Go To Last Post
13 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi all,

 

I need to know how to make a function that will accept data from an SRAM array or from PROGMEM and act accordingly.  The actual data I am trying to pass to the function is either font data or image data (being displayed on a graphics VFD screen).

 

For simplicity, here's what I am trying to do:

 

// unsigned data in PROGMEM
const uint8_t u_pgm[] PROGMEM = {
    0x13,
    0x21,
    0x44,
    0x85,
};
// signed data in PROGMEM
const int8_t s_pgm[] PROGMEM = {
    0x12,
    0x33,
    0x54,
    0x88,
};
// unsigned data in SRAM
const uint8_t u_sram[] = {
    0x00,
    0x11,
    0x22,
    0x33,
};

// signed sata in SRAM
const int8_t s_sram[] = {
    0x55,
    0xAA,
    0xFF,
    0x00,
};

 

Now I have functions that are supposed to accept any one of those (typical code below):

 

void drawImage (const void *data PROGMEM)
{ // should grab data from a PROGMEM array
    uint8_t c, x;
    for (x = 0; x < 32; x++) {
        c = pgm_read_byte (data + x);
        // do something with c
    }
}

void drawImage (const void *data)
{ // should grab data from an SRAM array
    uint8_t c, x;
    char *ptr;
    ptr = (char *)(data); // make a pointer
    for (x = 0; x < 32; x++) {
        // do something with c
        c = ptr[x];
    }
}

 

My hope was that having "PROGMEM" in the first function (and in it's declaration in the .h file) would make all PROGMEM data go to IT and the SRAM data go to the other one.

 

But, the compiler doesn't seem to see the difference and complains that "error: 'void drawImage(const void*)' cannot be overloaded"

 

I'm sure that's because the compiler doesn't know the difference between "void *" and "void * PROGMEM".

 

My question is, CAN this be done and if so, HOW?

 

Thanks!

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Last Edited: Sat. Aug 4, 2018 - 06:11 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Use __flash (for your data in progmem) and __memx ( in your function) instead...
.
https://gcc.gnu.org/onlinedocs/gcc/Named-Address-Spaces.html

:: Morten

 

(yes, I work for Atmel, yes, I do this in my spare time, now stop sending PMs)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

When you use __memx you can then use __builtin_avr_flash_segment to determine if the pointer points to flash or RAM

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

__flash and __memx?

Doesn't the error message implies C++?

Stefan Ernst

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If we are talking C++, yes, it's possible, I had a similar problem that I solved by creating a class and a macro to hide PROGMEM related stuff.

This is the source for the class and macro: https://github.com/ElTangas/jtag...

 

This took me quite a while to get working. Here is the OP code converted using my technique:

 

#include <stdint.h>
#include<avr/pgmspace.h>


#define FLASH const PROGMEM flash

template <typename T>
class flash {
    private:
    const T data;

    public:
    // normal constructor
    constexpr flash (T _data) : data(_data) {}
    // default constructor
    constexpr flash () : data(0) {}

    operator T() const {
        switch (sizeof(T)) {
            case 1: return pgm_read_byte(&data);
            case 2: return pgm_read_word(&data);
            case 4: return pgm_read_dword(&data);
        }
    }
};



// unsigned data in PROGMEM
FLASH<uint8_t> u_pgm[] = {
    0x13,
    0x21,
    0x44,
    0x85,
};
// signed data in PROGMEM
FLASH<int8_t> s_pgm[] = {
    0x12,
    0x33,
    0x54,
    0x88,
};
// unsigned data in SRAM
const uint8_t u_sram[] = {
    0x00,
    0x11,
    0x22,
    0x33,
};

// signed sata in SRAM
const int8_t s_sram[] = {
    0x55,
    0xAA,
    0xFF,
    0x00,
};

template <typename T>
void drawImage (FLASH<T> *data )
{ // should grab data from a PROGMEM array
    T c, x;
    for (x = 0; x < 32; x++) {
        c = data [x];
        // do something with c
        PORTB = c;
    }
}


void drawImage (const int8_t *data)
{ // should grab data from an SRAM array
    uint8_t c, x;
    char *ptr;
    ptr = (char *)(data); // make a pointer
    for (x = 0; x < 32; x++) {
        // do something with c
        c = ptr[x];
    }
}

int main () {
	drawImage(u_pgm);
	drawImage(s_pgm);
	drawImage(u_sram);
	drawImage(s_sram);
}

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

meolsen wrote:
Use __flash (for your data in progmem) and __memx ( in your function) instead... . https://gcc.gnu.org/onlinedocs/g...

 

I looked at the link you provided and.... sadly I don't get it.

 

I tried to use "__flash" in various ways such as  drawImage (const __flash char * data, int x, int y, .......etc)  and everything I tried resulted in "error: '__flash' does not name a type". sad

 

I then got the "great" idea of seeing if I could tell the difference between PROGMEM and SRAM by the pointer's address. Sadly, looking at "&data" told me nothing useful (I guess the address offsets are stripped out?)

 

Ready to try more......

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

El Tangas wrote:

If we are talking C++, yes, it's possible, I had a similar problem that I solved by creating a class and a macro to hide PROGMEM related stuff.

This is the source for the class and macro: https://github.com/ElTangas/jtag...

 

This took me quite a while to get working. Here is the OP code converted using my technique:

 

 

OK I tried your code and have a few problems (errors). Before I start ripping into it, I'd like you to look at what I did and tell me if I screwed it up.

 

The changes I made are:

  added (uint8_t) and (int8_t) to the array defines to stop the complier from griping.

  added a small serial port driver so that I could see the output.

  had to add "CPPFLAGS+=-std=c++11" to the Makefile so it would recognize "constexpr".

 

Here are the warnings and errors I get:

 

----------------- snip ---------------------

In file included from readit.cpp:8:0:
serial.h:99:36: warning: unused parameter 'fp' [-Wunused-parameter]
 static int _outChar (char c, FILE *fp)
<=== my problem
                                    ^
serial.h:108:27: warning: unused parameter 'fp' [-Wunused-parameter]
 static int _inChar (FILE *fp)
<=== my problem
                           ^
readit.cpp:66:28: warning: '__progmem__' attribute ignored [-Wattributes]
 void drawImage (FLASH <T> *data)
                            ^
readit.cpp: In function 'int main()':
readit.cpp:116:18: error: invalid conversion from 'const uint8_t* {aka const unsigned char*}' to 'const int8_t* {aka const signed char*}' [-fpermissive]
  drawImage(u_sram);
                  ^
readit.cpp:86:6: note: initializing argument 1 of 'void drawImage(const int8_t*)'
 void drawImage (const int8_t *data)
      ^
<builtin>: recipe for target 'readit' failed
make: *** [readit] Error 1

----------------- snip ---------------------

 

Here's the test program itself:

 

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <stdint.h>
#include <ctype.h>
#include <avr/pgmspace.h>
#include <avr/interrupt.h>
#include "serial.h"

#define FLASH const PROGMEM flash

template <typename T>
class flash {
    private:
    const T data;

    public:
    // normal constructor
    constexpr flash (T _data) : data(_data) {}
    // default constructor
    constexpr flash () : data(0) {}

    operator T() const {
        switch (sizeof(T)) {
            case 1: return pgm_read_byte(&data);
            case 2: return pgm_read_word(&data);
            case 4: return pgm_read_dword(&data);
        }
    }
};

// unsigned data in PROGMEM
FLASH <uint8_t> u_pgm[] = {
    (uint8_t) 0x13,
    (uint8_t) 0x21,
    (uint8_t) 0x44,
    (uint8_t) 0x85,
};

// signed data in PROGMEM
FLASH <int8_t> s_pgm[] = {
    (int8_t) 0x12,
    (int8_t) 0x33,
    (int8_t) 0x54,
    (int8_t) 0x88,
};

// unsigned data in SRAM
const uint8_t u_sram[] = {
    (uint8_t) 0x00,
    (uint8_t) 0x11,
    (uint8_t) 0x22,
    (uint8_t) 0x33,
};

// signed sata in SRAM
const int8_t s_sram[] = {
    (int8_t) 0x55,
    (int8_t) 0xAA,
    (int8_t) 0xFF,
    (int8_t) 0x00,
};

template <typename T>

void drawImage (FLASH <T> *data)
{ // should grab data from a PROGMEM array
    T c, x;

    fprintf (stdout, "From FLASH\n\n");

    char *ptr;
    ptr = (char *)(data); // make a pointer

    for (x = 0; x < 32; x++) {
        if ((x % 4) == 0) {
            fprintf (stdout, "\n");
        }
        // do something with c
        c = ptr[x];
        fprintf (stdout, "0x%02X, ", c);
    }
}


void drawImage (const int8_t *data)
{ // should grab data from an SRAM array
    uint8_t c, x;

    char *ptr;
    ptr = (char *)(data); // make a pointer

    fprintf (stdout, "From SRAM\n\n");

    for (x = 0; x < 32; x++) {
        if ((x % 4) == 0) {
            fprintf (stdout, "\n");
        }
        // do something with c
        c = ptr[x];
        fprintf (stdout, "0x%02X, ", c);
    }
}

int main (void) {

    begin (115200, 0, "8n1"); // setup serial port
    fdevopen (_outChar, _inChar); // setup std streams

    __builtin_avr_delay_cycles (F_CPU / (1e3 / 1000)); // 1 sec delay for terminal to start

    fprintf (stdout, "\ntest starts...\n\n");

    drawImage(u_pgm);
    drawImage(s_pgm);
    drawImage(u_sram);
    drawImage(s_sram);
}

 

The serial code isn't relevant, but I'll post it if you like.  Thanks again!

 

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Last Edited: Sat. Aug 4, 2018 - 11:16 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It seems the compiler wants you to overload the drawImage function with a version taking a const uint8_t argument.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

El Tangas wrote:

It seems the compiler wants you to overload the drawImage function with a version taking a const uint8_t argument.

 

Actually, I "tinkered" with it and got it to work.  For the side that accepts a pointer from SRAM, I just changed it to "void *data" and then set a "real" pointer to it.

 

The PROGMEM part now works, and it seems as though this part:  

void drawImage (FLASH <T> *data)

...is the key.

 

What I don't understand is WHY this works, especially since the compiler spews this out:

 

readit.cpp:82:28: warning: '__progmem__' attribute ignored [-Wattributes]
 void drawImage (FLASH <T> *data)

 

I also don't understand what this part is supposed to do:

 

class flash {
    private:
    const T data;

    public:
    // normal constructor
    constexpr flash (T _data) : data(_data) {}
    // default constructor
    constexpr flash () : data(0) {}

    operator T() const {
        switch (sizeof(T)) {
            case 1: return pgm_read_byte(&data);
            case 2: return pgm_read_word(&data);
            case 4: return pgm_read_dword(&data);
        }
    }
};

 

I get what the 1/2/4 thing does (variable size) but my application will only do byte reads so "pgm_read_byte" is sufficient.

 

I don't understand, though, what the other stuff does (operator, constexpr, etc...)

 

I know C fairly well, but this fancy new C++ stuff goes over my head.  If you could be so kind as to explain how this is working I would appreciate it.

 

Thanks again.

 

-- Roger

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Last Edited: Sun. Aug 5, 2018 - 08:14 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ok, let me see where to begin...

 

As you noticed, the compiler can't distinguish between PROGMEM variables and normal variables as different types, so the drawImage() function cannot be overloaded in this way.

 

The solution to create a new type, in this case the "flash" class, and make it behave as closely as possible like a built in numeric type, but one that gets all its data from the flash memory instead of the SRAM. This is the purpose of everything inside the class, to mimic a numeric type behaviour.

 

Now, there are many numeric types, and you don't want to write a different class for each one. So I created a class template, an entity that can become a class based on the template argument we supply:

template <typename T>
class flash {

 

This means that when you create a variable of type flash<uint8_t> (let's ignore the macros involved for simplicity sake):

// unsigned data in PROGMEM
FLASH <uint8_t> u_pgm[] = {
    (uint8_t) 0x13,
    (uint8_t) 0x21,
    (uint8_t) 0x44,
    (uint8_t) 0x85,
}

 

the  T  template argument becomes uint8_t and the compiler automagically creates a new class:

class flash {
    private:
    const uint8_t data;

    public:
    // normal constructor
    constexpr flash (uint8_t _data) : data(_data) {}
    // default constructor
    constexpr flash () : data(0) {}

    operator uint8_t() const {
        switch (sizeof(uint8_t)) {
            case 1: return pgm_read_byte(&data);
            case 2: return pgm_read_word(&data);
            case 4: return pgm_read_dword(&data);
        }
    }
};

 

Since sizeof(uint8_t) is 1 and known at compile time, the compiler optimizations will just discard the other cases and simplify this to:

class flash {
    private:
    const uint8_t data;

    public:
    // normal constructor
    constexpr flash (uint8_t _data) : data(_data) {}
    // default constructor
    constexpr flash () : data(0) {}

    operator uint8_t() const {
        return pgm_read_byte(&data);
    }
};

 

The same process happens each time you use flash<type> with a different type; a new class specific for that type is created at compile time.

 

(to be continued...)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Which chip do you use?

 

Many of the "newer" chips have memory mapped flash (so you can use the same pointer)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

(continued from #10)

 

So now to explain how the flash class works. First we have:

    private:
    const uint8_t data;  // PROGMEM should ideally be here, but it doesn't work

 

This is a private variable that holds the value stored in each flash<uint8_t> class instance. I want this to reside in the flash, so it needs to be const PROGMEM. Unfortunatelly, PROGMEM also doesn't seem to work inside class definitions, that's why I had to wrap the flash class inside the FLASH macro, that basically just adds the PROGMEM modifier to class instantiations:

#define FLASH const PROGMEM flash

 

So what happens when you create an array of FLASH<uint8_t> like here?

// unsigned data in PROGMEM
FLASH <uint8_t> u_pgm[] = {
    (uint8_t) 0x13,
    (uint8_t) 0x21,
    (uint8_t) 0x44,
    (uint8_t) 0x85,
}

 

The class constructor is called, once for each member of the array. In this case there are 2 versions of the constructor, one for when a value is supplied, the other for when no value is provided, that defaults to a zero value.

    public:
    // normal constructor
    constexpr flash (uint8_t _data) : data(_data) {}
    // default constructor
    constexpr flash () : data(0) {}

I made these constructors constexpr to make sure they cannot be called at run time. That's because you cannot be allowed to create flash variables at run time. The constructors initialize the data private class member either to zero (default) or to whatever value is supplied in the initialization.

 

(to be continued...)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

(continued from #12)

 

Finally, to explain the meaning of this part:

    operator T() const {
        switch (sizeof(T)) {
            case 1: return pgm_read_byte(&data);
            case 2: return pgm_read_word(&data);
            case 4: return pgm_read_dword(&data);
        }
    }

 

This is the cast operator of the flash class template. It gets called whenever an explicit or implicit cast of a flash<T> instance is needed.

In particular, whenever you try, for example, to use a flash<uint8_t> in a place where a uint8_t was expected, an implicit cast is made so this gets called. It retrieves the necessary data using the appropriate pgm_read() function, and returns it.

 

The result of all this, is that you have these types flash<T> that behave more or less as if they were type T but stored in flash memory (there are some limitations that I found when testing this, but I can't recall them in detail).

 

I also want to point out that this:

template <typename T>

void drawImage (FLASH <T> *data)
{ // should grab data from a PROGMEM array
    T c, x;

    fprintf (stdout, "From FLASH\n\n");

    char *ptr;
    ptr = (char *)(data); // make a pointer

    for (x = 0; x < 32; x++) {
        if ((x % 4) == 0) {
            fprintf (stdout, "\n");
        }
        // do something with c
        c = ptr[x];
        fprintf (stdout, "0x%02X, ", c);
    }
}

is a function template. It gets replaced at compile type by any number of different functions, depending on the T you used in your program.

So in this case you have:

    drawImage(u_pgm);
    drawImage(s_pgm);

u_pgm is of type  FLASH <uint8_t> *, while s_pgm is of type FLASH <int8_t> * (well, they are array types in reality, but these get cast automatically). This will cause the generation of functions based on the template, but where T is replaced by uint8_t or int8_t.

 

So, I hope this is helpful, it took me quite a while to come up with this method but it illustrates some of the flexibility of C++.