AVR Freaks

I guess the fact that you can't run code in page erased flash doesn't really surprise me, I just still have no way of knowing how to optimally place things in flash to take advantage of that. As far as I know, in the arduino environment the bootloader is stored in the core files as a hex already aligned to the 2kb fuseable boot section. If I export a hex of my code, it starts at page zero. Is there an easy way to tell the compiler to work backwards, high addresses to low?

In real time is probably not the best description, I guess I should say I need ~8K bytes of flash to update on occasion. These chunks of flash will not be updated regularly, only occasionally as I update MAF transfer curves or swap to different thermistors. The ~8k of bytes stores 1024 16bit words. Each corresponds with a ADC value and is used as a progmem lookup table with every ADC read. "Live" sensor values are all stored in ram, not to many of them as it's only around 16 bytes or so total.

Stuff like the thermocouple, baro sensor and others all run on timed intervals. Well, the thermocouple has about a 125ms read time as configured and I use a data ready pin as a trigger to update. Same on the baro sensor, it's set with a lot of filtering and outputs at ~10hz, using i2c reads to tell when data is ready. Once again, all my code works and is configured by and large from eeprom values on every startup. For the vast majority of configurable values (egt low map value, egt high map value, accelerometer output alignment, etc,) can all be stored in bytes in eeprom and read once on boot. I could and would do the same with these lookup tables, except they won't fit! I guess it's important to note that any "flash" to this board has to be followed by a reboot, doesn't matter if it's eeprom or flash memory. This is OK, still a lot easier than a full reflash every time I need to recalibrate sensors.

Either way I've figured out how to do what I want, and it looks a little like this.
 

// The temporary data (data that's read or is about to get written) is stored here
uint8_t ramBuffer[SPM_PAGESIZE];

// This array allocates the space you'll be able to write to
const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM = {

The other relevant part is making sure to use a proper optiboot bootloader and making sure I have a full 128 byte page residing in ram before I attempt to write to this aligned array as my CAN code is probably in a bad spot and won't run as soon as I erase a page. Since the code which writes flash is part of the bootloader it's in a place where the SPM command can actually run, It's pre-compiled hex already stored where I need it. Another advantage is I've got a brand new data stream to work with every time I write, I don't ever need to consider or save flash contents before erasing. I'll always be sequentially writing so I don't need to worry about the single address update per write limit.

Thank you all for the help! I think I can make this work from here!

I think I understand now, I can run code pretty much anywhere and use it to load the flash page buffer. In my application I am completely filling the table with new values I need not worry about the RWW and NRWW sections, I only need to make sure that the code that writes to flash starts from the BLS, since that's the only way SPM instruction will work. Since a page is small and easily fits in ram I can just buffer pages using code running anywhere. It seems to me almost anyone need not worry about this catch, just buffer all the data in ram and modify as necessary before calling a SPM, then write all new data sequentially. I honestly have no idea what I'm doing though, I mean optiboot does exactly what I want and was right in front of my face when I made this thread!

I can then compile code to hex and verify where the last used page of actual program is and then use all pages between it and optiboot. I can make sure I am page aligned and even use the addresses for my lookup tables in the code. It would probably be wise to make sure SPM can't access the BLS, but since SPM will always be able to overwrite my program I'll have to be super duper careful with my write calls. If I miss, I'm pulling the hardware and ISP programming to fix it. Because of this I'll probably use the PROGMEM array/compiler to handle addresses for me.  My ECU will stream 8 byte can packets in 128 byte intervals, pausing for a configurable period inbetween. It also sends table offsets for the corresponding data with every 8 bytes, so I shouldn't get anything out of order. I should be able to divide the requested write address (if greater than 128) to tell which page I am on. Then the difference between page*128 and the requested write offset should give the current ramBuffer offset. From there writing a page will look something like this, except I will fill the ramBuffer with CAN data.

    // Get all characters from the serial monitor and store it to the ramBuffer
    memset(ramBuffer, 0, sizeof(ramBuffer));
    uint16_t counter = 0;
    while (counter < SPM_PAGESIZE && charBuffer != terminationChar) 
    {
      if(Serial.available() > 0) 
      {
        charBuffer = Serial.read(); // read character from serial
        if(charBuffer != terminationChar)
        {
          Serial.write(charBuffer); // echo character back
          ramBuffer[counter] = charBuffer;
          counter++;
        }      
      }
    }
    charBuffer = 0;
    Serial.println(F("\n\nAll chars received \nWriting to flash..."));

    // WRITE RECEIVED DATA TO THE CURRENT FLASH PAGE
    // flash_buffer is where the data is stored (contains the memory addresses)
    // ramBuffer contains the data that's going to be stored in the flash
    // pageNumber is the page the data is written to
    optiboot_writePage(flashSpace, ramBuffer, pageNumber);

I do not necessarily need a initial copy in flash, part of the configurable code stored in eeprom controls weather to use raw ADC values or use the lookup table. The default will be raw, but I might program a rather useless default raw lookup table for fun. I would use something a little like your example there, I think Arduino makes it look a little more like this. But, this supposedly aligns pages and handles addressing for me.

const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM = {
0x02,0xBC,
0x02,0xBC,
0x02,0xBC,
0x02,0xBC,
0x02,0xBC,
0x02,0xBC,
0x02,0xBC,...
}

To read data from the flash in a lookup table fashion I just use a standard char array lookup. Here's one that prints all allocated space in hex. Instead I will multiply the ADC value by two, read that (and the next) array address and construct a signed 16bit value which is later broadcast over can.

      for(uint16_t page = 1; page < (NUMBER_OF_PAGES*SPM_PAGESIZE)+1; page++)
      {
        Serial.println(flashSpace[page], HEX);
      }

I won't be uploading a new table at run time, more often than not the thing will power on and read eeprom for the config file and then run while only outputting data over can, no updates to persistent memory. Every once in a while I may move a thermocouple, pressure sensor, or try and rescale my MAF table. I've also thought about a way of logging suspension position, and a linear potentiometer logging off a control arm seems easiest, I'd need a non-linear table to make sense of that but wouldn't change it once it's flashed, at least till I removed it. 

Of course I could just log the raw ADC values and use my logging software to correct the values later, what I do now, but where's the fun in that!

I also made a display gauge that has all it's settings configured through a rotary encoder menu. And I suck at coding, it's a mess but it works. Changing it after the fact sucks! Instead of using an encoder menu I am thinking I'll use this can passthru feature. If I can make it work with this stuff on a simpler 8 bit I'll stand a much better chance on the SAMD51 I am using now. ...

https://www.youtube.com/watch?v=...

Anyway, TL;DR to answer the thread title. Optiboot lets you write to flash and even provides an example. You do not need to know where your code resides in memory, a PROGMEM array will handle all addressing for you. You just need to have optiboot flashed to the default location even if you aren't planning on using uart programming. If you can (I think you must) manage a page sized ram buffer (128 bytes, 64 words on 328s) then just copy the current data and modify ram copy before using the example to write the updates to flash.

Thank you all for the advise!