Remarks on using avr-gcc on 128/256kB FLASH AVR devices

Go To Last Post
19 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The following text assumes fair familiarity with avr-gcc and its suite (including binutils - namely avr-as and avr-ld - and avr-libc).

One of the key issues in using avr-gcc is the lack of memory classes (a.k.a. named address spaces), i.e. for a variable, it cannot be specified, in which type of memory (RAM, FLASH, EEPROM - all have different methods of access) it is located. (This problem is aggravated when using pointers, but we don't get into that level of details here). This is due to gcc's majority usage as for "big, uniform (von Neumann) address space" processors, and is unlikely to change dramatically in the foreseable future. Even if some named address spaces support are to be built in in forthcoming versions (see for example https://www.avrfreaks.net/index.p... ), it's still a couple of months if not years from now. Avr-libc provides a workaround - (and associated functions in library) for FLASH and for built-in EEPROM (we are not going to deal with EEPROM now, that's relatively problem-free).

Although a kludge, and say handling of strings and pointers located in FLASH is not quite trivial, with a bit of imagination it can be get working. A simple example, we program a single number (byte) in FLASH and we read it out in this way:

uint8_t PROGMEM v = 5;
uint8_t a;
[...]
a = pgm_read_byte(&v);

Note, that pgm_read_byte takes the address into FLASH as its argument. (One more note: variables with PROGMEM attributes cannot be used as local variables, they have to be either global or tagged "static").

This works reasonably well until 64kB of FLASH. Above that, to address a FLASH byte, one needs more than 16 bits, which is the width of pointers avr-gcc uses. It means, that one cannot take the address of a FLASH variable above the 64kB limit. Say, if there are too many variables in the .progmem section (to which these variables are located through the PROGMEM macro), and variable v gets located say at 0x10123,

 a = pgm_read_byte(&v);

will read from FLASH address 0x0123 and not even issue a warning.

There is a workaround in : a class of functions/macros, accessing all the FLASH, pgm_read_xxx_far. Problem is, these take a 32-bit address, while pointers are only 16-bit, so it may seem we will need to allocate these variables on a fixed address by hand. Carlos Lamas came with a solution: in the header "morepgmspace.h", there is a macro GET_FAR_ADDRESS() defined, and this returns the 32-bit address of an object, if it is a constant known at compile-time. Our example from above would then turn to:

a = pgm_read_byte_far(GET_FAR_ADDRESS(v));

which finally works as intended.

Carlos also provided a library of functions similar to those prototyped in which in turn came from the standard , suffixed by _PF, such as memcpy_PF() and similar (no printf_PF() and similar, though). Unfortunately, his work is still not included in the official issue of avr-libc.

Could we then use uniformly pgm_read_xxx_far() and family, and forget the whole issue? Unfortunately, no. First, the macro and the functions are more resource-hungry (registers, program space) than the "original" ones. Second, the GET_FAR_ADDRESS macro won't work on objects which address is known only at runtime, e.g. array members with variable index. The workaround is to calculate the address "manually" from taking the base using the macro and adding the offset, for example:

uint16_t PROGMEM v[10] = 
   {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
uint16_t a, i;
[...]
// a = pgm_read_word(&v[i]);
a = pgm_read_word_far(GET_FAR_ADDRESS(v[0]) + i * sizeof(v[0]));

If the array is a member of a struct, the function offsetof() prototyped in can be used for the same purpose a bit more comfortably.

But there are even more problems, although of a rather subtle form, when the FLASH gets above the 128kB limit. This is associated not with FLASH data as such, but with the addressing scheme the program uses: as the AVR execution core fetches always 2 bytes, instructions must be aligned on a word boundary, hence the 16-bit pointers can be mapped to program jump labels below 128kB. There is no problem with "pure" code - the linker internally works with 32-bit addresses, the JMP/CALL instructions are capable of addressing up to 22 bits, CALL/RET specifically on these devices stores/restores 3 bytes to/from the stack, so everything works fine. The problem is with indirect jumps, which boils down to two cases: some "switch" commands executed through jumptables, and function pointers. Both *are* solved in newer editions of avr-gcc and suite, through a trampoline mechanism, i.e. if one takes address of a function, in fact is given address of a "stub", which is a JMP located at the lower addresses, jumping to the function potentially located at the higher addresses. These stubs are then all collected into a linker section .trampolines, which comes after the .progmem section (with the argument that while .progmem data must be below 64kB to pgm_read_xxx() work properly, .trampolines may be above 64kB as long as it is below 128kB). If somebody inflates the .progmem section (reading from it using pgm_read_xxx_far()) so that (part of) the .trampolines gets above 128kB, the whole purpose of trampolines gets defeated and it stops working. (There may be more similar issues I am not aware of at the moment).

If we know beforehand we will have a lot of FLASH data and we plan to access them using the pgm_read_xxx_far() family, it is a good idea to place them not only above the .trampolines, but above any code (.text) at all, so that as much functions as possible remain in the precious lower 128kB area. To achieve that, we need to modify the default linker script ([WinAVR]/avr/lib/ldscripts/avr6.x for the 256k devices, avr5.x and avr51.x for the 128k devices, see chapter "13.1.1 Machine-specific options for the AVR" in avr-libc documentation for proper assignment of devices to "avrN" groups). First, make a backup of the file. Search for .fini0, and insert the following line (line preceding and following shown too):

    KEEP (*(.fini0))
    *(._progmem_far)  /* constant FAR data in FLASH, accessed by ELPM */
     _etext = . ;

Once modifying the script, add also the following lines the purpose of which we will discuss a bit later (again, one preceding and following already existing line shown):

  }  > data

  /* segmented constant FAR data in FLASH, accessed by ELPM */
  .progmem_segment1 0x10000 : {
    *(._progmem_segment1)
    __progmem_segment1_end = .;
  }  > text
  .progmem_segment2 0x20000 : {
    *(._progmem_segment2)
    __progmem_segment2_end = .;
  }  > text
  .progmem_segment3 0x30000 : {
    *(._progmem_segment3)
    __progmem_segment3_end = .;
  }  > text


  .eeprom  :

These changes, together with macro PROGMEM_FAR defined in the modified morepgmspace.h from the attachment, can be used to locate large FLASH data to the "high" memory. Usage is little different from the case when variable was in .progmem but above 64kB:

uint16_t PROGMEM_FAR v
uint16_t a;
[...]
a = pgm_read_word_far(GET_FAR_ADDRESS(v));

The purpose of "segmented" usage is to allow for faster and more comfortable address calculation, if we know beforehand that we can fit all FLASH data into a 64k "segment", (segment1 is 0x10000-0x1FFFF, segment2 is 0x20000-0x2FFFF, segment3 is 0x30000-0x3FFFF). To access them, we still need the pgm_read_xxx_far() (and xxx_PF()) family, but we can use the "native" pointer mechanism and then add the segment address, e.g.

uint16_t PROGMEM_SEG1 v[10] = 
   {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
[...]
  a = pgm_read_word_far(PROGMEM_SEG1_BASE + (uint16_t)&v[i]);
[or]
  a = pgm_read_word_seg1(&v[i]);

There is a set of optimised assembler macros provided for reading byte/word/dword from segment1, pgm_read_xxx_seg1() (and can be easily modified also for segment2 and 3); however, the first mechanism is still to be used for the xxx_PF() functions.

For both PROGMEM_FAR and PROGMEM_SEG1, there are macros similar to PSTR() to create a string locally, PFSTR(), PS1STR(), PS2STR(), PS3STR. These again shall be used with the xxxx_PF() family of functions. As there is no printf_PF() which is most probably the most often demanded function for strings, the following hack can be used as a workaround:

#define localPFSTR(s) (({char __c[strlen(s) + 1]; strcpy_PF(__c, PFSTR(s)); &__c[0];}))
[...]
  printf(localPFSTR("bbb"));
} 

The .progmem section could still be usefule for a smaller amount of FLASH variables to be accessed by the relatively less costly "classic" methods. It is a good idea to check whether .progmem; I've written a utility which reads the .map file and throws an error if a checked section is above a planned size.

Comments, please.

Jan Waclawek

Attachment(s): 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

How is this handled in the XMEGA devices with large linear address spaces? Gracefully? I will eventually find out, since the programming I am working on is the first step in migrating my code to an environment where I can use either MEGA or XMEGA.

I'd rather see this kind of mechanism built into the language. Even if we have to declare a data array "far", and do it manually, the manipulation of the pointers would perhaps be more elegantly done off stage.

Harvey

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

NightSky wrote:
How is this handled in the XMEGA devices with large linear address spaces? Gracefully?

I haven't worked with the xMegas yet, just had a brief look at the docs; but I doubt the "Harvardeness" did change. In other words, the FLASH is still a separate address space from register/IO/RAM/XRAM, so the kludges are still needed.

NightSky wrote:
I'd rather see this kind of mechanism built into the language.
Sure, so do I. But I had to live with what I had and couldn't wait for a year or two to this to happen (nor could I invest the time and effort to dwelve into gcc and try to fix things for myself, not that I have the capability to do that).

JW

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Jan,

AFAICS the one difference the (A1) Xmeags have is the EBI which provides "wide" addressing so, like some devices flash space, it is now possible to have a RAM region larger than 64K which means that it starts to suffer from 16bit pointer width issues like the flash already did.

Cliff

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Oh I see, it's the other way round than I though (I thought the FLASH might have been merged into a large uniform data space, which is not the case).

Yes, a similar set of kludges might be needed to be able to use XRAM above 64k. I'd not see this as a limitation: anybody needing large data spaces should see that as it is: a large data space, with a need for special access. Those who mistake AVRs for general purpose processors with native large data spaces should consider to move on, there are ample of choices nowadays (or, more precisely, and sadly, they are gradually narrowing down due to the immense advertisement push from the ARM licencees).

JW

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Just looking at it, the EBI allows access to up to 16 megabytes (or more?) of memory, depending.

The chip has the capability of looking at that large address (data) space. The problem is if the language ever does.

I don't think that the flash is merged, the EEPROM and I/O registers are merged, (EEPROM optionally).

If Atmel wants people to use the XMEGA chips successfully, and without workarounds, then either GCC will have to change (and other chips have similar problems, and I bet, have them solved), or Atmel dumps GCC, but still must come up with a free (or *VERY* inexpensive) solution to the problem.

Not yet having a working XMEGA design (have two schematics, need to make the boards or use the XPLAIN board to test bed the programs), I can't comment. Also need to keep rewriting code to make all this good stuff work.

Harvey

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

NightSky wrote:
If Atmel wants people to use the XMEGA chips successfully, and without workarounds, then either GCC will have to change (and other chips have similar problems, and I bet, have them solved), or Atmel dumps GCC, but still must come up with a free (or *VERY* inexpensive) solution to the problem.
Ah, but the problem here is not Atmel. Atmel's "primary" compiler is not GCC. In fact, if you look at the code they supply for their App Notes, you'll see it is written for the commercial IAR compiler, not GCC.

Professional programmers accept that their tools will cost something to obtain. Even Micro$oft's Visual Studio suite is inexpensive compared to, for example, Texas instrument's compiler for their DSPs or the ARM compiler provided by some SOC manufacturers. Hey, compiler writers need to feed the kids and make rent payments just like anyone else. To demand free tools but then turn around and demand payment for the applications developed with those tools is hypocritical.

GCC has the advantage that it costs the user (you) nothing in dollars/euros/frog-skins to purchase. In return, the user (you) must accept that GCC is written for many platforms and thus suffers from compromises made for those other platforms. Also, the user (you) must accept that practically all development on the compiler, the port, and the library is done by folks who do it for the fun and the glory (of which there's damned little) of doing it. And since the pay is so good (not!), there are always too few of the volunteers.

If you think this sucks, you have two options: First, you can purchase one of the commercial compiler alternatives to GCC. Or second, you can volunteer your time to the GCC project to make the Atmel port more akin to your liking. Whining that "Atmel should do something about this" is unlikely to get much action.

Stu

Engineering seems to boil down to: Cheap. Fast. Good. Choose two. Sometimes choose only one.

Newbie? Be sure to read the thread Newbie? Start here!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You have third option – not to use AVRs. For the same price you have a plenty of options – ARM for example. So it is Atmel problem to support users with good compiler.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hmmm, well, if Atmel wishes to have low cost support for the great grand unwashed multitude who buy their chips, then they're going to need a low cost (or free) C compiler.

Given that they do provide support for it in Studio, do provide examples written in GCC, I was under the impression that it was supported in some semi official way.

Based on Stu-san's mention (I think it was he) that Atmel might be dropping GCC for something else (hopefully with the same low cost), then I kept on thinking it was officially supported.

Analog Devices DSP chips are inexpensive, I have one or two. The prices on their compilers are about 2500 dollars, so I guess I won't be using the chip.

I could write an assembler for it, but I've got other things higher up on the list.

Harvey

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

Atmel might be dropping GCC

Atmel can't drop it. They have not picked it up.

As for offering a "really cheap tool chain" then I'm all for it, and the hobbyists would propser from it. But I guess that for Atmel a hobbyist guy like me, or someone pushing a few AVR based devices on the market every month is not of interest. As someone formulated it here a while back: "If youre a microwave oven manufacturer wanting 100K AVRs" then you're of genuine interest to Atmel. And if you are that big, the cost of the compiler is a wee in the Mississippi.

Now I'm only speculating :wink: , but if Atmel would support a free C tool chain in the future I would guess that it would be gcc- and avrlibc-based. As the WinAVR install package has been announced to have seen it's last release, such a gcc-based tool chain released by Atmel itself would be very fitting.

Hey, wait... Maybe they will get it ready in time for the next major AVR Studio, that we have seen the heads up for... Hey, wait... What was the name of that guy doing the WinAVR install package, and where does he work? Uhmmmm...

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

Hmmm, well, if Atmel wishes to have low cost support for the great grand unwashed multitude who buy their chips, then they're going to need a low cost (or free) C compiler.

Rubbish, Atmel aren't interested in the 50's, 500's or even 5000's they sell to small operations or hobbyists. They are interested in the big contracts selling 1/10/50 million to car and domestic goods manufaturers. Those developers will think nothing of spending $3000 for copies of IAR if that's what's needed.

Sure GCC helps to get hobbyists interested in AVRs (because it is "free") and the hope for Atmel is that when these people graduate they'll then persuade their 1/10/50 million using employer to consider using AVR. But I don't think it's vital that GCC be at the leading edge of LARGE/HUGE memory model programming or Harvard support for AVR.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
and the hope for Atmel is that when these people graduate they'll then persuade their 1/10/50 million using employer to consider using AVR.
I have always asked myself how well that strategy works in general.

I have a black list of companies in my mind I never consider buying from. Some of the oldest "entries" are from my student times.

And I have a similar gray list in my mind (avoid if possible). Again, some old "entries" are from my student times.

And unless my memory fails, both lists are well defined and I hardly fail to remember if a company is on one of that lists in my mind.

But a white list? I don't think I have a clearly defined company white list in my mind. I do prefer kinds of products from some companies, but can do with other companies, too. And it is usually not all products of a company, but just one group or even one product I prefer.

When I was a student companies were of course coming on campus to win our hearts and minds. And they sponsored labs, gave books away (these things made from paper) and did roadshows. But non made such a lasting impression that I would say I would by blindly from that company.

Stealing Proteus doesn't make you an engineer.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
And they sponsored labs, gave books away (these things made from paper) and did roadshows. But non made such a lasting impression that I would say I would by blindly from that company.

I can remember visiting a design consultancy in San Diego (a lovely place!) called Doctor Design who were vying to get part of the work for a digital TV system in Europe, they gave my colleague and I jackets, mugs, towels and various other "goodies" when we visited - we were almost swayed right there to give them part of the multi-million dollar contract ;-)

(in the end other factors overtook our decision and the system was based on OpenTV in San Jose - they didn't give us half as many giveaways when we visited - so personally we weren't too pleased)

[I can't find the same "Doctor Design" on the internet any more so I assume they've gone their separate ways or been absorbed into a larger corporation...

...ah found them - Wind River bought them and they are now "Wind River Services" - I remember they were huge VxWorks fans so I guess that makes sense]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Atmel has shown 'some' interest in hobby use of their products in the Dragon. They have even expanded the Dragon to >32K and it now supports some of the AVR32 and xmega family, something I hadn't expected. Full xmega PDI support may even be in the works. Microchip has introduced lower cost tools (pickit and open source C compiliers) so Atmel is playing catchup. Atmel's online store is also a kneejerk reaction to Microchip doing the same.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

open source C compiliers

Sorry for off-topic but tell me about that? It seems to me that for PIC18 the only options are C18 or Hitec. I have C18 installed. Meanwhile Atmel have clearly been promoting the development of avr-gcc since they employed Eric several years ago (and possibly before that).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I remember reading a while ago that Microchip had released some development code as open source. I can't find it now, but I think it was for the larger pic's not the 16 family. There are 'free' (as in beer) versions, and one that works under Linux too.
http://www.piclist.com/techref/m... I know MC did open source the PicKit2 drivers, the drivers for Atmels JTAG (Avarice) had to be reverse engineered somewhat as Atmels documentation was not complete.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Microchip's in-house C compiler offerings for PIC24/dsPIC and their MIPS-based (PIC32) devices are both based on GCC. The evaluation versions are available for free, but they lose certain optimization options after 60 days. Fully functional versions require a purchase.

(They have complied with all the relevant GPL requirements, and released the source code for these compilers. With the source code it's possible to remove the pieces which would differentiate the free version from the fully functional version -- except there is reasonable grounds for concern that Microchip would not give permission for you to use their proprietary-licensed runtime libraries with such a set-up.)

(Or you can buy the Hi-Tech compilers, which also belong to Microchip as a wholly owned subsidiary. But Microchip seems to recommend that everybody should be using the GCC-derived compilers for dsPIC and PIC32 for all new designs.)

For PIC18 and below, I don't know of Microchip having endorsed any open source compilers.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

For PIC18 and below, I don't know of Microchip having endorsed any open source compilers.

And it's PIC16/18 that compete with AVR8s, not PIC24 or dsPIC

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well the PIC24/32 family compete with AVR32 and SAM7