Module code size limitation issue (possibly)

Go To Last Post
17 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi,

I am using WinAVR to compile and link my code.

Until today everything was working as expected. I then added a few lines of code to an existing C module (i.e. file) in my project and the hardware started resetting itself.

In order to verify whether the added code was the troublemaker, I simply removed the added code and things started to work again. The code that was added was never called during run time. Therefore, this failure is not due to a bug in the new code.

I looked a bit closer to the C file where I added the code and following is the picture I got, which made me suspicious a bit:

1 - It is the largest .c file in my project which is 322 kBytes in size.

2 - The object code generated by the same module is also the largest and its size is 154 Kbytes.

3 - The added new code does not introduce new RAM variables but simply uses the existing ones. So I don't think the problem is related to excessive RAM usage either.

I must also add that I am currently using the default Make file and have not modified anything there.

My questions are:

Q1 - Is there a C file size limitation when using WinAVR? I did search the web but did not come across such limitation.

Q2 - Is there an object code size limitation for each C file in a project?

Q3 - Is what I am seeing due to the fact that I am using the default WinAVR Make file? Do I need to modify something to allow large C files/modules to be compiled reliably?

If you have any other suggestions for the problem I am seeing, I will be grateful.

Regards...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

It is the largest .c file in my project which is 322 kBytes in size.

No single source file should be that long. Consider modularising the code. Aim for something like 2,000 lines as being the maximum complexity for any source file. Putting everything in a single file otherwise makes maintenance much trickier.
Quote:

2 - The object code generated by the same module is also the largest and its size is 154 Kbytes.

How are you measuring the 154K? You don't just mean the binary size on disk do you? That's no indication of AVR binary size. Use avr-nm, avr-size or similar to determine that figure.
Quote:

Q1 - Is there a C file size limitation when using WinAVR? I did search the web but did not come across such limitation.

Nope there is no such limit (or if there is it is so infinitely large that no mortal will ever hit it).
Quote:

Q2 - Is there an object code size limitation for each C file in a project?

Nope but note again that the size of the object bears no relation to the size of the binary. For example:

E:\sdboot-trunk\Debug>dir *.o
 Volume in drive E is VBOX_windows
 Volume Serial Number is 0000-0801

 Directory of E:\sdboot-trunk\Debug

04/10/2013  16:56            10,140 mmc.o
04/10/2013  16:56             1,696 asmfunc.o
04/10/2013  16:56             9,716 uart.o
04/10/2013  18:23            13,460 main.o
               4 File(s)         35,012 bytes
               0 Dir(s)  339,298,762,752 bytes free

E:\sdboot-trunk\Debug>for %a in (*.o) do avr-size %a

E:\sdboot-trunk\Debug>avr-size mmc.o
   text    data     bss     dec     hex filename
    678       0       0     678     2a6 mmc.o

E:\sdboot-trunk\Debug>avr-size asmfunc.o
   text    data     bss     dec     hex filename
    150       0       0     150      96 asmfunc.o

E:\sdboot-trunk\Debug>avr-size uart.o
   text    data     bss     dec     hex filename
    290       0       0     290     122 uart.o

E:\sdboot-trunk\Debug>avr-size main.o
   text    data     bss     dec     hex filename
    650       0       0     650     28a main.o

E:\sdboot-trunk\Debug>
E:\sdboot-trunk\Debug>dir *.elf
 Volume in drive E is VBOX_windows
 Volume Serial Number is 0000-0801

 Directory of E:\sdboot-trunk\Debug

04/10/2013  18:23            17,349 pfboot.elf
               1 File(s)         17,349 bytes
               0 Dir(s)  339,298,762,752 bytes free

E:\sdboot-trunk\Debug>avr-size pfboot.elf
   text    data     bss     dec     hex filename
   1514       0     513    2027     7eb pfboot.elf

Quote:

Q3 - Is what I am seeing due to the fact that I am using the default WinAVR Make file? Do I need to modify something to allow large C files/modules to be compiled reliably?

Who can possibly say until you have identified what the actual issue is?

As this is likely a GCC specific question I'll move this to the GCC forum...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

Quote:

It is the largest .c file in my project which is 322 kBytes in size.

No single source file should be that long. Consider modularising the code. Aim for something like 2,000 lines as being the maximum complexity for any source file. Putting everything in a single file otherwise makes maintenance much trickier.


Ouch--I resemble that remark. A current family of apps has main source files right around 700KB. But that ends up to be about 70KB of code size. [So I have about 10000 source characters for 1000 bytes of machine code. At an average of three flash bytes per instruction (50% mix of 1-/2-word instructions) that would be about five printed pages of whitespace and comments for each K of final code. OP has a density far greater.]

I'm a heretic in this respect. I don't add structure just for the sake of adding structure. IME AVR8 apps are "tighter" with a flattish structure and global variables.

Now, I can modularize with the best of them. Many past lives to demonstrate that. I've got my AVR roots in '8535 and '4433 apps. Not nearly as much allowance for flash space and SRAM as the kids have nowadays.

But indeed I tend to add more structure when the app size >32KB. And also, when the "team size" is greater than one. ;)

All that to comment on the OP's query. I wouldn't think that in this day and age any mainstream platform would have problems per se with the size of the source file. A simple test would be to take the biggest "working" version and add in Jabberwocky in comment block(s)...

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

How much SRAM is used by the strings in the unused code?
"resetting" probably means a jump or wrap around to zero.

Try using avr-size on both versions of your code.

"Demons after money.
Whatever happened to the still beating heart of a virgin?
No one has any standards anymore." -- Giles

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I have looked into the problem for a few more hours and saw that my RAM usage has already exceeded the maximum allowed.

I used the avr-size utility and saw that RAM usage was %105. I cannot understand how the code has been running as expected although I have exceeded the RAM capacity already!

Another thing I do not get is why the linker reports no errors on this excessive RAM use. It could have helped me detect the issue mush sooner.

Now I have go review my code and make the RAM usage as tight as possible. I am sure this will be a fun exercise !
:cry: :cry: :cry:

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
How are you measuring the 154K? You don't just mean the binary size on disk do you? That's no indication of AVR binary size. Use avr-nm, avr-size or similar to determine that figure.

Absolutely right. I have seen that during my investigations. There is no comparison whatsoever. I thought I was nearly using 50% of the flash but avr-size tells me I am well under 25% !

Quote:
How much SRAM is used by the strings in the unused code?
"resetting" probably means a jump or wrap around to zero.
Try using avr-size on both versions of your code.

Spot on comment! That is exactly what tripped me up. I created a help menu for a test command interface and lost track of how many test commands I ended up putting in there. I will need to do some Spring clean up first thing tomorrow.

Thank you all so much for coming to the rescue so quickly. I appreciate all your comments on this.

Regards...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
I have looked into the problem for a few more hours and saw that my RAM usage has already exceeded the maximum allowed.

Weren't you placing string constants in ram?

No RSTDISBL, no fun!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Brutte wrote:
Quote:
I have looked into the problem for a few more hours and saw that my RAM usage has already exceeded the maximum allowed.

Weren't you placing string constants in ram?

Perhaps this was meant to say "weren't you placing string constants in flash"

If you are not familiar with this idea have a look at the strings info @ http://www.nongnu.org/avr-libc/user-manual/pgmspace.html

regards
Greg

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
Perhaps this was meant to say

Sorry for my English and thank you for the correction. It seems that the meaning of this sentence was inverse to the intention.
What I wanted to say was that for avr-gcc OP should not declare constant strings as:

const char MyConstString[]="Ala ma kota.";

No RSTDISBL, no fun!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

Another thing I do not get is why the linker reports no errors on this excessive RAM use. It could have helped me detect the issue mush sooner.

It pools linker scripts. These "X files" cover all the AVRs:

 Directory of C:\Program Files\Atmel\Atmel Toolchain\AVR8 GCC\Native\3.4.2.1002\avr8-gnu-toolchain\avr\lib\ldscripts

06/06/2013  07:51             6,600 avr1.x
06/06/2013  07:51             6,605 avr2.x
06/06/2013  07:51             6,606 avr25.x
06/06/2013  07:51             6,607 avr3.x
06/06/2013  07:51             6,608 avr31.x
06/06/2013  07:51             6,607 avr35.x
06/06/2013  07:51             6,605 avr4.x
06/06/2013  07:51             6,607 avr5.x
06/06/2013  07:51             6,608 avr51.x
06/06/2013  07:51             6,608 avr6.x
06/06/2013  07:51             6,085 avr7.x
06/06/2013  07:51             6,041 avrtiny.x
06/06/2013  07:51             6,610 avrxmega1.x
06/06/2013  07:51             6,610 avrxmega2.x
06/06/2013  07:51             6,610 avrxmega3.x
06/06/2013  07:51             6,610 avrxmega4.x
06/06/2013  07:51             6,610 avrxmega5.x
06/06/2013  07:51             6,610 avrxmega6.x
06/06/2013  07:51             6,610 avrxmega7.x
              19 File(s)        124,457 bytes

If there were individual X files per device there would need to be 303 individual files (just as there are 303 crt*.o files).

Now I'm not sure if you've said which AVR you are building for but I see in another thread you mentioned tiny1634. If I build code for that device I see from the .map file:

c:/program files/atmel/atmel toolchain/avr8 gcc/native/3.4.2.1002/avr8-gnu-toolchain/bin/../lib/gcc/avr/4.7.2/../../../../avr/lib/avr35/crttn1634.o (exit)

The "/avr35/" in there tells me that this device is in the avr35 architecture family. These are all the avr35 devices:

 Directory of C:\Program Files\Atmel\Atmel Toolchain\AVR8 GCC\Native\3.4.2.1002\avr8-gnu-toolchain\avr\lib\avr35

06/06/2013  07:51             4,236 crta5505.o
06/06/2013  07:51             4,236 crta6617c.o
06/06/2013  07:51             4,236 crta664251.o
06/06/2013  07:51             4,632 crtm16u2.o
06/06/2013  07:51             4,632 crtm32u2.o
06/06/2013  07:51             4,632 crtm8u2.o
06/06/2013  07:51             4,588 crttn1634.o
06/06/2013  07:51             4,236 crttn167.o
06/06/2013  07:51             4,632 crtusb162.o
06/06/2013  07:51             4,632 crtusb82.o

That includes (for example) the 32K ATmega32U2.

Anyway the .x file it uses will be avr35.x. Near the top of that file you will see:

C:\Program Files\Atmel\Atmel Toolchain\AVR8 GCC\Native\3.4.2.1002\avr8-gnu-toolchain\avr\lib\ldscripts>head -n13 avr35.x

/* Default linker script, for normal executables */
OUTPUT_FORMAT("elf32-avr","elf32-avr","elf32-avr")
OUTPUT_ARCH(avr:35)
MEMORY
{
  text   (rx)   : ORIGIN = 0, LENGTH = 64K
  data   (rw!x) : ORIGIN = 0x800060, LENGTH = 0xffa0
  eeprom (rw!x) : ORIGIN = 0x810000, LENGTH = 64K
  fuse      (rw!x) : ORIGIN = 0x820000, LENGTH = 1K
  lock      (rw!x) : ORIGIN = 0x830000, LENGTH = 1K
  signature (rw!x) : ORIGIN = 0x840000, LENGTH = 1K
  user_signatures (rw!x) : ORIGIN = 0x850000, LENGTH = 1K
}

Notice in that the size of .text. It has "LENGTH = 64K". So yes, if you build more than 64K of code for a tiny1634 (four times too much!) you will get a linker error but this file has to have "LENGTH = 64K" to cater for at least the Atmega32U2 that is also in the avr35 family and possibly any future 64K device that comes along that joins the avr35 family.

You can, if you want, have a device specific .x file. Take a copy of avr35.x from the location above, copy it to your project directory, edit LENGTH= on .text to reflect your device's flash size then link with "-T localavr35.x" or whatever you called it and that local file will be used in preference over avr35.x. This way, as soon as .text exceeds LENGTH here you will get a linker error.

BTW if we really are talking about the 16K tiny1634 here then it's quite some achievement to write a 322KB source file that only generates 4KB of code (your "25%" of 16K)!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

While the linker does not report on size conflicts ( in most cases ), as you found, avr-size does. For that reason, I always include a call to avr-size as part of the build process so that I can check the flash and ram usage as I'm developing ( and so that they change as expected ). By default a section oversize will not cause the compilation/link to fail, but it would not be that difficult to parse the avr-size return and break on greater than 100 percent ( or, some other threshold ) values to ensure that the program is running with sufficient headroom.

Martin Jay McKee

As with most things in engineering, the answer is an unabashed, "It depends."

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

avr-size does.

But it doesn't set an ERRORLEVEL does it? I think it'll just happily report 105% but won't cause an error condition because of it.

I believe this is why, in AS6, Atmel wrote their own tool to post-process the output of avr-size and to finally create such an error when the size of the part being built is exceeded.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi All,

Thank you once again for these extremely helpful comments. With such help at hand, no problem looks too big :)

@gregd99
Thank you for that link. Yesterday while trying to understand what the hell was wrong, I did come across that same link via other sources. However, I clicked on the one you provided to shift the strings to the program memory just now and it looks great! Managed to get the RAM usage down to 79% from a whopping 105%. I will look at my code to spot if further savings will be possible.

By the way, I was not placing the constants to flash earlier because I did not think they would grow to be so large. It's one of those things you leave til later when such issues arise!

@clawson
I actually did not tell you the AVR device I am working with. Sorry... I am actually using Atmega2560 at the moment. You have probably spotted my notes on tiny1634 which were for a future project. The Atmega2560 will keep me busy for a while longer while debugging the new hardware I built.

Thank you so much for the pointer on getting a linker error prompt. Your instructions are clear and make perfect sense. I will see if I can get that to work.

@mckeemj
The instructions you have provided will require some mods on the makefile I presume. At the moment, I call the avr-size from a command prompt after the build is completed. It will be great to get it reporting automatically as you have explained. Can I implement that break condition you have described as part of the makefile? I have not dealt with makefiles that much but I think it will be necessary to simplify life at some stage.

Regards...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
But it doesn't set an ERRORLEVEL does it? I think it'll just happily report 105% but won't cause an error condition because of it.

I believe this is why, in AS6, Atmel wrote their own tool to post-process the output of avr-size and to finally create such an error when the size of the part being built is exceeded.

[shameless self-advertisement]My crude checkmap utility (which, as its name hints, postprocesses the mapfile output by linker), does. And does more... ;-) [/shameless self-advertisement]

JW

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

@clawson
Correct. There is no ERRORLEVEL value set on section overflow, so, while the processed value contains the desired information, it is not in a form that can be used, directly, to control a build script. An actual break would indeed need a separate utility, or, at least, I wouldn't want to attempt it in make alone. It needn't be difficult with a bit of Python though.

Martin Jay McKee

As with most things in engineering, the answer is an unabashed, "It depends."

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

@wek

Thank you for that link to the utility. I tried to have a go at using it. Following are my comments and status:

1 - Build stage

I invoked the nyweb.exe checkmap.w as instructed in the readme file and got the following output. I am not sure this is what I should be seeing. It does not look like this build has been successful as I don't know what to expect.

-- Utils to walthrugh

-- Walkthrough
1. find main()
2. recursive walkthrough (checking for recursion in the call tree to avoid infinite loops)
3. sort? and outpu results
4. deallocate memory

--------------- changelog (sort of)
2009 - first two versions, second done specifically to be able to display individual functions' resource usage
7.2011 - v1.2
- list file processing
- callgraph/stack "analysis"
7.2012 - v1.3
- fix - "numeric" labels are local too

I then compiled the checkmap.pas by invoking the "fpc checkmap.pas" and the checkmap.o object file was created. So I presume that step was successful.

2 - Usage
Here I will need a config file. I do not have a cfg file of my own so I will need to use the default one that is provided. What I am not sure is whether I need to make any changes to it based on the Atmega 2560 device I am using. If I need to make any changes, what are they? The current cfg file is as follows:

*** example config file for checkmap utility
*************************************************
* section lo hi
*************************************************
.progmem 0 0x10000 * progmem is read through LPM - can't be over 64kB
.text 0 0x10000
.data 0x00800060 0x00800220
.bss 0x00800060 0x00800220
COMMON 0x00800060 0x00800220
.noinit 0x00800060 0x00800220 * end of RAM - reserve for stack 0x40 bytes

Given the above uncertainties, I invoked the checkmap.exe with my map file input anyways and following is what I got:

>checkmap.exe CagApp_AS4.map
Error: in section .data high limit 0x00800220 violated by value 0x00800262, in map file line 740.

I checked my map file and according to the above error message lowPriorityTasks variable (as seen below) is the culprit. However, I am not sure if this really is a problem and I need to change anything in my code.

.data 0x00800200 0x82 WhizOS.o
0x00800262 lowPriorityTasks
0x00800222 highPriorityTasks
0x00800242 mediumPriorityTasks

Your feedback and suggestions on the above will be greatly appreciated.

Regards...