Linker

Go To Last Post
24 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi,
I am trying to implement the u8glib libraries for my mega2560.
It all compiles, the problem is too much compiles. Apparently I am using up 330% of my flash!

The author of the library told me that I need to set up these because it is building more of the files than it should:

Quote:

gcc options during compile step:

-ffunction-sections
-fdata-sections

gcc options for linking:

-Wl,--gc-sections


The -f ones seem to be to do with optimization, and I have put them under there, the -W I think is no debugging, so I have set that, its just the last one Im not sure where it goes, or what its to do with (which might give me a clue).

Im using Eclipse.
Thanks

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Fixed it.
--gc stands for general...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

gc stand for garbage collection. See recent thread in GCC forum about this very thing. You've just proven why it's often a bad idea for library design!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

No, he's proven why those options should be enabled by default.

Sid

Life... is a state of mind

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yeah I did actually have a google, but it comes up with very litte (when all you have to go on is the statement).

Whats it actually doing?

Stab in the dark. Could this be the thing that is causing me problems trying to build LUFA?

../LUFA-120730/Projects/USBtoSerial/Descriptors.h:83: error: conflicting types for 'CALLBACK_USB_GetDescriptor'
../LUFA-120730/Projects/USBtoSerial/../../LUFA/Drivers/USB/Core/AVR8/../Device.h:133: error: previous declaration of 'CALLBACK_USB_GetDescriptor' was here
../LUFA-120730/Projects/USBtoSerial/Descriptors.c:55: error: 'FIXED_CONTROL_ENDPOINT_SIZE' undeclared here (not in a function)
../LUFA-120730/Projects/USBtoSerial/Descriptors.c:65: error: 'FIXED_NUM_CONFIGURATIONS' undeclared here (not in a function)
../LUFA-120730/Projects/USBtoSerial/Descriptors.c:217: error: conflicting types for 'CALLBACK_USB_GetDescriptor'
../LUFA-120730/Projects/USBtoSerial/../../LUFA/Drivers/USB/Core/AVR8/../Device.h:133: error: previous declaration of 'CALLBACK_USB_GetDescriptor' was here
make: *** [LUFA-120730/Projects/USBtoSerial/Descriptors.o] Error 1

What I mean by that is there are some Linkers I need to include?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ChaunceyGardiner wrote:
No, he's proven why those options should be enabled by default.
What linkers do that by default? Does Visual Studio do that? I am pretty sure old versions did not. Maybe there are some reasons behind that? Compatibility, you know?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

Whats it actually doing?

As I said there was a thread about this the other day

https://www.avrfreaks.net/index.p...

Did your search not find that?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ezharkov wrote:
ChaunceyGardiner wrote:
No, he's proven why those options should be enabled by default.
What linkers do that by default? Does Visual Studio do that? I am pretty sure old versions did not. Maybe there are some reasons behind that? Compatibility, you know?

Compatibility with what ? Something that requires the inclusion of unused functions ?

It is completely ridiculous that in 2013, a library developer is forced to create one source file per function.

I understand that a handful of people think that's a good way to structure their source code, but I have yet to see any good reasons for forcing this system upon everybody else.

It's a circular argument. You should structure your source code like that because of the lame way the tools work, and the tools have to be lame like that because some people like that sort of file structure.

It is possible to make something that works for everybody.

Sid

Life... is a state of mind

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ChaunceyGardiner wrote:
Compatibility with what ? Something that requires the inclusion of unused functions ?
Not "unused functions", but "unreferenced sections". There are more than one example for that.

ChaunceyGardiner wrote:
It is completely ridiculous that in 2013, a library developer is forced to create one source file per function.

I understand that a handful of people think that's a good way to structure their source code, but I have yet to see any good reasons for forcing this system upon everybody else.

Neither does the "one source file per function" method force anything to anyone, nor is a library developer forced to that method (*1). YOU are the one who tries to force something to everybody else (namely using always a specific option when linking). And your argumentation for that forcing is a bad joke: "The option is good for the majority anyway. There are alternatives preserving the freedom of choice (*2), but I don't like them, so dash this freedom. Let us simply ignore the remaining minority. If they want to do something that conflicts with my own habits, then it must be something very unusual anyway. Their problem how to achieve that together with that forced option."

(*1) Georg-Johann has already shown an alternative in the other thread, what you silently ignored. But don't bother to look at that in detail, I can predict that you won't like that either.

(*2) a "traditional" library does not prevent using that option

Quote:
It's a circular argument. You should structure your source code like that because of the lame way the tools work, and the tools have to be lame like that because some people like that sort of file structure.
Another "great" example of your argumentation. The second part ("and the tools have to be lame") is nonsense. As if "not lame" tools would not allow that sort of file structure.

Stefan Ernst

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sternst wrote:
YOU are the one who tries to force something to everybody else (namely using always a specific option when linking).

Where did I do that ? I don't think these options should always be used. I think they should be enabled by default.

That would allow a library developer to distribute his library in confidence that any user that includes unused stuff in his executable has actively chosen to do so.

sternst wrote:
And your argumentation for that forcing is a bad joke: "The option is good for the majority anyway.

How is that a joke ? It is a fact that for the vast majority of AVR programs, unused stuff is not desirable in the executable.

sternst wrote:
There are alternatives preserving the freedom of choice (*2), but I don't like them, so dash this freedom. Let us simply ignore the remaining minority. If they want to do something that conflicts with my own habits, then it must be something very unusual anyway. Their problem how to achieve that together with that forced option."

You're lying. That is not what I want, I have argued for keeping the options we already have in order to allow the tiny fraction of code that relies on this to still work.

The only difference is that if those options were enabled by default, a handful of people would have to disable those options once in a blue moon, instead of everybody else being forced to enable them all the time to take advantage of important tool features that the vast majority of programs would benefit from.

sternst wrote:
(*1) Georg-Johann has already shown an alternative in the other thread, what you silently ignored. But don't bother to look at that in detail, I can predict that you won't like that either.

Who is Georg-Johann ? SprinterSB ? If you're talking about the last message he posted in that thread, I didn't ignore it I just didn't understand it. He states that libgcc.a contains 800 functions and is built from just 4 source files and yet does not suffer from the problem we have been discussing. He didn't bother explaining how this is done, though, and I know from experience that he doesn't like explaining things that he deems self-evident.

sternst wrote:
Quote:
It's a circular argument. You should structure your source code like that because of the lame way the tools work, and the tools have to be lame like that because some people like that sort of file structure.
Another "great" example of your argumentation. The second part ("and the tools have to be lame") is nonsense. As if "not lame" tools would not allow that sort of file structure.

My point, precisely.

Sid

Life... is a state of mind

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

He didn't bother explaining how this is done

Oh come on, the source is open, anyone can view it. Without looking I rather suspect it's going to be a lot of #if/#ifdef.

EDIT:

http://gcc.gnu.org/viewcvs/trunk...

That does seem to be quite a lot of:

 	#if defined (L_??????)

and the building looks like it centres around this:

http://gcc.gnu.org/viewcvs/trunk...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ChaunceyGardiner wrote:
sternst wrote:
YOU are the one who tries to force something to everybody else (namely using always a specific option when linking).

Where did I do that ? I don't think these options should always be used. I think they should be enabled by default.
Are you kidding? Do you still don't get the point? Creating libraries with -ffunction-sections (what you propagate) forces every user of that libraries to use --gc-sections.

Quote:
The only difference is that if those options were enabled by default, a handful of people would have to disable those options once in a blue moon,
And these people then can no longer use any library created the way you propagate. Your proposal to use -ffunction-sections for libraries turns this option into a requirement. A "traditional" library doesn't exclude anyone.

Quote:
sternst wrote:
Quote:
It's a circular argument. You should structure your source code like that because of the lame way the tools work, and the tools have to be lame like that because some people like that sort of file structure.
Another "great" example of your argumentation. The second part ("and the tools have to be lame") is nonsense. As if "not lame" tools would not allow that sort of file structure.

My point, precisely.
No, the point with this "circular argument" is that you put a silly argument into other peoples mouth. You convey that this "circular argument" is used to justify the status quo. But no one did that. YOU created this nonsense.

Stefan Ernst

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Imagine a set of tools that produces a library from "normal" source files, a library which is identical to the library it would create if each function was in a separate file.

That seems very desirable, and looks like something that should be pretty easy to create.

Using separate source files - or compiling each source file repeatedly with different preprocessor symbols - is the way it had to be done in the stone age. Some platforms and some developers have moved on since then.

Sid

Life... is a state of mind

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ChaunceyGardiner wrote:
Some platforms and some developers have moved on since then.
Maybe this is a business opportunity for you - develop and sell your own non-ridiculous 21-st century tools?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ChaunceyGardiner wrote:
Imagine a set of tools that produces a library from "normal" source files, a library which is identical to the library it would create if each function was in a separate file.

That seems very desirable,

Of course. The result is what I call "traditional library" all the time. What I argue against is the proposal to "emulate" similar behavior with -ffunction-sections. The result of that is a different kind of library.

Quote:
looks like something that should be pretty easy to create
Perhaps. As always with Open Source, if it is not there, then because no one felt motivated enough to implement it. So the simple truth could be (don't really know, I can speak only for myself), that the majority of C programmers don't have the same huge pain with "stone age methods" than you have.

Stefan Ernst

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ezharkov wrote:
ChaunceyGardiner wrote:
Some platforms and some developers have moved on since then.
Maybe this is a business opportunity for you - develop and sell your own non-ridiculous 21-st century tools?

I don't understand why you think that would be a good idea. I would obviously not find any customers in here, so where would I find them ?

Sid

Life... is a state of mind

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

From the GGC Manual (emphasis mine)...

Quote:
-ffunction-sections
-fdata-sections
Place each function or data item into its own section in the output file if the
target supports arbitrary sections. The name of the function or the name of
the data item determines the section’s name in the output file.
Use these options on systems where the linker can perform optimizations to
improve locality of reference in the instruction space. Most systems using the
ELF ob ject format and SPARC processors running Solaris 2 have linkers with
such optimizations. AIX may have these optimizations in the future.
Only use these options when there are significant benefits from doing so. When
you specify these options, the assembler and linker will create larger object and
executable files
and will also be slower. You will not be able to use gprof on all
systems if you specify this option and you may have problems with debugging
if you specify both this option and ‘-g’.

Oops. :oops:
Someone didn't RTFM.
That pops one argument off the stack, lets address the others.

----------------------------------

Some have made the claim that the tools must be 'broken'. :shock:

I use a straight blade screwdriver on all screws. It often works. When it doesn't work, it must be that the screwdriver is faulty.

It is not a matter of faulty tools, it is a matter of using the correct tool for the task.
These tools, these options, were not designed for the task some are advocating.

I happen to think the tools are OK.
Certainly, not perfect.
But not broken as claimed, they work well for the task they were designed for.

If one bothers to RTFM, one would know which were the right tools for the job and, since those tools are freely available, there is little excuse not to use them.

If you really feel the tools are broken, you should bring up the issue here... http://gcc.gnu.org/bugzilla/

Better yet, fix the 'problem' yourself ...http://gcc.gnu.org/contribute.html#patches

I'm sure we all will be waiting breathlessly for your results.

POP _broken_argument;
----------------------------------------

These options sometimes break things.
Not because the options themselves are defective.
Because what they do is often not compatible with the developers intent.
That is why, these options are... optional.

Some argue these options should be enabled by default.
Even if they were default, they would still be... optional.

Some argue that they should be mandatory. I happen to disagree, but...
As long as they are options, serious programmers must treat them as... optional.

POP _default_argument;
POP _mandatory_argument;
--------------------------------------

The Arduino Library is not the same animal as an archive library, but will serve to illustrate an important point.

The Arduino Way works. I think it is fair to say it works pretty well.

It works because Arduino controls everything.
Arduino provides the IDE, the compiler, linker, all the tools.

Arduino also controls the build process.
Of specific relevance here, Arduino controls the compiler and linker options too

Without that kind of control, the 'Arduino Way' does not work as well.

When distributing an archive library, you do not have that sort of control.

POP _control_argument
--------------------------------------

Some are complaining about the extra work required to build a library the traditional way.

While it is indeed a tiny bit more work, that burden should be on you, as the library developer.

But rather than offer a solid product, you would rather save yourself that tiny bit of effort, and put the burden on your customer to find a way to make your product work for them?

That smacks of laziness and selfishness.

POP _crybaby_argument;
--------------------------------------

Some have argued that one should not be forced to write a library the traditional way.

You are right, nobody should be forced to do so (as long as they are working for themselves).

And, nobody is forcing you to do so (unless you are working for me).

You have been advised, that's all. If you wish to use the wrong tool for the job, we all wish you the best of luck.

(P.S. You're fired) :lol:

POP _forced_argument
---------------------------------------

Some have made the incredible statement that, without the use of these options, unused functions and data must therefore be included in the final build.

Not only is this utter nonsense, it betrays a complete lack of understanding of the tools and how they work.

POP _unused_functions_argument
----------------------------------------

Some have argued that making section garbage collection the default, or even mandatory, would 'benefit the 99%', because it removes 'dead code'.

I must say, I do not believe that serious programmers write dead code 99% of the time.
Nor can I believe that 99% of serious programmers write dead code.
Hell, I can't believe even that 99% of rank, amateur student code contains any dead code.

Yes, there are times when it happens. Existing compiler warnings give plenty of indication of this, and serious programmers will take heed of these warnings, because they want to write good code.

So, I must assume that by 99% you must mean beginners who have not yet learned any better, and sometimes produce dead code... until they learn better.

You seem to be advocating a major change in the tools, which will severely affect the ability of almost all serious programmers to write reliable code for interrupt service routines, device drivers, jump tables... as well as require re-writing of thousands of thousands of lines of pre-existing library code.

For what? It will simply encourage the students and amateurs to continue to produce crappy code, that's all.

POP _majority_argument
---------------------------------------

Some have accused me of bias.

A traditional library works.

One that relies on garbage collection: not always

I must admit that I am biased.

POP _biased_argument;

EOF;

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

mikericetga wrote:
From the GGC Manual (emphasis mine)...

Quote:
-ffunction-sections
-fdata-sections
Place each function or data item into its own section in the output file if the
target supports arbitrary sections. The name of the function or the name of
the data item determines the section’s name in the output file.
Use these options on systems where the linker can perform optimizations to
improve locality of reference in the instruction space. Most systems using the
ELF ob ject format and SPARC processors running Solaris 2 have linkers with
such optimizations. AIX may have these optimizations in the future.
Only use these options when there are significant benefits from doing so. When
you specify these options, the assembler and linker will create larger object and
executable files
and will also be slower. You will not be able to use gprof on all
systems if you specify this option and you may have problems with debugging
if you specify both this option and ‘-g’.
I get why the object files would be larger: more data.
Why would they be slower?
What's the problem with grpof and debugging?

"Demons after money.
Whatever happened to the still beating heart of a virgin?
No one has any standards anymore." -- Giles

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

skeeve wrote:
I get why the object files would be larger: more data.
Why would they be slower?
I would think that "be slower" is related to "assembler and linker".
Quote:
the assembler and linker will ... and will also be slower.

Stefan Ernst

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Assembler and linker may be slower.
The final application will ( hopefully ) run faster.

The problem with -g, simply put, is that code and variables, may not wind up where the debugger expects. Excerpt from the manual's description of the -g option...

Quote:
GCC allows you to use ‘-g’ with ‘-O’. The shortcuts taken by optimized code
may occasionally produce surprising results: some variables you declared may
not exist at all; flow of control may briefly move where you did not expect it;
some statements may not be executed because they compute constant results
or their values were already at hand; some statements may execute in different
places because they were moved out of loops.

Nevertheless it proves possible to debug optimized output. This makes it rea-
sonable to use the optimizer for programs that might have bugs.


You can see how this would affect the profiler as well.

The 'sections' mechanism allows the linker greater freedom to move things around in order to increase 'locality of reference'... putting functions and data which are often used together, close together.

The big beneficiaries of the 'sections' mechanism are processors with large caches or branch predictors. See this page for a decent description of how this works... http://www.princeton.edu/~achaney/tmve/wiki100k/docs/Locality_of_reference.html

AVR32 may be able to take advantage, I have never investigated it.

For AVR8, there is little benefit aside from the 'side effect' of pruning 'dead code' and data, using the --gc-sections linker option.

Don't let all this Brouhaha prevent you from using these options.

They are useful tools when used appropriately, as with an 'Arduino style 'library'. But even there, these options can cause problems.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I must be missing it. How is the behaviour of -g ith optimised code in any way related to arguments for/against using function-sections/gc-sections? You aren't suggesting that the code within the functions will be built/optimised in a "better" way to make debugging optimised code easier are you? I'd love to see an example of that if that's what you believe.

BTW regarding the -g/-O thing, I wrote this previously:

Optimisation and the importance of volatile in GCC

Yes -O1/2/3/s and -g make debugging "interesting" but not impossible - but I don't see how -ffunction-sections changes anything?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

All the manuals description of -g is saying, is that certain compiler optimizations can make debugging, as you said, interesting. :(

The manuals description of --function-sections merely states that things may become even more interesting. :x

The sections mechanism has little to no effect on code inside functions.

Think of it this way. The linker loads each object module which is referenced by main(), directly or indirectly.

It picks a place in the memory map, places the entire object module in that spot, and then fixes up pointers and the like.

It does nothing to the code inside the object module ( except to fix up pointers and the like ).

When --function-sections is enabled, the linker is free to place each function anywhere in the memory map, as if it were an object module.

That's all the magic there is to function and data sections.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

It does nothing to the code inside the object module (

Then I've completely missed the point you were trying to make above. I'm not sure why you introduced -g/-O into the discussion of the merits of -ffunction-sections/-gc-sections if they aren't relevant?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson,

Sorry for any confusion I may have caused.

In response to the OP, I copied the entire manual description of function and data sections, and pasted it to my post.

I made no representation that the entire description was relevant to my arguments.

Indeed, I specifically highlighted the part of the description which is relevant.

In that post I did not mention debugging (except by pasting from the GCC manual)... I cannot see from where you derive that it was I who injected debugging into the discussion.

Indeed, it was a different poster, skeeve, who brought it up, not I.

Quote:
I get why the object files would be larger: more data.
Why would they be slower?
What's the problem with grpof and debugging?
Perfectly valid questions, although not directly related to the original discussion.

My second post addressed those questions, and specifically stated that these issues should not prevent anyone from using these options where appropriate.