Instrument functions on XMEGA (function pointer size problem)

Go To Last Post
12 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm having a problem using -finstrument-functions on an XMEGA 128A3U. I get "'builtin_return_address' contains only 2 bytes of address [enabled by default]" when compiling. My functions are as follows:

 

void __cyg_profile_func_enter (void *func,  void *caller) __attribute__((no_instrument_function));
void __cyg_profile_func_enter (void *func,  void *caller)
{
}
 
void __cyg_profile_func_exit (void *func, void *caller) __attribute__((no_instrument_function));
void __cyg_profile_func_exit (void *func, void *caller)
{
}

 

I guess this is because with 128k of flash memory, function pointers can be larger than 16 bits. Since these are built-in functions I'm not sure how to work around this.

 

I could disable the warning but then the addresses would wrap around. I tried changing "void *" to "uint32_t" but of course it conflicts with the builtin prototype. It seems like maybe it's just broken on AVRs with >16 bit program counters.

 

Incidentally, using instrument functions also causes builtin_avr_delay_cycles() to fail compilation as it thinks that the delay amount is no longer a compile time constant. In other words. _delay_ms(1) doesn't compile.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Don't see an easy way around this without pulling GCC source and modifying the call made to the profile function. While you could just implement these in .S where there's no check of parameter types or how you handle them the fact is that the calling code is only going to be passing parm 1 in R25:R24, not a group of 4 so it won't help. It needs to be persuaded to pass in more than 16 bits.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks Cliff, I though as much. I'll see if I can submit a bug report.

 

Any idea why it breaks builtin_avr_delay_cycles()? If I could get it to compile with warnings then it wouldn't be too bad with only 16 bit addresses. What are the chances of a random 16 bit address having a function entry point in both the upper and lower 64k?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Wait, I'm being daft. Function pointers are word addressed, so actually 16 bits will be fine, it's only the bootloader section that actually needs 17 bits.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

mojo-chan wrote:
Any idea why it breaks builtin_avr_delay_cycles()?

Well what it's doing is adding a call to the entry hook to the prologue of every function and a call to the exit hook to the epilogue. _delay_ms() works by being a function that can be completely optimised away but as soon as you add prologue/epilogue calls to it that optimisation presumably cannot occur.

 

But you wouldn't be leaving these instrumentation functions in your final code would you? Presumably this is just for testing coverage or something isn't it? Once you have proven 100% coverage you can switch it off for the real issue of the Release code can't you? In fact I'd be tempted to make the whole thing part of "Debug", maybe just building the hooks when either -DDEBUG or !(-DNDEBUG).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Sure, I'm trying to debug a very annoying problem. I have a large number of devices (>10,000) and about 0.5% of them randomly lock up. No consistent circumstances, and the RTC keeps going so they are at least executing that interrupt. Can't get them to do it when attached to the debugger of course, and they often takes weeks to fail.

 

I though I could grab the address of the function where they are getting stuck. The watchdog is enabled so it must be getting reset or somehow turned off (sadly can't permanently enable it via fuses via the bootloader). There is no code anywhere that turns it off, so... I checked every point at which WDR() is called, and I'm now at a bit of a loss and in need for more debug info. If I store the function address in a .noinit section I can recover it after reset.

 

I suppose I can add code to every function to simulate this feature, but it's quite a large project and quite a bit of work. It also doesn't help that __func__ is kinda broken on AVR too, as it stores strings in RAM due to the lack of a __flash attribute. I guess something with __LINE__ is the only option.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The way we used to do coverage was to have a header that effectively redefines { and } so that code such as:

void fn(int a, int b) {
    if (a > b) {
        PORTB = 0x55;
    }
}

(say) would become something like:

void fn(int a, int b) {
    enter(__FILE__, __FUNCTION__, __LINE__);
    if (a > b) {
        enter(__FILE__, __FUNCION__, __LINE__);
        PORTB = 0x55;
        leave(__FILE__, __FUNCTION__, __LINE__);
    }
    leave(__FILE__, __FUNCTION__, __LINE__);
}

and those functions just logged all the execution.

 

To be honest it wasn't quite this simplistic and I forget all the details. I know that each branch point actually involved having it's own incremented counter so it was also possible to say "it spent 31% of the time in fn()" etc. but the core idea is something like this.

 

I'm sure we weren't the first (or last) to think of doing something like this and I'm sure their must be authoritative articles about such ideas on the internet.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

mojo-chan wrote:
Can't get them to do it when attached to the debugger of course, and they often takes weeks to fail.

  • lint
  • Static analysis
  • Logic analyzer

 

Consider running lint or a static analyzer over all of the source code.

Whole program might be necessary (function interactions, globals, interrupts, etc)

 

An AVR port write will be completed before a reset; captured via a logic analyzer.

With a fast SPI and XMEGA DMA might be able to push enough data fast enough before the reset; captured via a logic analyzer or SPI-to-USB.

General :

http://www.embedded.com/design/debug-and-optimization/4236800/Troubleshooting-real-time-software-issues-using-a-logic-analyzer

Specific :

Microchip Technology Inc

Microchip Technology

MPLAB REAL ICE In-Circuit Emulator User's Guide For MPLAB X IDE

http://ww1.microchip.com/downloads/en/DeviceDoc/50002085D.pdf (page 50, 6.3 INSTRUMENTED TRACE)


Gimpel Software Blog

Gimpel Software Blog: MISRA C, Toyota and the Death of Task X

http://blog.gimpel.com/2015/02/misra-c-toyota-and-death-of-task-x.html

...

... please "lint early and lint often.”

Edit : added lint and above URL

"Dare to be naïve." - Buckminster Fuller

Last Edited: Tue. Jul 19, 2016 - 03:17 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:

    enter(__FILE__, __FUNCTION__, __LINE__);

File, function, and line also appear in the source code for 'assert.h'.

http://www.nongnu.org/avr-libc/user-manual/assert_8h_source.html

 (line 104)

http://www.nongnu.org/avr-libc/user-manual/group__avr__assert.html#ga0041af519e0e7d47c9bcc83760c4669e

Would need to implement STDERR.

clawson wrote:
I'm sure we weren't the first (or last) to think of doing something like this and I'm sure their must be authoritative articles about such ideas on the internet.
A very recent and short read (for a very difficult bug) :

Maximizing visibility through logging, as on Mars Pathfinder

by

July 12, 2016

http://www.embedded.com/electronics-blogs/say-what-/4442366/Maximizing-visibility-through-logging--as-on-Mars-Pathfinder

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

gchapman wrote:
also appear in the source code for 'assert.h'.

Not entirely sure I see how that helps OP here? How would you propose to use that to trace flow of execution?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Does not help mojo-chan.

The intent was to make visible that exceeding a constraint could cause an anomaly or an invalidity (stack overflow, equality operator instead of a relational operator, incorrect indexing, incorrect increment or decrement, overflow, underflow; some of those could cause an infinite loop caught by the watchdog).

 

Edit : last sentence

 

"Dare to be naïve." - Buckminster Fuller

Last Edited: Tue. Jul 19, 2016 - 03:06 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The problem is that the device goes non-communicative. I was thinking of adding an interrupt triggered by a switch I'll solder on to an I/O pin, which will then dump some critical data like the stack. The device doesn't reset, it gets stuck in some state where it won't respond to any external stimulus but is still running the RTC (via interrupt). There are multiple external interrupts that should make it talk (RX pin on two USARTs, for example) and those interrupts should never be disabled... And on >99.5% of loggers there are never any problems.

 

It has to be either some rare firmware interaction that causes multiple interrupts not to fire but the RTC one keeps working, or a hardware glitch. I have found a few hardware issues with the XMEGA range, like static damage causing them to wake up slowly.