Unit Test / Regression Test framework for AVR (8-bit)

Go To Last Post
9 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Dear all,

On a host programming environment, I have used both the Boost Test Library (http://www.boost.org/doc/libs/1_...) and cppunit for C++ code, both for test-driven development and automatic regression testing.

On the AVR, I have a bunch of numeric routines that I would like to do unit tests on, and it would also be nice to perform regression testing during builds.

I have been stung in the past by testing C-code on a host environment, only to see an integer overflow in some rare cases on a microcontroller (PSoC in that case, but same would have been true for AVR).

Is there any unit testing or regression testing framework for AVR? Has anyone used something similar? How easy has it been to use?

Damien.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Nothing particular that executes on an AVR. What I do is run tests on the PC where I'm developing the firmware. When I'm developing something I want to test, I make sure that I keep parts that are algorithmically interesting separate from code that interacts with the hardware (e.g. sets register values). Then I compile the interesting bits with my PC as the target so I can unit test it, with the hardware accessing layer of the code replaced by stubs or mocks as the case may be. That way, I can at least get some testing done. It doesn't worry me that the code coverage is not 100%. It makes more sense for larger projects where there's a larger proportion of code that is not tightly coupled to the micro's hardware.

Sometimes I've even used fairly sophisticated fakes so I can do a degree of integration testing more easily. For example, I wrote some FAT code, but replaced the driver code that accessed an SD card with a driver that read data from an ISO image on my hard drive mounted as a loopback device. That way, the FAT code got a workout with real data.

It would be an interesting exercise to write a testing framework that can execute test code inside an AVR simulator - that could make it easier to test the boundaries of your code that interact with the AVR hardware. It's on my list of things to do, but given that it's a near infinitely long list it's highly unlikely I'll ever get around to it.

Michael

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

damien_d wrote:
I have been stung in the past by testing C-code on a host environment, only to see an integer overflow in some rare cases on a microcontroller (PSoC in that case, but same would have been true for AVR).
I see three potential cures:
Run in a simulator.
Get or make a compiler with a 16-bit int.
Disallowing int might be possible,
but it would be tricky.
Quote:
Is there any unit testing or regression testing framework for AVR? Has anyone used something similar? How easy has it been to use?

Iluvatar is the better part of Valar.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I haven't done anything with testing frameworks on the AVR but it is an interesting idea. While I rarely use anything as formalized as Boost Test in my own work, it would be correct to consider my style test driven development. At this point, on the AVR, that basically means building a program in steps or writing a testing harness for specific routines. Something as simple as a framework that would allow runtime success/failure info to be sent to an led or the USART could be nice. In addition, the framework could be entirely code based and, therefore, allow the construction and compilation of testing fixtures or, alternatively, could use a generator program in addition to a small runtime library so that tests could be written entirely separately and stitched together later.

I'll have to think about that...

Martin Jay McKee

As with most things in engineering, the answer is an unabashed, "It depends."

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

One can make a gnu C compiler that uses 16-bit ints.
I'm told it's a configuration option when compiling the compiler.
I've not done it myself.

Iluvatar is the better part of Valar.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I also thought about the simulator approach for unit testing. :idea: The idea was to wrap simulavrx into a Python module (e.g. with swig). Python would offer a easy way of scripting a regression test suite. The point is that a micro always interacts with real environment, maybe its possible to mock some simple hardware and environment parameters with python functions.
But at this point the STDLE (standard lame excuse) triggers: So many ideas, so little time :-(

/Axel

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I am interested to test some of the limits of the variables and values with respect to the other variable. For example in a function foo(int a, int b), b must always be less than a.  a and b values are decided during compile time and they do not change during run time. If I check this condition inside the function, I am wasting code space. Also there is no way to report the error. So I thought unit testing for limits and certain constraints on values might help. For such situations, are there any free unit testing tools available? I have not used unit testing tools and not sure of their features with respect these features. Other alternate may be to go for debug version and release version.

Last Edited: Sun. Apr 17, 2016 - 05:38 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

joneewheelock wrote:
If I check this condition inside the function, I am wasting code space.
That's what C's assert is for; it's worth it.

joneewheelock wrote:
So I thought unit testing for limits and certain constraints on values might help.
If the ranges are well constrained; otherwise it's a large or very large matrix though ideally sparse.

Easier to use lint and maybe a static source code analyzer IF affordable.

Frama-C has a form of unit test case generation in PathCrawler that could be evaluated for usefulness.

oneewheelock wrote:
For such situations, are there any free unit testing tools available?
There's a plethora so I cannot create a complete answer.


Frama-C Software Analyzer

Frama-C

PathCrawler plug-in

http://frama-c.com/pathcrawler.html

PathCrawler automatically finds test-case inputs.

http://www.throwtheswitch.org/

http://www.electronvector.com/blog/portable-reproducible-embedded-build-environments-with-vagrant

 

"Dare to be naïve." - Buckminster Fuller

Last Edited: Mon. Apr 18, 2016 - 04:12 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

joneewheelock wrote:
a and b values are decided during compile time and they do not change during run time. If I check this condition inside the function, I am wasting code space. Also there is no way to report the error.

Well usually you would use:

#include <assert.h>

void foo(inta, int b) {
    assert(b < a);
    // rest of code
}

It's true that the assert() check does generate code but as the user manual:

 

http://www.nongnu.org/avr-libc/u...

 

states:

The assert() macro may be removed at compile time by defining NDEBUG as a macro (e.g., by using the compiler option -DNDEBUG).

In Stduio Atmel predefine DEBUG for "Debug" builds and NDEBUG for Release builds. So you test stuff like this using your "Debug" build then there's no overhead in your final "Release".

 

BTW as the manual also tells you:

As there is no standard error output stream available for many applications using this library, the generation of a printable error message is not enabled by default. These messages will only be generated if the application defines the macro

 

__ASSERT_USE_STDERR

So if you want the Debug build to generate messages to show that an assert() has occurred then do the usual FDEV_SETUP_STREAM stuff to connect an output device to stderr then define the above macro.