Math Error using WinAVR with uchar & ulong

Go To Last Post
7 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Math Error using WinAVR

WinAVR-20090313-install.exe
GNU Compiler Collection (GCC) 4.3.2
avr-libc 1.6.6
AVR AtMega328P

Mixing unsigned char and unsigned long

count1 and count2 should be equal.
But for some input values they are not.

The equation that uses unsigned long vars is always right.

The equation that uses unsigned char vars is sometimes right and sometimes wrong.

--------- Source -----------------------

unsigned char Byte0, Byte1, Byte2;
unsigned long count1, count2, b0, b1, b2;

Byte0 = 121;
Byte1 = 192;
Byte2 = 33;

b0 = (unsigned long)Byte0;
b1 = (unsigned long)Byte1;
b2 = (unsigned long)Byte2;

count1 = (65536 * Byte2) + (256 * Byte1) + Byte0;

count2 = (65536 * b2) + (256 * b1) + b0;

--------- Results -----------------------

Byte0=121 Byte1=192 Byte2=33 Count1=2146425 Count2=2211961 <-- Different

Byte0=231 Byte1=85 Byte2=34 Count1=2250215 Count2=2250215 <-- Same

--------------------

The fact that the value of count1 and count2 are
often the same, but ocasionally different indicates a bug in the library.

Can anyone confirm this?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

the problem is that you are overflowing in the calculation that is using only bytes, because the value is exceeding the size of an int which is 16bits. Casting your constants to long using the L or UL (for unsigned long) suffix should solve the problem. (in this case only the one stage needs to be casted, as it is the only part that will overflow)

count1 = (65536UL * Byte2) + (256U * Byte1) + Byte0; 

Writing code is like having sex.... make one little mistake, and you're supporting it for life.

Last Edited: Sat. Oct 31, 2009 - 04:21 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Glitch,
Thanks for your response.

If that was the problem, why does it give the corect answer most of the time?

See the second result where count1 and count2 are the same.
It is the fact that the results are inconsistent that makes me think it's a bug.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

you may also have to cast the second constant to unsigned (U suffix). Otherwise the intermediate result will be a signed int, which is then sign extended up to 32bits. [confirmed, see my edit above... not a compiler bug] I think you'll find that any value over 127 in the 2nd position is where it fails.

Failing case: [Byte1>127]
               Byte0 Byte1 Byte2  Count1  Count2
      Decimal:  121  192    33   2146425  2211961
          Hex:  79   c0     21   20c079   21c079

As unsigned 32bit calc:
  0x00000079         = 0x00000079
+ 0x000000c0 * 256   = 0x0000c000
+ 0x00000021 * 65536 = 0x00210000
                      -----------
                       0x0021c079
                       
As mixed calc (signed)
                   value       32bit extended
  0x79         =       0x79(u) = 0x00000079
+ 0xc0 * 256   =     0xc000(s) = 0xffffc000 [sign extended]
+ 0x21 * 65536 = 0x00210000(s) = 0x00210000
                                 ----------
                                 0x0020c079
Note the second operand ends up as signed in the intermediate stage, becase one of the input operands was signed, thus is sign extended when cast to 32bits

As mixed calc (unsigned)
                   value       32bit extended
  0x79           =       0x79(u) = 0x00000079
+ 0xc0 * 256U    =     0xc000(u) = 0x0000c000
+ 0x21 * 65536UL = 0x00210000(u) = 0x00210000
                                   ----------
                                   0x0021c079
second byte is no longer signed in the intermediate stage, because both operands were unsigned                              
                              
Passing case: [Byte1<128]

               Byte0 Byte1 Byte2  Count1  Count2
           dec: 231   85    34    2250215 2250215
           hex: e7    55    22    2255e7  2255e7 

As unsigned 32bit calc:
  0x000000e7         = 0x000000e7
+ 0x00000055 * 256   = 0x00005500
+ 0x00000022 * 65536 = 0x00220000
                      -----------
                       0x002255e7
                       
As mixed calc (signed)
                   value       32bit extended
  0xe7         =       0xe7(u) = 0x000000e7
+ 0x55 * 256   =     0x5500(s) = 0x00005500
+ 0x22 * 65536 = 0x00220000(s) = 0x00220000
                                 ----------
                                 0x002255e7
Note no sign extension happens because the MSB of the intermediate 16bit result is 0

As mixed calc (unsigned)
                   value       32bit extended
  0xe7           =       0xe7(u) = 0x000000e7
+ 0x55 * 256U    =     0x5500(u) = 0x00005500
+ 0x22 * 65536UL = 0x00220000(u) = 0x00220000
                                   ----------
                                   0x002255e7

[bunch of edits to fix the formatting]

Writing code is like having sex.... make one little mistake, and you're supporting it for life.

Last Edited: Sat. Oct 31, 2009 - 05:07 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Glitch,
I think you're right about the second term.

If it promotes it to a signed int, and if the value is large enough,
it will see it as a negative number instead of a positive one.
So anytime Byte1 is greater than 127, it generates a negative term and causes the error.

Thanks. Every time I think there is a bug in the compiler/lib, it turns out that I'm the bug.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

added the math to show what is going on. Promotion rules will get you every time unless you're careful.

Writing code is like having sex.... make one little mistake, and you're supporting it for life.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

Thanks. Every time I think there is a bug in the compiler/lib, it turns out that I'm the bug.

You are not alone - about 99 out of 100 reports of a "compiler bug" here are the user's misunderstanding (probably 50 of those are folks that don't know about 'volatile'!)