## Getting weird results from integer math.. help?

36 posts / 0 new
Author
Message

I am trying to non linearly decrease the volume byte from 255 to 0. The results I get are 255 for every i value and when i becomes greater than 255 * 4, suddenly volume becomes 0. (release has a value of 255).

I don't understand why this happens. By my calculations an integer division should give eg for i=1000 : (255 * 255 * 4) / ( 1000 + (255 * 4)) =  260100 / 2020 = 128. But I get 0 or 255. However if I change 255 to 255.0, the results are as expected.

But.. I would prefer to avoid using a float expression cause it is computationally expensive. Could you help me understand why I get this result? I am using an ATMega328p chip and trying to incorporate a release mechanic for a synth.

for (int i=0; i<1500; i+=20)
{
volume = (255 * release * 4) / (i + (release * 4));
BYTEtoSerial(volume);
BYTEtoSerial(i);
BYTEtoSerial(i >> 8);
UDR0='\n';
_delay_ms(20);
}

Serial out:

...

255
252
003

000
016
004

...

This topic has a solution.

TO THE FINDER... THE ISLE OF KOHOLINT, IS BUT AN ILLUSION... HUMAN, MONSTER, SEA, SKY... A SCENE ON THE LID OF A SLEEPER'S EYE... AWAKE THE DREAMER, AND KOHOLINT WILL VANISH MUCH LIKE A BUBBLE ON A NEEDLE... CAST-AWAY, YOU SHOULD KNOW THE TRUTH!

Last Edited: Thu. Sep 17, 2020 - 09:55 PM

How is "volume" declared?

Jim

Until Black Lives Matter, we do not have "All Lives Matter"!

it is a uint8_t

TO THE FINDER... THE ISLE OF KOHOLINT, IS BUT AN ILLUSION... HUMAN, MONSTER, SEA, SKY... A SCENE ON THE LID OF A SLEEPER'S EYE... AWAKE THE DREAMER, AND KOHOLINT WILL VANISH MUCH LIKE A BUBBLE ON A NEEDLE... CAST-AWAY, YOU SHOULD KNOW THE TRUTH!

This reply has been marked as the solution.

From your example, 260100 will not fit into a 16-bit integer (signed or unsigned).  This is a common problem, having intermediate calculations exceed the range of the int type.  Try declaring your constants (255 and 4) as UL (i.e. 255UL, 4UL), that should force 32-bit operations (making your code slower, but correct).

Last Edited: Thu. Sep 17, 2020 - 05:03 PM

Go on.  You just have to put some sample numbers in to your expression.  And write down the result.

Quite honestly,  if f-p arithmetic does what you want,  use it.   It does not involve much Flash memory.  You have a 32kB Flash.   Speed is definitely unimportant.  You twiddle thumbs for 20ms.

(255 * release * 4) / (i + (release * 4))
(255 * 255 * 4) / (i + (255 * 4))
(260100) / (i + 1020)
i = 0   : xpr = 255
i = 500 : xpr = 171
i = 1000: xpr = 128
i = 1500: xpr = 103

Note that you need to use 32-bit arithmetic to cope with numbers like 260100.   Just cast one of the ingredients e.g.

(255uL * release * 4) / (i + (release * 4))

C will do the arithmetic for you.   But I bet it makes your head hurt.

David.

Last Edited: Thu. Sep 17, 2020 - 05:09 PM

re volume, a linter produces an informational message stating 18 bits for size.

A loss of sign informational message is corrected by making i of type uint16_t.

"Dare to be naïve." - Buckminster Fuller

gchapman wrote:

re volume, a linter produces an informational message stating 18 bits for size.

A loss of sign informational message is corrected by making i of type uint16_t.

Lint should spot that the 260100 will overflow int16_t and uint16_t

Intermediate value overflow in Integer expressions is very dependent on integer width.

It seldom causes a problem in f-p expressions.

David.

Or you could simplify the expression:

volume = (255 * release) / (( i / 4 ) + release );

Thank you all I learnt so much! I got where I needed

TO THE FINDER... THE ISLE OF KOHOLINT, IS BUT AN ILLUSION... HUMAN, MONSTER, SEA, SKY... A SCENE ON THE LID OF A SLEEPER'S EYE... AWAKE THE DREAMER, AND KOHOLINT WILL VANISH MUCH LIKE A BUBBLE ON A NEEDLE... CAST-AWAY, YOU SHOULD KNOW THE TRUTH!

One linter is producing an information message about precision loss.

AVR GCC has 24-bit integers as of v4.7 (XMEGA)

https://gcc.gnu.org/wiki/avr-gcc#Types

50931 – [avr] Support a 24-bit scalar integer mode

"Dare to be naïve." - Buckminster Fuller

A linter indicates 16 bits for size.

"Dare to be naïve." - Buckminster Fuller

tellSlater wrote:
I got where I needed

Jolly good - now please mark the solution.

See Tip #5 in my signature, below:

Top Tips:

1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...

Or you could simplify the expression:

volume = (255 * release) / (( i / 4 ) + release );

That's very nice, especially since I counts by 20 (from zero),thus perfectly divisible by 4   ....wonder if the compiler recognizes that (vs say counting by 13) ...why deal with all the unnecessary extra match of larger numbers.

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

I realized you just cant have a term in a calculation going over the 16bit maximum so no more than 65535 .

As kk6gm and  david.prentice  said above, I need to declare some of the numbers in the expression as long so that a 32bit calculation will execute.

Even better I can simplify the calculation as  N.Winterbottom demonstrated - no need for 32bit calculation at all.

Again, thank you all for your help

TO THE FINDER... THE ISLE OF KOHOLINT, IS BUT AN ILLUSION... HUMAN, MONSTER, SEA, SKY... A SCENE ON THE LID OF A SLEEPER'S EYE... AWAKE THE DREAMER, AND KOHOLINT WILL VANISH MUCH LIKE A BUBBLE ON A NEEDLE... CAST-AWAY, YOU SHOULD KNOW THE TRUTH!

Last Edited: Thu. Sep 17, 2020 - 10:05 PM

david.prentice wrote:
Note that you need to use 32-bit arithmetic to cope with numbers like 260100.   Just cast one of the ingredients e.g.

(255uL * release * 4) / (i + (release * 4))

C will do the arithmetic for you.   But I bet it makes your head hurt.

Note that

(255 * release * 4uL) / (i + (release * 4))

will not work.  255*release will overflow a signed int.

Iluvatar is the better part of Valar.

skeeve wrote:
Note that

(255 * release * 4uL) / (i + (release * 4))

will not work.  255*release will overflow a signed int.

My own style would be to write 255UL as well as 4UL, but I would have expected the single 4UL to make the entire calculation UL.  In fact, I'd expect the compiler to roll the 255 * 4UL into a single constant.

One point I would make: do away with all use of int, long, uL, long long etc... they are completely non-portable, depending on the particular implementation to define their size. Use the standard int8_t, int16_t, int32_t, int64_t (and their unsigned pairs uint8_t etc). While you're at it you should also use bool for boolean types rather than int (not in your example, just as a general thing).

By doing this you are writing code which:

• clearly indicates the bit width and signedness of a variable, irrespective of architecture and compiler
• makes it obvious to you and anyone else in the future how large those variables were intended to be
• helps to mark out of range issues such as you had - i.e. uint_8 * uint_8 always requires a uint16_t to hold the result safely, and so on
• is architecture and compiler safe

It will stop a lot of heartache in the long run :)

Neil

barnacle wrote:

• helps to mark out of range issues such as you had - i.e. uint_8 * uint_8 always requires a uint16_t to hold the result safely, and so on

Assuming uint8_t to be a standard library type, uint8_t * uint8_t produces signed int, always.

The multiplication might produce signed overflow, which is undefined.

To be pedantic:

uint8_t and uint16_t are not required to exist on all platforms.

Neither requires the existence of the other.

Once upon a time, there was a machine with a 60-bit word.

At this writing, I suspect that few, if any, platforms do not

have 8-bit bytes or do not do 16-bit integer arithmetic.

That said, we are dealing with an 8-bit microcontroller.

Pretty much by definition, it has an 8-bit byte.

C requires at least a 16-bit byte.

To be truly pedantic:

In principle, even on an AVR, the compiler could inflict a 16-bit or even 9-bit bytes on its users.

Yum.

Iluvatar is the better part of Valar.

uint8_t and uint16_t are not required to exist on all platforms.

Perhaps true, but if they didn't exist on a platform, wouldn't there be an error to alert you to the problem?  That seems like a decent safety net, compared to the alternative of using possibly "random" bit-width types & getting unexpected results (well they actually should be expected, once you looked up the exact definitions for the platform's defined bit-widths).  The processor doen't lie!

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

skeeve wrote:

To be truly pedantic:

In principle, even on an AVR, the compiler could inflict a 16-bit or even 9-bit bytes on its users.

Yum.

True, and the customers could inflict the trash bin on that compiler, followed by unspeakable verbal and physical abuses upon it.

Last Edited: Sat. Sep 19, 2020 - 05:40 PM

This is reminiscent of the issues that surrounded the move from 32- to 64-bits on Intel platforms and operating systems. e.g. https://en.wikipedia.org/wiki/64.... There is apparently still a difference between Windows and Linux.

Many 64-bit platforms today use an LP64 model (including Solaris, AIX, HP-UX, Linux, macOS, BSD, and IBM z/OS). Microsoft Windows uses an LLP64 model. The disadvantage of the LP64 model is that storing a long into an int may truncate. On the other hand, converting a pointer to a long will “work” in LP64. In the LLP64 model, the reverse is true. These are not problems which affect fully standard-compliant code, but code is often written with implicit assumptions about the widths of data types. C code should prefer (u)intptr_t instead of long when casting pointers into integer objects.

AVR gcc/g++ seems quite happy with intptr_t

obdevel wrote:
This is reminiscent of the issues that surrounded the move from 32- to 64-bits on Intel platforms....

Code Geezer story.  I had to fix code that broke when Windows transitioned from 16-bit to 32-bit integers...

kk6gm wrote:
skeeve wrote:

To be truly pedantic:

In principle, even on an AVR, the compiler could inflict a 16-bit or even 9-bit bytes on its users.

Yum.

True, and the customers could inflict the trash bin on that compiler, followed by unspeakable verbal and physical abuses upon it.

Not to dwell too much upon the obvious: Yes.  Yes.  No.

How would one inflict physical abuse on a compiler.

That said, its humans should watch out.

Iluvatar is the better part of Valar.

To repeat what I've said in both this thread and others:

• C has a number of implicit and *hidden* type casts, promotions, and assumptions about variable sizes - in particular, any 8-bit type is promoted to 'int' before any arithmetic is performed and the result cast to the type of the receiving variable.
• Int in C is compiler dependent. It is in common use 16, 32, and 64 bits wide on common architectures today.
• If you don't define the width of *all* your variables you are playing guessing games - if not on your own target, as soon as you try and build the code on something different
• the u/int8_t, 16_t etc types define the exact size of the variable irrespective of architecture and compiler - they are defined in stdint.h (from C99 I think) which incidentally offers other precise width variables: https://www.ibm.com/support/know...
• casting does *not* change the value of a variable; it changes the way in which a particular bit pattern is interpreted by the arithmetic.

Using uintXX_t lets you know what you've got. Casting removes the surprises. Using uncharacterised types is system dependent and not portable.

Neil

barnacle wrote:

• casting does *not* change the value of a variable; it changes the way in which a particular bit pattern is interpreted by the arithmetic.

int8_t x = -1;

(uint8_t)x has the value 255

The value has changed from -1 to 255.

But, yes, the bit pattern hasn't changed, assuming using 2's complement for signed types.

EDIT which, given that the exact width stdint types are by definition 2's complement is a safe assumption.

If it was signed char instead of int8_t, in theory it might not be 2's complement (but in practice it will be unless it's some very exotic processor).

Last Edited: Sun. Sep 20, 2020 - 11:22 AM

barnacle wrote:

• casting does *not* change the value of a variable; it changes the way in which a particular bit pattern is interpreted by the arithmetic.

MrKendo wrote:
The value has changed from -1 to 255.

I think that's what Neil meant by, "interpretation" ?

I think what Neil was trying to say is that the bit pattern stored in the variable is not changed ?

EDIT

SO perhaps we could say,

casting does *not* change the bit pattern stored in memory; it changes the way in that bit pattern is interpreted by the arithmetic

?

Top Tips:

1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
Last Edited: Sun. Sep 20, 2020 - 11:37 AM

awneil wrote:

SO perhaps we could say,

casting does *not* change the bit pattern stored in memory; it changes the way in that bit pattern is interpreted by the arithmetic

int8_t x = -1;

(int16_t)x has the same value, -1, but the bit pattern is different in that it is now 16 bits not 8.

(Yes, it's the original bit pattern sign extended)

Maybe I'm just being too pedantic.

You can certainly say that casting to the same or wider width of the same signedness as the original is guaranteed not to change the value.

A cast is a conversion (an explicit conversion) from one type to another.

The actual rules (described in terms of value) are as given in https://www.avrfreaks.net/commen...

If you want to think in terms of bit pattern, assuming 2's complement for signed, it boils down to (hopefully I get this right)

unsigned -> unsigned

new width same:  no change

new width wider: extend with 0

new width narrower: truncation

signed -> signed

new width same: no change

new width wider: sign extend

new width narrower: truncation (for some values implementation defined)

unsigned -> signed

new width same: no change (for some values implementation defined)

new width wider: extend with 0

new width narrower: truncation (for some values implementation defined)

signed -> unsigned

new width same : no change

new width wider: sign extend

new width narrower: truncate

MrKendo wrote:
Maybe I'm just being too pedantic.
I too

#include <stdint.h>
#include <stdio.h>
int main () {
int8_t x = -1;
printf("%u",(unsigned int)(uint8_t)x);
printf("%u",(uint8_t)x);
}

On the last printf, a linter generates an informational message of an inconsistency.

No defects :

#include <stdint.h>
#include <stdio.h>
int main () {
int8_t x = -1;
printf("%i",(int)(int16_t)x);
printf("%i",(int16_t)x);
}

PC-lint Plus Online Demo - Gimpel Software - The Leader in Static Analysis for C and C++ with PC-lint Plus

"Dare to be naïve." - Buckminster Fuller

The value of the variable is a matter of interpretation. The bit pattern remains the same until it is used (and the routines to use it may well be different depending what changes).

e.g.

uint8_t a = 0x80
uint8_t b = 0x80
// both of these will be automatically sign extended to an 'int'; let's assume 16 bits for argument
uint8_t c = (0x0080 + 0x0080) && 0xff = 0x00    // overflow! but no warning, no carry...
uint16_t d = 0x0080 + 0x0080 = 0x0100           // that's fine, as expected

// but
int8_t e = 0x80
int8_t f = 0x80
int8_t g = (0xff80 + 0xff80) && 0xff = 0x00 // another overflow with no warning (also changed sign!)
int16_t h = 0xff80 + 0xff80 = 0xff00        // fine, as expected, though there is an invisible sign extension

// and
uint8_t i = 0x80
int8_t j = 0x80
int8_t k = (0x0080 + 0xff80) && 0xff = 0x00     // correct value, correct sign, no overflow
uint8_t k = (0x0080 + 0xff80) && 0xff = 0x00    // correct value, correct sign, no overflow
int16_t l = (0x0080 + 0xff80) = 0x0000          // correct value, correct sign, no overflow
uint16_t m = (0x0080 + 0xff80) = 0x0000         // correct value, correct sign, no overflow

The bit patterns don't change, but the invisible sign extension bits may change. But the answer may be wrong due to sign change or overflow, which may or may not be significant in your application. At the hardware level, on pretty much any generally available modern processor (i.e. anything built since the seventies!) addition/subtraction are always performed using unsigned variables - or rather, there is no concept of signed or unsigned, positive or negative, at the processor register/memory level. It's all in the interpretation.

Neil

MrKendo wrote:
(int16_t)x has the same value, -1, but the bit pattern is different

but the point that the bit pattern in x is not changed.

Top Tips:

1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
Last Edited: Sun. Sep 20, 2020 - 02:54 PM

IIRC the most recent C standard requires twos complement.

Even so, in principle, a cast between integer types of the same size could change the bit pattern.

Integer types of different ranks could have  different endianness.

I know of no real world examples.

For earlier standards, a cast from a negative ones

complement value to unsigned would change the bit pattern.

Negative values have few constraints on their representations.

All that said, integer types of the same rank with the

same value are required to have same bit pattern.

BTW there are cases where the size of int matters even if all ones variables have specific widths.

Iluvatar is the better part of Valar.

skeeve wrote:
a cast between integer types of the same size could change the bit pattern.

but the point is that it doesn't change what's actually stored in the memory location of the variable - that remains unchanged.

In other words, there is no modification of the variable itself - just what happens to it after it has been read from memory.

Top Tips:

1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...

awneil wrote:

MrKendo wrote:

(int16_t)x has the same value, -1, but the bit pattern is different

but the point that the bit pattern in x is not changed.

Oh right, i see what you were talking about now.

Yes, the value of x itself (ie. the value stored in x) is not changed, just as the expression ~x or !x or -x does not change the value stored in x (unless of course you assign the result back to x).

it is rather tricky to explain in words.

I'll have to try to think of a way to illustrate it diagrammatically...

Top Tips:

1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...

awneil wrote:

skeeve wrote:
a cast between integer types of the same size could change the bit pattern.

but the point is that it doesn't change what's actually stored in the memory location of the variable - that remains unchanged.

In other words, there is no modification of the variable itself - just what happens to it after it has been read from memory.

In

int i=-1;
unsigned u=(unsigned)i;
unsigned short s=u;

If int is ones complement, i and u will have different bit patterns.

Though extremely unlikely,

even if int and short have the same size,

s and u could have different bit patterns.

Evaluating (unsigned)i does not change the value of i anymore than evaluating 1+i does.

No one was even hinting that a cast was an assignment operator.

Iluvatar is the better part of Valar.