## Any advice (understanding variable truncation)

5 posts / 0 new
Author
Message

Hello people,

i

I wrote this function on AVR

and Novf , TCNT0_Counts are a global variables

u16 Novf=0;

u8 TCNT0_Counts;

when i write this function it returns 0 always , when i tried to display the Novf value on LCD it was in the range of 340 to 440

u32 calc_dist(void)
{
u32 dist;
if(Novf < (u16)730)
{
dist =((TCNT0_Counts +(Novf*256))*2144)/1000000;
}
return dist;

}

and when i modified the function like below it worked fine and returned result as it should be

u32 calc_dist(void)
{
u32 dist;
if(Novf < (u16)730)
{
dist =(((u32)TCNT0_Counts +((u32)Novf*256))*2144)/1000000;
}
return dist;

}

can anyone tell me how the processor handle those casting , and why it differs from the first function???

Last Edited: Tue. Mar 24, 2020 - 09:15 AM

Simply put;  Your variables are of all different sizes so in the first example values get truncatedf but in the second example you're casting so values are same size therefore no truncation.

Happy Trails,

Mike

JaxCoder.com

if Novf = 450;

how is Novf  truncated?

It will depend on the size of an int.

I assume this is for AVR8 where int will be 16 bits.

In which case, all the operations before you finally divide by 1000000 will be performed as 16 bits (as unsigned int in this case since Novf is u16).

So you end up with an unsignd 16 bit value, which can only ever be max 65535, and then divide that value by 1000000, so result willl always be 0.

(The fact that for the division the operation will be performed as 32 bits doesn't help).

Take Novf * 256.

Novf is u16 which is unsigned int assuming int is 16 bits.

256 is of type int.

C rules say

i) apply integer promotion to operands. In this case that doesn't change anything since both operands are already int sized

ii) then find common type for the operation. In this case you have unsigned int  and int, the rules say the common type is unsigned int.

So Novf * 256 is performed as unsigned int ie. 16 bits. Clearly this can overflow the range of 16 bits.

Yoiu then add a u8. That add will be done as 16 bits.

You then multiply by 2144, that multiply will be done as 16 bits, so can overflow.

By casting Novf to 32 bits you have forced the multiply, add, multiply to all be done as 32 bit so no overflow, and you now have a 32 bit value divided by 1000000 rather than a 16 bit value divided by 1000000.

If you ran your original code (without the extra u32 cast) on a PC with 32 bit int you would see it probably works, since all operations are promoted to int which would now be 32 bits.