## Bizar results with basic float math operations

7 posts / 0 new
Author
Message

Hi All,

I'm currently hacking my way in to a TI83 (84) graphing calculator. TI represents their floating point numbers in a not so obvious way... Anyway they look like this:

```typedef struct
{
unsigned char flags;
unsigned char exponent;
unsigned char mantissa [7];
} TIdata;
```

The mantissa is coded in BCD (with bit 7 of flags being the signbit for the mantissa), thus allowing for a 14 digit resolution.
When I convert de TIData with the code below integers become floats (eg. 64 is interpreted as 63.99993, wich is within the limits of a IEEE floating point resolution.

```double tofloat (TIdata * pdata)
{
char y = pdata -> exponent - 0x80;
double n = 0;

for (unsigned char i = 0; i < 7; i ++)
{
unsigned char x = pdata -> mantissa[i];

y -= 2 * i;

n += ((x & 0xF0) >> 4) * pow (10, y);
n += (x & 0x0F) * pow (10, y - 1);

}

if (pdata -> flags & 0x80) n *= -1;

return n;
}
```

A possible workarround would be to work with long integer math but i would prefer some basic straight to the point strategies.

Any input is appriciated, especially the integers within the byte 0 - 255 limits should be correctly as they are used for HW setup. (TI setting AVR SFR's)

KK.

PS. I think this is compiler specific enough for the AVR - GCC forum?!

Assumptio mater errorum est

The binary number is just mantissa*2^exponent... so dont do all those pows... just do one power of two shift to get the exponent, and one multiply

Imagecraft compiler user

Bob,

Sorry, i do not quitte understand the method you are proposing... Maybe i'm missing something?

KK

Assumptio mater errorum est

You want to convert TI format fp to IEEE format fp. You know the TI fraction is 14 bcd digits. You know the TI exponent is 2 bcd digits. Convert fraction to binary by 10*previous digit metod. Convert exponent same way. Now convert both to IEEE format using an fp multiply and an integer power of two (lookup?). ieeefp=binaryfraction x 2^exponent. Sorry if I didn't understand the problem correctly and gave you a solution that doesnt do you any good.

Imagecraft compiler user

I believe that'll do, i 'll try first thing in the morning.

Thanks Bob.

KK

Assumptio mater errorum est

Hi All,

I changed the conversion routine according to Bob's suggestions...

```double tofloat (TIdata * pdata)
{
double n = 0;

for (unsigned char i = 0; i < 7; i ++)
{
unsigned char x = pdata -> mantissa[i];

n += ((x & 0xF0) >> 4);
n *= 10;

n += (x & 0x0F);
n *= 10;
}

char y = pdata -> exponent - 0x80 - 14;

n *= pow (10.0, (double) y);

if (pdata -> flags & 0x80) n *= -1;

return n;
}
```

When TI transmits the value 64 up until the pow() function call to adjust the exponent everything goes well. (n equals 6.4e+14) After the pow()function is called n equals 'nan'. (AVRStudio and JTAGICE mkII Mega16)
Any input is appriciated.

KK

Assumptio mater errorum est

```pow (10, y);