Initialized Float Value Error?

Go To Last Post
6 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm encountering an error with my float values. I was able to determine this error by using the AVR Debugger and monitoring the value of the float while I stepped through the program.

If I initialize the float as

float y = 4567.12

The debugger will show the value as 4567.1201.

A value of 0.0001 seems to be added to whatever value I initialize the float to be.

Does this sound familiar? If so how can I work around this. Thanks.

AVR Studio 4
STK500
ATmega16
Windows 2000

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think its due to the finite precision
of the number representation. Decimal
fractions often can not be represented exactly
as machine-numbers. And the Output-routines
also often do not output the "exact" value.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

Does this sound familiar? If so how can I work around this. Thanks.

Remember that float is NOT exact. Out of the many constant values that you might assign to a float, very few as a percentage can be represented exactly.

The shown value is the closest (or the one not greater than, or the one not less than, whatever the rules are) to your constant value.

With a float remember that you only have about 21/22 bits of mantissa; this is about 5 or 6 decimal digits and is exactly what you are seeing.

Lee

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

To get some idea of what's going on, go to this page:

http://babbage.cs.qc.edu/IEEE-75...

type 4567.12 into the top box and click the "not rounded" button.

While it can be represented in 64 bits, it's not accurate in the 32 bit floats that most compilers use.

Then turn this round. Go to:

http://babbage.cs.qc.edu/IEEE-75...

and type the previous 458EB8F5 value into the box then [compute] - so it is 4567.11962890625. Now try the very next value 458EB8F6 (which just increments the 23rd bit of the mantissa) and you get 4567.1201171875 (which I think is the value you are seeing in fact).

It's not possible to accurately represent a value between those two and 4567.12 is one of them!

Cliff

PS BTW if you think that's bad try just simply 0.1 - this cannot be represented as a binary fraction because, like 1/3 is not representable for us decimal thinkers (it's the recurring 0.33333) in binary 0.1 is a recurring fraction. 3DCCCCCC and 3DCCCCCD are on either side. Even 64 bits cannot get close to this one. (bit of a shame when we do all our sums in multiples of 10!)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks for the information. I didn't really understand how float worked.

I read in "Embedded C Programming and the Atmel AVR" that a double is only 32 bits as well. If so is there any way to get more precision than 7 sig-figs?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

I read in "Embedded C Programming and the Atmel AVR" that a double is only 32 bits as well. If so is there any way to get more precision than 7 sig-figs?

I'd ask for my money back. 'double' has to be a minimum of 32bit (same as 'float') but some C compilers implement it as 64 bit. The Pro version of Imagecraft does and IAR also does. I think both CodeVision and GCC have 'double' as 32 bit so they are synonymous with 'float'.

While your first post didn't say I guess the mention of AVR Studio means you *may* be using GCC?

Of course another way, depending on the expected range of numbers, is to use scaled ints. Possibly 'long long' which is held in 64 bits.

This wouldn't be for GPS by any chance would it? If so then while you may want oodles of d.p. the angles pesumably never get beyond 360. Also the circumference of the Earth is about 27,000 miles which is roughly 50,000km so 50,000,000 metres which may set the largest value in the range you need to calculate? Meanwhile 2^64-1 (largest value in unsigned long long) is 18,446,744,073,709,551,615 which has room to hold 50,000,000 as 50,000,000 . 000,000,000,0 (10dp)

Cliff