AVR Freaks

Hi all

Well, I am working on an arduino library and I had to provide float support and for my own project I wanted to use the same library rather than writing new code and go through testing and stuff. If you do not like float, I am sorry but a lot of projects in arduino world use float. I understand why you are all against it but then again I needed it to make sure my library supports all data types. I was trying to push threads at the same time in arduino and here and probably did not spend enough time on avr freaks to elaborate that. My apologies for that.

Regarding the way dtostrf works, I find some that are unsafe at least in my point of view and some that are safe in my point of view. Now if you disagree with that, that is your choice and I respect it. But I was happy to see so many safe versions which take the buffer size from user.

About the naming of decimal point, I am not sure what is the best but the current version is definitely not correct.

With some great help from westfw in Arduino I did the followings:

1) I documented the dtoa_prf function which is the actual function when dtostrf is called. The code does not have any documentation and I think it has been implemented in a very smart and efficient way and without documentation it is hard to understand it.
2) I made a safe version and tried to test it as much as I could in few aspects.
   i) dtosrf and myDtostrf produce the same exact results when size is not an issue. (it was tested for hours by generating random float numbers)
   ii) NAN, INF, -INF are tested with different sizes to make sure correct result is obtained
   iii) size was tested on some variables (I have not seen any suspicious behavior but I think more testing is required which hopefully happens over time when I am testing other parts of the project)
3) finding out that another person has faced the same issue but the whole thread is in german 

#define FTOA_MINUS  1 // the number is negative
#define FTOA_ZERO 2 // the number is zero
#define FTOA_INF  4 // the number is infinity
#define FTOA_NAN  8 // it is not a number. The float format (IEEE754) has cases where the number is invalid and does not translate to a number. This flag is set in those cases
#define FTOA_CARRY  16 // this is set when the number has been rounded up when printed by __ftoa_engine // Carry was to master position.



#define DTOA_SPACE  0x01     /* put space for positives  */
#define DTOA_PLUS   0x02     /* put '+' for positives    */
#define DTOA_UPPER  0x04     /* use uppercase letters    */
#define DTOA_ZFILL  0x08     /* fill zeroes              */
#define DTOA_LEFT   0x10     /* adjust to left           */

#define       DTOA_EWIDTH     (-1) /* Width too small   */
#define       DTOA_NONFINITE  (-2) /* Value is NaN or Inf      */


extern "C" int __ftoa_engine (double val, char *buf, unsigned char prec, unsigned char maxdgs); // there is an interesting concept behind this line, please refer to http://www.geeksforgeeks.org/extern-c-in-c/

typedef union  
{
	float asFloat;
	uint32_t asLongInteger;
	uint8_t asByteArray[4];
} floatUnion;


floatUnion myNumber;
bool toggle;

void setup() 
{
	Serial.begin(115200);	
}


// the loop function runs over and over again forever
void loop() 
{
	if(toggle)
		myNumber.asFloat = (float) random(0xFFFFFFFF, 0x7FFFFFFF) / (float) random(0xFFFFFFFF, 0x7FFFFFFF);
	else
		myNumber.asFloat = (float) random(0xFFFFFFFF, 0x7FFFFFFF) * (float) random(0xFFFFFFFF, 0x7FFFFFFF);


	toggle = !toggle;

	// myNumber.asLongInteger = 0xFFFFFFFF; // NAN
	// myNumber.asLongInteger = 0x7F800000; // INF
	// myNumber.asLongInteger = 0xFF800000; // -INF
	// myNumber.asLongInteger = 0x00000000;  // 0

	int8_t buffer1[30] = {'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 
						  'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 
						  'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 0};

	int8_t buffer2[30] = {'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 
						'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 
						'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 0};

	bool didItFit;


	// int exponent = __ftoa_engine(myNumber.asFloat, buffer1, 7, 3);

	// if(buffer1[0] & FTOA_CARRY)
	// {
	// 	Serial.println("overflow");
	// }
	// Serial.println(buffer1[0], HEX);
	// Serial.println("buffer:");
	// Serial.write((uint8_t*)buffer1, 9);
	// Serial.println("End");

	dtostrf(myNumber.asFloat, 0, 4, buffer1);
	myDtostrf(myNumber.asFloat, 0, 4, buffer2, 30, didItFit);


	if(!didItFit)
	{
		Serial.print("the float number in bytes is: 0x");
		Serial.print(myNumber.asByteArray[3], HEX);
		Serial.print(", 0x");
		Serial.print(myNumber.asByteArray[2], HEX);
		Serial.print(", 0x");
		Serial.print(myNumber.asByteArray[1], HEX);
		Serial.print(", 0x");
		Serial.println(myNumber.asByteArray[0], HEX);
		Serial.print("; 0x");
		Serial.println(myNumber.asLongInteger, HEX);
		Serial.println("myDtostrf said data does not fit. buffer size 30. prec 4. width 0");
	}


	for(uint8_t i = 0; i < 30; i++)
	{
		if(buffer1[i] != buffer2[i])
		{
			Serial.print("the float number in bytes is: 0x");
			Serial.print(myNumber.asByteArray[3], HEX);
			Serial.print(", 0x");
			Serial.print(myNumber.asByteArray[2], HEX);
			Serial.print(", 0x");
			Serial.print(myNumber.asByteArray[1], HEX);
			Serial.print(", 0x");
			Serial.println(myNumber.asByteArray[0], HEX);
			Serial.print("; 0x");
			Serial.println(myNumber.asLongInteger, HEX);
			Serial.println("dtostrf and myDtostrf results are not matching");
			Serial.write( (uint8_t *)buffer1, 30);
			Serial.println("");
			Serial.write( (uint8_t *)buffer2, 30);
			Serial.println("");
		}
	}

	// Serial.print( (char *) buffer1);
	// Serial.println('$');
	// Serial.print( (char *) buffer2);
	// Serial.println('$');
	// while(1);
}




/*
   int __ftoa_engine (double val, char *buf,
                      unsigned char prec, unsigned char maxdgs)
 Input:
    val    - value to convert
    buf    - output buffer address
    prec   - precision: number of decimal digits is 'prec + 1'
    maxdgs - (0 if unused) precision restriction for "%f" specification

 Output:
    return     - decimal exponent of first digit
    buf[0]     - flags (FTOA_***)
    buf[1],... - decimal digits
    Number of digits:
	maxdgs == 0 ? prec+1 :
	(buf[0] & FTOA_CARRY) == 0 || buf[1] != '1' ?
	    aver(1, maxdgs+exp, prec+1) :
	    aver(1, masdgs+exp-1, prec+1)

 Notes:
    * Output string is not 0-terminated. For possibility of user's buffer
    usage in any case.
    * If used, 'maxdgs' is a number of digits for value with zero exponent.
*/
/*
the infromation above is provided by __ftoa_engine implimentation file
maxdgs is very smart and it limits number of digits after printing the number in non scientific form
prec is limited to 7, so any number higher than 7 is set to 7
the output become unusuall when the number is 0.999999 and maxdgs is set 2, we expect to get [FTOA_CARRY][1][0][0] but the function gives you [FTOA_CARRY][1][0]
this special condition is checked by "(buf[0] & FTOA_CARRY) || buf[1] == '1"' 

*/


// user must make sure width < size (very important) width has to be less than size not even less than equal
// The size must include a null terminating character
// didItFit returns false if data does not fir and return true if the data fits
// take note that the function print as much as it can in the buffer and returns a false if the data does not fit
int myDtoa_prf(const double val, int8_t *s, uint8_t width, const uint8_t prec, uint8_t flags, const uint8_t size, bool& didItFit)
{
	uint8_t nBytesPrinted = 1; // this is done because we need to reserve a place for a null at the end of the data buffer
	int16_t exponent; // this is int16_t because the function return int16_t ("exp" original name)
	uint8_t vType; // stores the flags from  __ftoa_engine ("vtype" original name) (FTOA_MINUS, FTOA_ZERO, FTOA_INF, FTOA_NAN, FTOA_CARRY)
	int16_t printedNumberSize_index = 0; // this number stores the size of the number after it is being printed and after that as an index for printing ("n" original name)
	uint8_t sign; // this variable stores the sign initially but it has multiple uses as the function go on.
	uint8_t nDigits; // You need to know how __ftoa_engine works to understand this variable. This variable store the number of digits printed from __ftoa_engine output. However it has been used along the way. 
	// Take note that not all the printed numbers from __ftoa_engine may not be used in final version ("ndigs" original name)
	uint8_t buf[9]; // this buffer stores the output from __ftoa_engine. Take note that the size is 9, because that is the maximum output from __ftoa_engine

	didItFit = false;

	// nDigits is supposed to store the number of digits printed from __ftoa_engine. So this is one step required for final value
	// moreover this number (prec + 1: 60) is needed to call __ftoa_engine. So first we do this step and call __ftoa_engine with nDigits
	nDigits = prec < 60 ? prec + 1 : 60;

	exponent = __ftoa_engine(val, (int8_t *)buf, 7, nDigits); // please read the description provided about __ftoa_engine

	vType = buf[0]; // the first byte in the buf contains flags produced by __ftoa_engine: FTOA_MINUS, FTOA_ZERO, FTOA_INF, FTOA_NAN, FTOA_CARRY

	sign = 0; // we set the sign to zero meaning that it does not have to be printed
	if( ( vType & (FTOA_MINUS | FTOA_NAN) ) == FTOA_MINUS ) // we are checking if __ftoa_engine is not FTOA_NAN (not a number) and it is negative. For NAN case negative sign is not printed that is why we make sure it is not NAN
		sign = '-'; // setting the sign to '-'. Which is also a non zero number. Take not, sign is only not printed, if sign is zero.
	else if(flags & DTOA_PLUS) // we know the number is positive, therefore we check if from the formatting provided by the user the '+' should be printed.
		sign = '+'; // setting the sign to '+'. Which is also a non zero number. Take not, sign is only not printed, if sign is zero.
	else if (flags & DTOA_SPACE) // We know the number is positive and the user does not want to print '+' but he may want to print ' '(space) instead.
		sign = ' '; // setting the sign to ' '(space). Which is also a non zero number. Take not, sign is only not printed, if sign is zero.

	
	////////// Taking care of Not a Number (NAN) ////////// 
	if(vType & FTOA_NAN) { // checking if the number is NAN
		nDigits = sign ? 4 : 3; // The function is going to print "(sign)"+"NAN". Sign may be ' '(space) or '+'. That would be 3 without sign or 4 with sign
		width = (width > nDigits) ? width - nDigits : 0; // the number of spaces which should be printed to take care of minimum width requested by the user
		
		if( !(flags & DTOA_LEFT) ) { // checking if the minimum width has to be adjusted from right side, if yest, then we need to print some ' '(space)
			while(width) { // making sure enough number of space are printed
				*s++ = ' '; // printing a space
				width--; // decrement the number of the spaces printed
				// nBytesPrinted++; // increamenting number of bytes printed in buffer is not neccessary because we have enough space that is why these spaces get printed. Assuming that user has provided size > width
			}
		}

		if(sign && (nBytesPrinted < size) ) {// checking if sign is non zero, if non zero, it has to be printed and it stores the ascii code for what has to be printed. In this case ' 'space or '+'
			*s++ = sign; // printing the sign
			nBytesPrinted++; // increamenting number of bytes printed in buffer
		}
		if( (flags & DTOA_UPPER) ) { // checking if it has to be printed in Upper case or lower case
			if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
				*s++ = 'N'; // printing in buffer
				nBytesPrinted++; // increamenting number of bytes printed in buffer
				if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
					*s++ = 'A'; // printing in buffer
					nBytesPrinted++;  // increamenting number of bytes printed in buffer
					if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
						*s++ = 'N'; // printing in buffer. At this point the nBytesPrinted is not important because only null has to be printed or the space to meet width in which size has been taken care of. Assuming that user has provided size > width
						didItFit = true;
					}
				}
			}
		} else {
			if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
				*s++ = 'n'; // printing in buffer
				nBytesPrinted++; // increamenting number of bytes printed in buffer
				if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
					*s++ = 'a'; // printing in buffer
					nBytesPrinted++;  // increamenting number of bytes printed in buffer
					if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
						*s++ = 'n'; // printing in buffer. At this point the nBytesPrinted is not important because only null has to be printed or the space to meet width in which size has been taken care of. Assuming that user has provided size > width
						didItFit = true;
					}
				}
			}
		}

		while(width) { // This condition is correct only if it has to be adjust from left and minimum width is not reached and it is making sure enough number of space are printed
			*s++ = ' '; // printing a space
			width--; // decrement the number of the spaces printed
			// nBytesPrinted++; // increamenting number of bytes printed in buffer but it does not matter any more
		}
		*s = 0; // printing the null at the very end
		return DTOA_NONFINITE; // informing user that the number is non finite 
	}
	////////// END: Taking care of Not a Number (NAN) ////////// 

	////////// Taking care of Infinity (INF) ////////// 
	if (vType & FTOA_INF) {
		nDigits = sign ? 4 : 3; // The function is going to print "(sign)"+"INF". Sign may be '-', ' '(space) or '+'. That would be 3 without sign or 4 with sign
		width = (width > nDigits) ? width - nDigits : 0; // the number of spaces which should be printed to take care of minimum width requested by the user
		
		if( !(flags & DTOA_LEFT) ) { // checking if the minimum width has to be adjusted from right side, if yest, then we need to print some ' '(space)
			while(width) { // making sure enough number of space are printed
				*s++ = ' '; // printing a space
				width--; // decrement the number of the spaces printed
				// nBytesPrinted++; // increamenting number of bytes printed in buffer is not neccessary because we have enough space that is why these spaces get printed. Assuming that user has provided size > width
			}
		}

		if(sign && (nBytesPrinted < size) ) {// checking if sign is non zero, if non zero, it has to be printed and it stores the ascii code for what has to be printed. In this case ' 'space or '+'
			*s++ = sign; // printing the sign
			nBytesPrinted++; // increamenting number of bytes printed in buffer
		}
		if( (flags & DTOA_UPPER) ) { // checking if it has to be printed in Upper case or lower case
			if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
				*s++ = 'I'; // printing in buffer
				nBytesPrinted++; // increamenting number of bytes printed in buffer
				if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
					*s++ = 'N'; // printing in buffer
					nBytesPrinted++;  // increamenting number of bytes printed in buffer
					if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
						*s++ = 'F'; // printing in buffer. At this point the nBytesPrinted is not important because only null has to be printed or the space to meet width in which size has been taken care of. Assuming that user has provided size > width
						didItFit = true;
					}
				}
			}
		} else {
			if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
				*s++ = 'i'; // printing in buffer
				nBytesPrinted++; // increamenting number of bytes printed in buffer
				if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
					*s++ = 'n'; // printing in buffer
					nBytesPrinted++;  // increamenting number of bytes printed in buffer
					if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
						*s++ = 'f'; // printing in buffer. At this point the nBytesPrinted is not important because only null has to be printed or the space to meet width in which size has been taken care of. Assuming that user has provided size > width
						didItFit = true;
					}
				}
			}
		}

		while(width) { // This condition is correct only if it has to be adjust from left and minimum width is not reached and it is making sure enough number of space are printed
			*s++ = ' '; // printing a space
			width--; // decrement the number of the spaces printed
			// incrementing++; // increamenting number of bytes printed in buffer but it does not matter any more
		}
		*s = 0; // printing the null at the very end
		return DTOA_NONFINITE; // informing user that the number is non finite 
	}
	////////// END: Taking care of Infinity (INF) //////////

	printedNumberSize_index = (sign ? 1 : 0) + (exponent>0 ? exponent+1 : 1) + (prec ? prec+1 : 0); // we are calculating the number of bytes we should print if we are dealing with a number
	// if there is a sign, we add one. 
	// we add number of digits on the left of the decimal point. exponent + 1 if it is positive, because the format is x.xxxxx so if it is 1, the number become xx.xxxx which means 2 digits befor and if it is 0 or negative we either have x.xxxx or 0.xxxxxxx which means 1 digit
	// we add the number of digits on the right of decimal point and the decimal point (decimal point is needed only if precision is positive)
	
	if( printedNumberSize_index < size) didItFit = true; // if the number fits we set didItFit to true 

	width = width > printedNumberSize_index ? width - printedNumberSize_index : 0; // the number of spaces which should be printed to take care of minimum width requested by the user

	if( !(flags & DTOA_LEFT) && !(flags & DTOA_ZFILL) ) { // checking if the minimum width has to be adjusted from right side and the user wants space instead of 0, if yest, then we need to print some ' '(space)
		while(width) { // making sure enough number of space are printed
			*s++ = ' '; // printing a space
			width--; // decrement the number of the spaces printed
			// incrementing++; // increamenting number of bytes printed in buffer is not neccessary because we have enough space that is why these spaces get printed. Assuming that user has provided size > width
		}
	}

	if(sign && (nBytesPrinted < size) ) {// checking if sign is non zero, if non zero, it has to be printed and it stores the ascii code for what has to be printed. In this case ' 'space or '+'
		*s++ = sign; // printing the sign
		nBytesPrinted++; // increamenting number of bytes printed in buffer
	}

	if( !(flags & DTOA_LEFT) ) { // checking if the minimum width has to be adjusted from right side, if yest, then we need to print some '0'
		while(width) { // making sure enough number of 0 are printed
			*s++ = '0'; // printing a 0
			width--; // decrement the number of the 0 printed
			// incrementing++; // increamenting number of bytes printed in buffer is not neccessary because we have enough space that is why these 0s get printed. Assuming that user has provided size > width
		}
	}

	nDigits += exponent;           /* exp is resticted approx. -40 .. +40    */
	
	sign = buf[1]; // here sign is holding the ascii code for the first digit from __ftoa_engine output. this has nothing to do with sign
	if ((vType & FTOA_CARRY) && (sign == '1' )) // this is too check for a very special case. if val was 0.999 and you call __ftoa_engine with mxdigit = 2, the data buffer returned is going to store [1][0] instead of [1][0][0]. take note that 2 things has happened, 1) the number has been rounded up (FTOA_CARRY) 2) the output does not have 3 digits in the output, it has only 2, that is why we subtract 1
		nDigits -= 1; // not sure why "nDigits--" is not used"


	if ((signed char)nDigits < 1) // this condition is checking if nDigits is 0 or more than 128 all in one shot. I am not sure why zero become 1 and also not sure why higher numbers are promoted to 1 instead of zero. would be glad if someone can point it out
		// take note that if prec < 60, nDigits = prec + 1 + exponent. this only become more than 128 if exponent is a negative number and "prec + 1 + exponent" become negative (since nDigits is unsigned) or if exponent is a very big positive number. the original programmer has documented "//exp is resticted approx. -40 .. +40" which probably is done through this
		nDigits = 1; // at this point nDigits shows number of digits which are printed from the __ftoa_engine output
	else if (nDigits > 8) // of course we cannot have more than 8 digits because __ftoa_engine is limited to 8 only so here we make sure the numbers higher become 8 which is the actual maximum
		nDigits = 8; // at this point nDigits shows number of digits which are printed from the __ftoa_engine output
	// if nDigits was between 1 and 8 it would be untouched. 


	printedNumberSize_index = exponent > 0 ? exponent : 0; // printedNumberSize_index from here is used as an index for printing. index is actually power of 10. ex. decimal point is printed at "printedNumberSize_index = -1" where the decimal point has to be printed before digit.
	do {
        if( (printedNumberSize_index == -1) && (nBytesPrinted < size) ) // printing the decimal point when the right time comes
        {
            *s++ = '.'; // printing the decimal point
            nBytesPrinted++; // increamenting number of bytes printed in buffer
        }

        flags = (printedNumberSize_index <= exponent && printedNumberSize_index > exponent - nDigits) ? buf[exponent - printedNumberSize_index + 1] : '0';
        // placing the digit that has to be printed in flags
        if ( --printedNumberSize_index < -prec ) // checking if this should be the last digit, take note last digit is printed outside do while
            break;

        if(nBytesPrinted < size)
        {
        	*s++ = flags; // placing the digit
        	nBytesPrinted++; // increamenting number of bytes printed in buffer
        }

     } while (1);

     if ( printedNumberSize_index == exponent && (sign > '5' || (sign == '5' && !(vType & FTOA_CARRY))) ) // checking for a special case where the first digit from __ftoa_engine is right after when printing finish. for example consider 0.005 and lets say the user choose 2 for prec. that prints as "0.00" however if correctly rounded it should be 0.01. this if take care of similar conditions. FTOA_CARRY also make sure it has not been rounded up to become 5.
        flags = '1';

     if(nBytesPrinted < size) *s++ = flags;

	while(width) { // This condition is correct only if it has to be adjust from left and minimum width is not reached and it is making sure enough number of space are printed
		*s++ = ' '; // printing a space
		width--; // decrement the number of the spaces printed
		// incrementing++; // increamenting number of bytes printed in buffer but it does not matter any more
	}

	if(nBytesPrinted <= size) *s++ = 0; // placing the last null character. Due to special case we check for "nBytesPrinted < 0". the special case is when size is equal to zero

	return 0;
     
}

int8_t* myDtostrf (double val, int8_t width, uint8_t prec, int8_t *sout, const uint8_t size)
{
	bool dummy;
	return myDtostrf(val, width, prec, sout, size, dummy);
}


int8_t* myDtostrf (double val, int8_t width, uint8_t prec, int8_t *sout, const uint8_t size, bool& didItFit)
{
	uint8_t flags;

	/* DTOA_UPPER: for compatibility with avr-libc <= 1.4 with NaNs   */
	flags = width < 0 ? DTOA_LEFT | DTOA_UPPER : DTOA_UPPER;
	myDtoa_prf (val, sout, abs(width), prec, flags, size, didItFit);
	return sout;
}

myDtoa_prf part of this code is exactly same as dtoa_prf and buy reading the comments you can easily understand the original dtoa_prf too.

There is only one part which I did not understand and that is 

if ((signed char)nDigits < 1) 
	nDigits = 1;
else if (nDigits > 8) 
	nDigits = 8;

I tested with

if ((signed char)nDigits < 0) 
	nDigits = 1;
else if (nDigits > 8) 
	nDigits = 8;

and it produce exact same results after testing with random float number, but I think it is faster because it can be branch if minus. (I am not very familiar with AVR assembly instruction yet but from what I recall from 6800 instruction set, branch if minus was faster than comparing and then branching)
I would be happy if anyone can point out why there is a 1 there instead of 0. I am afraid it is for a very special case and the random number generator has not generated that.

Moreover you may share your opinion about how the testing is carried out (in the first code posted)

And lastly,
https://www.mikrocontroller.net/topic/301125
 shows folks who solved the same problem. However my version has one difference. Their version does not print the number at all if it does not fit. My version it prints as much as it can and then set the bool argument which passed by reference to false.

I would like to thank you all for the support, suggestions and your efforts.