How to safely and reasonably convert a float or double to string or char array?

Go To Last Post
29 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi all

 

As you all know dtostrf convert a float to char array. However the problem is that it does not get the limit in the buffer and it keeps writing in the memory till the number is fully printed.

 

What is a safe way to make sure this does not happen and the dtostrf become safe. BTW the reasonable part in the question is to make sure solutions such as checking the boundary are out. The main reason for that being the fact that the boundary is dependent on the format and also it takes huge amount of time to compare floats. Also considering that dtostrf actually have to rip float apart before printing make me think it is either better to make a safe dtostrf or efficiently checking the number of digits.

 

Side note: I posted this on arduino forum and the answers are off topic and not related. https://forum.arduino.cc/index.php?topic=510327.0

 

thank you.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Avr-gcc doesn’t have double floats unless it is a new addition. Even so, you can gauge the maximum amount of chars it can emit. As well, there’s printf/sprintf where you can format the output, further giving a limit on the maximum number of chars emitted. Or make your buffer large enough.
If you are really unhappy with these solutions, write your own conversion functions - gcc is open source.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alireza.safadri wrote:
What is a safe way to make sure this does not happen and the dtostrf become safe.
Over allocate the buffer.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Am I missing something...

 

Quote:

The dtostrf() function converts the double value passed in val into an ASCII representationthat will be stored under s. The caller is responsible for providing sufficient storage in s.

my bold.

 

 

...?

 

'This forum helps those who help themselves.'

 

pragmatic  adjective dealing with things sensibly and realistically in a way that is based on practical rather than theoretical consideration.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Have you considered just not using floats at all ... ?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I decided to read the linked thread. Basically they’ve said much the same as we have. You’re trying to solve a problem downstream where you should be looking upstream.
As for printf/sprintf/snprintf with Arduino, i’m not sure what can be done there - they might hard code some build options that closes that door so your options might be to go to Atmel Studio and use Arduino libraries or write your own conversion function. Or don’t use floats. Do your values really need the dynamic range?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Depending on the range of the float and how many decimals you want, you could multiply by a power of 10 (for example, x100 to get 2 decimals) then cast to an int or long.

 

Then use ltoa or itoa to convert to string, and finally edit the string to insert the decimal point.

 

Alternatively, you could cast the original value to get the integer part, then mul the original x100 and modulus %100 to get the decimal part.

 

Naturally, I'm not sure this works and plenty of testing would be needed.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

@Kartman

It is very difficult to know how many digits will be in the number if that number is set by an external device. Moreover if a number due to some unknown/rare condition become higher than it should, it will screw the memory and that can be very dangerous. Using sprintf is a little bit too much over head and I do not find it accessory. If you check the link to the post in arduino forum, either arduino has implemented the dtostrf wrong or their documentation is wrong. I think probably their implementation is wrong, because I checked the implementation in here and it is different from the one in Arduino. I am fixing the issue but I do not want to invent the wheel. In dace the wheel has been invented if I find a way to use __ftoa_engin function in Arduino IDE then I can actually make the dtostrd or just use the implementation from previous link.
as for your second command, I think everyone consider the arduino dtostrf to work as the documentation and that is why what you say is correct based on documentation but not based on the way the actual function works/has been implemented. BTW I do not understand the meaning of "
You’re trying to solve a problem downstream where you should be looking upstream."

And honestly speaking no I do not need floats, but I have to ask a user to enter a float number. I can fake it to look like a float while it is not a float but would not that take so much of effort (longer code/more debugging/ more complex to maintain)? This could have been done for most of my current project, but I am making a menu library which should allow the user to use float and it is targeted for Arduino users. So they may want to use a float and that means flaot printing must be done safely. Some suggested dtostrfe but scientific notation is not that great either. I think I would just use a correctly implemented dtostrf.

 

@clawson

Considering the nature of float, how much buffer would you allocate?

 

@Brian Fairchild

How much buffer would you allocate to be safe? 50? 100? the nature of the float makes it difficult. specially considering the arduino implementation of dtostrf or their documentation is wrong. You may check the arduino forum link I left in my original post for more information.

 

@awneil

They just become float in the last stage to show the numbers to the user on screen. Before they become float, they are all integer numbers

 

@El Tangas

I think it makes the code more complicated and a lot of memory allocation would be required.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It think it should be pretty obvious to you by now that here doesn’t seem to be a canned solution that fits your requirements. Thus the solution is to write your own. The floating point format is documented and there’s plenty of example code for you to draw from.
You bemoan the performance of doing fp compares but yet we’re talking about a function that is quite slow anyways.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

@Kartman

I am actually working on mine. I have already looked at dtostrf implementations both in avr and also in elsewhere. In arduino IDE currently there is file called "ftoa_engine.h" and it has function called "__ftoa_engine". This function does most of the hard work and probably more efficient then what I could write. However I do not know how to include it or use it. If I learn to do that, the rest is done in few minutes. But I do not know how to include this file.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The code is open source - just look at the code for ftoa_engine! It's probably in avrlibc - you can Google that.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The avr-libc sources are on Savannah: http://savannah.nongnu.org/proje... , and the ftoa_engine specifically is here: http://svn.savannah.gnu.org/view...

(click view link on any revision to see the actual source).

 

EDIT: Corrected URLs. 

"He used to carry his guitar in a gunny sack, or sit beneath the tree by the railroad track. Oh the engineers would see him sitting in the shade, Strumming with the rhythm that the drivers made. People passing by, they would stop and say, "Oh, my, what that little country boy could play!" [Chuck Berry]

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

Last Edited: Mon. Nov 13, 2017 - 07:03 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alireza.safadri wrote:

How much buffer would you allocate to be safe? 50? 100? the nature of the float makes it difficult. specially considering the arduino implementation of dtostrf or their documentation is wrong. You may check the arduino forum link I left in my original post for more information.

But some people already pointed out that the width parameter is in fact the minimum width of the created string; not an upper limit. This has already been pointed out in another post. I don't agree that the implementation is wrong. I think you are misunderstanding the documentation.

 

What you should consider is this question: if you get a very large value, how should that be handled? Do you want to output the whole string? Then you need around 2kB of buffer to print in. Does is actually make sense to display a value that large?

  • If the answer is yes - go ahead and do that. Make sure to use a device with enough RAM.
  • If no - then that implies that you want to do an upper boundary check of the input value first, which you objected to for performance reason (which to me does not makes much sense...)

 

I cannot see any other reasonable decision path.

 

 

/Jakob Selbing

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

@Kartman and probably everyone else:

So here is the list of files which were needed to just use stuff available already:
ftoa_engine.h
http://svn.savannah.gnu.org/viewvc/avr-libc/trunk/avr-libc/common/ftoa_engine.h?revision=1218&view=markup

ftoa_engine.S
http://svn.savannah.gnu.org/viewvc/avr-libc/trunk/avr-libc/libc/stdlib/ftoa_engine.S?revision=2191&view=markup

macros.inc
http://svn.savannah.gnu.org/viewvc/avr-libc/trunk/avr-libc/common/macros.inc?revision=2542&view=markup
 

sectionname.h
http://svn.savannah.gnu.org/viewvc/avr-libc/trunk/avr-libc/common/sectionname.h?revision=2166&view=markup

#include <avr/io.h>
I could not find this one though. I thought this is supposed to be the easiest one.

So I started a new project and the code did not compile but not because something is missing. It could not understand the .S (assembly file) and the .INC file. So then I remembered that Arduino IDE and assembly do not go well together.

 

I also thought of just using the assembly part in a function and use "asm volatile( // code );", so after finishing I noticed the code does not compile mainly because it cannot connect the function's arguments to the the variables in assembly code (even though I used the same name). Which kind of make sense. 
So is there anyone here who knows how to connect the function arguments to assembly code? (we may need to relate the #defines to assembly code (I am not sure if this is done currently))
Or is there anyone who knows how we can trick the arduino IDE to use assembly code?
Or is there anyone who knows a way to use the "__ftoa_engine" function in Arduino IDE? Arduino dtosrf use this thing but the users cannot use it in the IDE environment. 

 

@JohanEkdahl the links are not working, but thank you I got an idea from the links.

 

@jaksel I think you have not read the whole post in arduino or the documentation. At this moment the documentation does not match the implementation.

the documentation says:  "prec determines the number of digits after the decimal sign" but in arduino it is implemented as"prec determines the number of digits after the decimal point". If you do not believe me, you may check the documentation and test the behavior of the dtostrf. The problem is the current dtostrf being very unsafe. You do not have a way to tell the dtostrf when to stop writing in the memory. So it keeps writing in the memory till that number finishes. This is the problem. There should be away to tell the dtostrf when the buffer ends and it has to stop there even if the number did not finish. It can return a false or a flag or something to say the operation was not successful but it must not go all over the place.  

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Why not just write the code in C and avoid the grief?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

getting there right now.

 

However I am sure the assembly functions beat the C code I write for 3 reasons:

1) I am writing the code (have to be realistic here :D )

2) It is in C++

3) I may not be able to take care of all the rare cases such as -0 and +0, + Infinity and - Infinity and lastly NaN.

 

But I am going to do it anyway.

 

this is my proposed function let me know what you think about it.

 

char* dtostrf   (double __val, signed char __width, unsigned char __prec, char * __s, unsigned char __maxSize, bool __result)
__val: same as current dtostrf
__width: same as current arduino dtostrf
__prec: same as current arduino dtostrf
__s: same as current arduino dtostrf
__maxSize: The number of bytes allocated for this number
__result: true if the process was successful and maxSize was large enough. Return false if the size was not enough.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

#include <avr/io.h>
I could not find this one though. I thought this is supposed to be the easiest one.

It has very little to do with what you're trying to do, but here it is:

 

http://svn.savannah.gnu.org/view...

"He used to carry his guitar in a gunny sack, or sit beneath the tree by the railroad track. Oh the engineers would see him sitting in the shade, Strumming with the rhythm that the drivers made. People passing by, they would stop and say, "Oh, my, what that little country boy could play!" [Chuck Berry]

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alireza.safadri wrote:
They just become float in the last stage to show the numbers to the user on screen. Before they become float, they are all integer numbers

Why on earth introduce float just for that??

 

surprise

 

Surely, there are easier ways to manually insert a radix point ... ??

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alireza.safadri wrote:

@jaksel I think you have not read the whole post in arduino or the documentation. At this moment the documentation does not match the implementation.

the documentation says:  "prec determines the number of digits after the decimal sign" but in arduino it is implemented as"prec determines the number of digits after the decimal point". If you do not believe me, you may check the documentation and test the behavior of the dtostrf. The problem is the current dtostrf being very unsafe. You do not have a way to tell the dtostrf when to stop writing in the memory. So it keeps writing in the memory till that number finishes. This is the problem. There should be away to tell the dtostrf when the buffer ends and it has to stop there even if the number did not finish. It can return a false or a flag or something to say the operation was not successful but it must not go all over the place.

I did read the post you linked to. I did look at the documentation. Did YOU read my full post?

 

alireza.safadri wrote:
"I took "The caller is responsible for providing sufficient storage in s" not as serious as I should. I thought if I do not provide enough width it automatically understand not to write in the rest of the memory. "

This strikes me as very odd. How on earth would the function "understand" not to write the rest of the memory? Do you understand that there is no run-time information passed with the buffer pointer that would indicate the buffer length? A buffer is just a pointer to memory. It contains no information regarding the buffer size. That's why the buffer size is usually passed as a separate parameter.

 

alireza.safadri wrote:
the documentation says:  "prec determines the number of digits after the decimal sign" but in arduino it is implemented as"prec determines the number of digits after the decimal point"

So you distinguish between "decimal sign" and "decimal point"? And exactly what would be the difference? Sounds like the same thing (the character ".") to me.

 

Either way, the function dtostr does NOT do any buffer overflow checking. In this case, I don't see the problem. Just use a buffer size that accommodates the largest possible string that you can normally handle. Then use range check to skip values that are larger than what can be accommodated in that string. How is that not the easiest option?

/Jakob Selbing

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

@JohanEkdahl Thank you.

@awneil Well, considering code readability and one of the libraries which is going to have general use, I must support float. (I am not sure if I wrote about the general library here or in Arduino forum)

@jaksel

I did read the post you linked to. I did look at the documentation

This strikes me as very odd. How on earth would the function "understand" not to write the rest of the memory? Do you understand that there is no run-time information passed with the buffer pointer that would indicate the buffer length? A buffer is just a pointer to memory. It contains no information regarding the buffer size. That's why the buffer size is usually passed as a separate parameter.

Ok, I think you did not read the whole post either carefully or fully. When I wrote that sentence in the same or previous post you can see that I was under the impression that width is the maximum width and not minimum. I also pointed out that I was not sure how the width works and some folks actually pointed it out as well. So, I thought the width is passing the size and we pass the buffer. I also assumed that providing enough storage is in respect to width. So if the width is 10 your array should be 10 byte long (11 if you need null at the end). Also my analogy was made because a float does not have a maximum with only few bytes differences (That is why you do not see me crying over itoa, I actually do but not as load :D ).

 

So you distinguish between "decimal sign" and "decimal point"? And exactly what would be the difference? Sounds like the same thing (the character ".") to me.

Actually having it as decimal sign makes the function safe, you may ask why? Let's say you set it to 5. That means 5 digits are only printed and in between there might be a decimal point and there might be a negative sign. So the largest array allocation to make sure nothing goes wrong is 5 + 2 = 7 (if null is not considered). I hope this clarified why I would live my life so much happier if it was a decimal sign and not decimal point.

 

Either way, the function dtostr does NOT do any buffer overflow checking. In this case, I don't see the problem. Just use a buffer size that accommodates the largest possible string that you can normally handle. Then use range check to skip values that are larger than what can be accommodated in that string. How is that not the easiest option?

There are many problems with your solution if you look at in the big picture. Firstly range checking becomes a very tedious job considering the precision. if the precision is changed you have to change the range or the buffer size. I honestly think this is very dangerous because you change dtosrf precision and then you have to go and change the code elsewhere to make sure it is error free. And regarding the largest possible string, it is even worst. Firstly how big is the largest possible string? then do you actually realize that you have to copy from the string to the actual string you want to use? And guess what all that could have happened in dtostrf. Depending on how dtostrf is implimented this actually happens once. dtostrf does not write in your buffer directly, it has its own buffer and then copies from there to the buffer you provide. Now you have to copy one more time from the buffer you provide to the buffer you actually need that float number in. I honestly think if you fix the dtosrf, the code looks cleaner, shorter and easier to maintain because once you for instance change the precision, you have to worry about nothing if the buffer is not big enough. Specially if rare cases may happen, you do not have to worry about anything also specially in debugging stage. It is very hard to find dtostrf buffer overflow. If you are lucky the whole code break, if you are not it may take ages for a bug to show itself.

 

 

Now on my dtostrf progress:

there are 3 ways to do this:
1) inefficient and in accurate:
This method is done by mathematically cracking up the float. A code example can be found here:
https://stackoverflow.com/questions/2302969/convert-a-float-to-a-string

2) More efficient with good accuracy (if not perfect) but limited range
In this method they use the largest integer supported on the software and then use ltoa to get the digits out. The limitation in range is due to 32bit or 64bit integer number. You can see the code here:
http://www.edaboard.com/thread5585.html

3) the accurate and the most efficient version:
There are algorithm used to do dtostrf and apparently the back story is kind of interesting. The initial algorithm was Dragon 2 and then Dragon 4. However it was then improved further and was named Grisu. The Grisu 2 and Grisu 3 also exist based on some of the documents I have read. The best link regarding Grisu is here:
http://www.ryanjuckett.com/programming/printing-floating-point-numbers/

A bench mark comparison between these can be found here 
https://github.com/miloyip/dtoa-benchmark

I either use method 3 or a hybrid version where less memory is used because Grisu needs some stuff stored in an array. At this point I am not sure what they are but I am reading on them. I also have a feeling that the arduino __floatEngine works with the same method because it appears to me that some numbers are stored however it could be method number 1. I can confirm this when I finish reading and understading Grisu

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alireza.safadri wrote:
considering code readability and one of the libraries which is going to have general use, I must support float. 

That doesn't make sense.

 

You said the only reason you were using float was for the display output

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

A couple of thoughts on the use of floating point numbers...

 

Quote:

If you think you need floating point maths to solve the problem you don't understand the problem.

 

...and...

 

Quote:

If actually do need floating point to solve the problem, now you have a problem you don't understand.

'This forum helps those who help themselves.'

 

pragmatic  adjective dealing with things sensibly and realistically in a way that is based on practical rather than theoretical consideration.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Very good.

 

Who are you quoting?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

awneil wrote:

Very good.

 

Who are you quoting?

 

It appeared a while ago on another forum (sorry, yes, I do visit elsewhere).

 

'This forum helps those who help themselves.'

 

pragmatic  adjective dealing with things sensibly and realistically in a way that is based on practical rather than theoretical consideration.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alireza.safadri wrote:

Actually having it as decimal sign makes the function safe, you may ask why? Let's say you set it to 5. That means 5 digits are only printed and in between there might be a decimal point and there might be a negative sign. So the largest array allocation to make sure nothing goes wrong is 5 + 2 = 7 (if null is not considered). I hope this clarified why I would live my life so much happier if it was a decimal sign and not decimal point.

The documentation says "prec determines the number of digits after the decimal sign". My interpretation is that "decimal sign" means decimal point. But YOU interpret it as "sign", i.e. sign for negative values. You say that the implementation assumes the former interpretation. Don't you think that you are simply misunderstanding the documentation, and that both the documentation AND implementation is actually correct?

alireza.safadri wrote:

There are many problems with your solution if you look at in the big picture. Firstly range checking becomes a very tedious job considering the precision.

Why would it? If you have prec = N, and your buffer size is M, then the maximum value you can safely convert would be sth like 10^(M - N - 1). How hard can that be?

 

alireza.safadri wrote:

if the precision is changed you have to change the range or the buffer size. I honestly think this is very dangerous because you change dtosrf precision and then you have to go and change the code elsewhere to make sure it is error free.

I have no idea what you're talking about. Each call to dtostrf has its own buffer, width, precision and so on. Remember - the caller is responsible for allocating enough buffer.

 

alireza.safadri wrote:

And regarding the largest possible string, it is even worst. Firstly how big is the largest possible string?

You are misunderstanding the point. When you do the call to dtostrf, what is the maximum string size that is of any interest in that context? That is application-specific, and up to you to decide.

/Jakob Selbing

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

jaksel wrote:
The documentation says "prec determines the number of digits after the decimal sign". My interpretation is that "decimal sign" means decimal point

​It looks like they are trying to accommodate the fact that some locales uses a symbol other than a dot (ie, a "point") to denote the decimal point.

 

eg, some nationalities use a comma.

 

I don't know if there is a preferred way to say "decimal point" in such cases - or if the term "decimal point" is still used by people who write it as, say, a comma ...

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

awneil wrote:
I don't know if there is a preferred way to say "decimal point" in such cases

Decimal mark?

"He used to carry his guitar in a gunny sack, or sit beneath the tree by the railroad track. Oh the engineers would see him sitting in the shade, Strumming with the rhythm that the drivers made. People passing by, they would stop and say, "Oh, my, what that little country boy could play!" [Chuck Berry]

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Which defines it in terms of the radix point - so back to square one.

 

frown

 

but they do give the alternative of radix character...

 

But nobody, it seems, says "decimal sign"; that is just plain confusing - and wrong 

 

EDIT

 

​and we all happily talk of "floating point" ...

Last Edited: Tue. Nov 14, 2017 - 10:46 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi all

 

Well, I am working on an arduino library and I had to provide float support and for my own project I wanted to use the same library rather than writing new code and go through testing and stuff. If you do not like float, I am sorry but a lot of projects in arduino world use float. I understand why you are all against it but then again I needed it to make sure my library supports all data types. I was trying to push threads at the same time in arduino and here and probably did not spend enough time on avr freaks to elaborate that. My apologies for that.

 

Regarding the way dtostrf works, I find some that are unsafe at least in my point of view and some that are safe in my point of view. Now if you disagree with that, that is your choice and I respect it. But I was happy to see so many safe versions which take the buffer size from user.

 

About the naming of decimal point, I am not sure what is the best but the current version is definitely not correct.

 

With some great help from westfw in Arduino I did the followings:

1) I documented the dtoa_prf function which is the actual function when dtostrf is called. The code does not have any documentation and I think it has been implemented in a very smart and efficient way and without documentation it is hard to understand it.
2) I made a safe version and tried to test it as much as I could in few aspects.
   i) dtosrf and myDtostrf produce the same exact results when size is not an issue. (it was tested for hours by generating random float numbers)
   ii) NAN, INF, -INF are tested with different sizes to make sure correct result is obtained
   iii) size was tested on some variables (I have not seen any suspicious behavior but I think more testing is required which hopefully happens over time when I am testing other parts of the project)
3) finding out that another person has faced the same issue but the whole thread is in german :)

 

#define FTOA_MINUS  1 // the number is negative
#define FTOA_ZERO 2 // the number is zero
#define FTOA_INF  4 // the number is infinity
#define FTOA_NAN  8 // it is not a number. The float format (IEEE754) has cases where the number is invalid and does not translate to a number. This flag is set in those cases
#define FTOA_CARRY  16 // this is set when the number has been rounded up when printed by __ftoa_engine // Carry was to master position.



#define DTOA_SPACE  0x01     /* put space for positives  */
#define DTOA_PLUS   0x02     /* put '+' for positives    */
#define DTOA_UPPER  0x04     /* use uppercase letters    */
#define DTOA_ZFILL  0x08     /* fill zeroes              */
#define DTOA_LEFT   0x10     /* adjust to left           */

#define       DTOA_EWIDTH     (-1) /* Width too small   */
#define       DTOA_NONFINITE  (-2) /* Value is NaN or Inf      */


extern "C" int __ftoa_engine (double val, char *buf, unsigned char prec, unsigned char maxdgs); // there is an interesting concept behind this line, please refer to http://www.geeksforgeeks.org/extern-c-in-c/

typedef union  
{
	float asFloat;
	uint32_t asLongInteger;
	uint8_t asByteArray[4];
} floatUnion;


floatUnion myNumber;
bool toggle;

void setup() 
{
	Serial.begin(115200);	
}


// the loop function runs over and over again forever
void loop() 
{
	if(toggle)
		myNumber.asFloat = (float) random(0xFFFFFFFF, 0x7FFFFFFF) / (float) random(0xFFFFFFFF, 0x7FFFFFFF);
	else
		myNumber.asFloat = (float) random(0xFFFFFFFF, 0x7FFFFFFF) * (float) random(0xFFFFFFFF, 0x7FFFFFFF);


	toggle = !toggle;

	// myNumber.asLongInteger = 0xFFFFFFFF; // NAN
	// myNumber.asLongInteger = 0x7F800000; // INF
	// myNumber.asLongInteger = 0xFF800000; // -INF
	// myNumber.asLongInteger = 0x00000000;  // 0

	int8_t buffer1[30] = {'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 
						  'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 
						  'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 0};

	int8_t buffer2[30] = {'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 
						'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 
						'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 0};

	bool didItFit;


	// int exponent = __ftoa_engine(myNumber.asFloat, buffer1, 7, 3);

	// if(buffer1[0] & FTOA_CARRY)
	// {
	// 	Serial.println("overflow");
	// }
	// Serial.println(buffer1[0], HEX);
	// Serial.println("buffer:");
	// Serial.write((uint8_t*)buffer1, 9);
	// Serial.println("End");

	dtostrf(myNumber.asFloat, 0, 4, buffer1);
	myDtostrf(myNumber.asFloat, 0, 4, buffer2, 30, didItFit);


	if(!didItFit)
	{
		Serial.print("the float number in bytes is: 0x");
		Serial.print(myNumber.asByteArray[3], HEX);
		Serial.print(", 0x");
		Serial.print(myNumber.asByteArray[2], HEX);
		Serial.print(", 0x");
		Serial.print(myNumber.asByteArray[1], HEX);
		Serial.print(", 0x");
		Serial.println(myNumber.asByteArray[0], HEX);
		Serial.print("; 0x");
		Serial.println(myNumber.asLongInteger, HEX);
		Serial.println("myDtostrf said data does not fit. buffer size 30. prec 4. width 0");
	}


	for(uint8_t i = 0; i < 30; i++)
	{
		if(buffer1[i] != buffer2[i])
		{
			Serial.print("the float number in bytes is: 0x");
			Serial.print(myNumber.asByteArray[3], HEX);
			Serial.print(", 0x");
			Serial.print(myNumber.asByteArray[2], HEX);
			Serial.print(", 0x");
			Serial.print(myNumber.asByteArray[1], HEX);
			Serial.print(", 0x");
			Serial.println(myNumber.asByteArray[0], HEX);
			Serial.print("; 0x");
			Serial.println(myNumber.asLongInteger, HEX);
			Serial.println("dtostrf and myDtostrf results are not matching");
			Serial.write( (uint8_t *)buffer1, 30);
			Serial.println("");
			Serial.write( (uint8_t *)buffer2, 30);
			Serial.println("");
		}
	}

	// Serial.print( (char *) buffer1);
	// Serial.println('$');
	// Serial.print( (char *) buffer2);
	// Serial.println('$');
	// while(1);
}




/*
   int __ftoa_engine (double val, char *buf,
                      unsigned char prec, unsigned char maxdgs)
 Input:
    val    - value to convert
    buf    - output buffer address
    prec   - precision: number of decimal digits is 'prec + 1'
    maxdgs - (0 if unused) precision restriction for "%f" specification

 Output:
    return     - decimal exponent of first digit
    buf[0]     - flags (FTOA_***)
    buf[1],... - decimal digits
    Number of digits:
	maxdgs == 0 ? prec+1 :
	(buf[0] & FTOA_CARRY) == 0 || buf[1] != '1' ?
	    aver(1, maxdgs+exp, prec+1) :
	    aver(1, masdgs+exp-1, prec+1)

 Notes:
    * Output string is not 0-terminated. For possibility of user's buffer
    usage in any case.
    * If used, 'maxdgs' is a number of digits for value with zero exponent.
*/
/*
the infromation above is provided by __ftoa_engine implimentation file
maxdgs is very smart and it limits number of digits after printing the number in non scientific form
prec is limited to 7, so any number higher than 7 is set to 7
the output become unusuall when the number is 0.999999 and maxdgs is set 2, we expect to get [FTOA_CARRY][1][0][0] but the function gives you [FTOA_CARRY][1][0]
this special condition is checked by "(buf[0] & FTOA_CARRY) || buf[1] == '1"' 

*/


// user must make sure width < size (very important) width has to be less than size not even less than equal
// The size must include a null terminating character
// didItFit returns false if data does not fir and return true if the data fits
// take note that the function print as much as it can in the buffer and returns a false if the data does not fit
int myDtoa_prf(const double val, int8_t *s, uint8_t width, const uint8_t prec, uint8_t flags, const uint8_t size, bool& didItFit)
{
	uint8_t nBytesPrinted = 1; // this is done because we need to reserve a place for a null at the end of the data buffer
	int16_t exponent; // this is int16_t because the function return int16_t ("exp" original name)
	uint8_t vType; // stores the flags from  __ftoa_engine ("vtype" original name) (FTOA_MINUS, FTOA_ZERO, FTOA_INF, FTOA_NAN, FTOA_CARRY)
	int16_t printedNumberSize_index = 0; // this number stores the size of the number after it is being printed and after that as an index for printing ("n" original name)
	uint8_t sign; // this variable stores the sign initially but it has multiple uses as the function go on.
	uint8_t nDigits; // You need to know how __ftoa_engine works to understand this variable. This variable store the number of digits printed from __ftoa_engine output. However it has been used along the way. 
	// Take note that not all the printed numbers from __ftoa_engine may not be used in final version ("ndigs" original name)
	uint8_t buf[9]; // this buffer stores the output from __ftoa_engine. Take note that the size is 9, because that is the maximum output from __ftoa_engine

	didItFit = false;

	// nDigits is supposed to store the number of digits printed from __ftoa_engine. So this is one step required for final value
	// moreover this number (prec + 1: 60) is needed to call __ftoa_engine. So first we do this step and call __ftoa_engine with nDigits
	nDigits = prec < 60 ? prec + 1 : 60;

	exponent = __ftoa_engine(val, (int8_t *)buf, 7, nDigits); // please read the description provided about __ftoa_engine

	vType = buf[0]; // the first byte in the buf contains flags produced by __ftoa_engine: FTOA_MINUS, FTOA_ZERO, FTOA_INF, FTOA_NAN, FTOA_CARRY

	sign = 0; // we set the sign to zero meaning that it does not have to be printed
	if( ( vType & (FTOA_MINUS | FTOA_NAN) ) == FTOA_MINUS ) // we are checking if __ftoa_engine is not FTOA_NAN (not a number) and it is negative. For NAN case negative sign is not printed that is why we make sure it is not NAN
		sign = '-'; // setting the sign to '-'. Which is also a non zero number. Take not, sign is only not printed, if sign is zero.
	else if(flags & DTOA_PLUS) // we know the number is positive, therefore we check if from the formatting provided by the user the '+' should be printed.
		sign = '+'; // setting the sign to '+'. Which is also a non zero number. Take not, sign is only not printed, if sign is zero.
	else if (flags & DTOA_SPACE) // We know the number is positive and the user does not want to print '+' but he may want to print ' '(space) instead.
		sign = ' '; // setting the sign to ' '(space). Which is also a non zero number. Take not, sign is only not printed, if sign is zero.

	
	////////// Taking care of Not a Number (NAN) ////////// 
	if(vType & FTOA_NAN) { // checking if the number is NAN
		nDigits = sign ? 4 : 3; // The function is going to print "(sign)"+"NAN". Sign may be ' '(space) or '+'. That would be 3 without sign or 4 with sign
		width = (width > nDigits) ? width - nDigits : 0; // the number of spaces which should be printed to take care of minimum width requested by the user
		
		if( !(flags & DTOA_LEFT) ) { // checking if the minimum width has to be adjusted from right side, if yest, then we need to print some ' '(space)
			while(width) { // making sure enough number of space are printed
				*s++ = ' '; // printing a space
				width--; // decrement the number of the spaces printed
				// nBytesPrinted++; // increamenting number of bytes printed in buffer is not neccessary because we have enough space that is why these spaces get printed. Assuming that user has provided size > width
			}
		}

		if(sign && (nBytesPrinted < size) ) {// checking if sign is non zero, if non zero, it has to be printed and it stores the ascii code for what has to be printed. In this case ' 'space or '+'
			*s++ = sign; // printing the sign
			nBytesPrinted++; // increamenting number of bytes printed in buffer
		}
		if( (flags & DTOA_UPPER) ) { // checking if it has to be printed in Upper case or lower case
			if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
				*s++ = 'N'; // printing in buffer
				nBytesPrinted++; // increamenting number of bytes printed in buffer
				if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
					*s++ = 'A'; // printing in buffer
					nBytesPrinted++;  // increamenting number of bytes printed in buffer
					if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
						*s++ = 'N'; // printing in buffer. At this point the nBytesPrinted is not important because only null has to be printed or the space to meet width in which size has been taken care of. Assuming that user has provided size > width
						didItFit = true;
					}
				}
			}
		} else {
			if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
				*s++ = 'n'; // printing in buffer
				nBytesPrinted++; // increamenting number of bytes printed in buffer
				if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
					*s++ = 'a'; // printing in buffer
					nBytesPrinted++;  // increamenting number of bytes printed in buffer
					if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
						*s++ = 'n'; // printing in buffer. At this point the nBytesPrinted is not important because only null has to be printed or the space to meet width in which size has been taken care of. Assuming that user has provided size > width
						didItFit = true;
					}
				}
			}
		}

		while(width) { // This condition is correct only if it has to be adjust from left and minimum width is not reached and it is making sure enough number of space are printed
			*s++ = ' '; // printing a space
			width--; // decrement the number of the spaces printed
			// nBytesPrinted++; // increamenting number of bytes printed in buffer but it does not matter any more
		}
		*s = 0; // printing the null at the very end
		return DTOA_NONFINITE; // informing user that the number is non finite 
	}
	////////// END: Taking care of Not a Number (NAN) ////////// 

	////////// Taking care of Infinity (INF) ////////// 
	if (vType & FTOA_INF) {
		nDigits = sign ? 4 : 3; // The function is going to print "(sign)"+"INF". Sign may be '-', ' '(space) or '+'. That would be 3 without sign or 4 with sign
		width = (width > nDigits) ? width - nDigits : 0; // the number of spaces which should be printed to take care of minimum width requested by the user
		
		if( !(flags & DTOA_LEFT) ) { // checking if the minimum width has to be adjusted from right side, if yest, then we need to print some ' '(space)
			while(width) { // making sure enough number of space are printed
				*s++ = ' '; // printing a space
				width--; // decrement the number of the spaces printed
				// nBytesPrinted++; // increamenting number of bytes printed in buffer is not neccessary because we have enough space that is why these spaces get printed. Assuming that user has provided size > width
			}
		}

		if(sign && (nBytesPrinted < size) ) {// checking if sign is non zero, if non zero, it has to be printed and it stores the ascii code for what has to be printed. In this case ' 'space or '+'
			*s++ = sign; // printing the sign
			nBytesPrinted++; // increamenting number of bytes printed in buffer
		}
		if( (flags & DTOA_UPPER) ) { // checking if it has to be printed in Upper case or lower case
			if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
				*s++ = 'I'; // printing in buffer
				nBytesPrinted++; // increamenting number of bytes printed in buffer
				if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
					*s++ = 'N'; // printing in buffer
					nBytesPrinted++;  // increamenting number of bytes printed in buffer
					if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
						*s++ = 'F'; // printing in buffer. At this point the nBytesPrinted is not important because only null has to be printed or the space to meet width in which size has been taken care of. Assuming that user has provided size > width
						didItFit = true;
					}
				}
			}
		} else {
			if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
				*s++ = 'i'; // printing in buffer
				nBytesPrinted++; // increamenting number of bytes printed in buffer
				if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
					*s++ = 'n'; // printing in buffer
					nBytesPrinted++;  // increamenting number of bytes printed in buffer
					if(nBytesPrinted < size) { // checking if we can print, take note place is reserved for null at the end of buffer
						*s++ = 'f'; // printing in buffer. At this point the nBytesPrinted is not important because only null has to be printed or the space to meet width in which size has been taken care of. Assuming that user has provided size > width
						didItFit = true;
					}
				}
			}
		}

		while(width) { // This condition is correct only if it has to be adjust from left and minimum width is not reached and it is making sure enough number of space are printed
			*s++ = ' '; // printing a space
			width--; // decrement the number of the spaces printed
			// incrementing++; // increamenting number of bytes printed in buffer but it does not matter any more
		}
		*s = 0; // printing the null at the very end
		return DTOA_NONFINITE; // informing user that the number is non finite 
	}
	////////// END: Taking care of Infinity (INF) //////////

	printedNumberSize_index = (sign ? 1 : 0) + (exponent>0 ? exponent+1 : 1) + (prec ? prec+1 : 0); // we are calculating the number of bytes we should print if we are dealing with a number
	// if there is a sign, we add one. 
	// we add number of digits on the left of the decimal point. exponent + 1 if it is positive, because the format is x.xxxxx so if it is 1, the number become xx.xxxx which means 2 digits befor and if it is 0 or negative we either have x.xxxx or 0.xxxxxxx which means 1 digit
	// we add the number of digits on the right of decimal point and the decimal point (decimal point is needed only if precision is positive)
	
	if( printedNumberSize_index < size) didItFit = true; // if the number fits we set didItFit to true 

	width = width > printedNumberSize_index ? width - printedNumberSize_index : 0; // the number of spaces which should be printed to take care of minimum width requested by the user

	if( !(flags & DTOA_LEFT) && !(flags & DTOA_ZFILL) ) { // checking if the minimum width has to be adjusted from right side and the user wants space instead of 0, if yest, then we need to print some ' '(space)
		while(width) { // making sure enough number of space are printed
			*s++ = ' '; // printing a space
			width--; // decrement the number of the spaces printed
			// incrementing++; // increamenting number of bytes printed in buffer is not neccessary because we have enough space that is why these spaces get printed. Assuming that user has provided size > width
		}
	}

	if(sign && (nBytesPrinted < size) ) {// checking if sign is non zero, if non zero, it has to be printed and it stores the ascii code for what has to be printed. In this case ' 'space or '+'
		*s++ = sign; // printing the sign
		nBytesPrinted++; // increamenting number of bytes printed in buffer
	}

	if( !(flags & DTOA_LEFT) ) { // checking if the minimum width has to be adjusted from right side, if yest, then we need to print some '0'
		while(width) { // making sure enough number of 0 are printed
			*s++ = '0'; // printing a 0
			width--; // decrement the number of the 0 printed
			// incrementing++; // increamenting number of bytes printed in buffer is not neccessary because we have enough space that is why these 0s get printed. Assuming that user has provided size > width
		}
	}

	nDigits += exponent;           /* exp is resticted approx. -40 .. +40    */
	
	sign = buf[1]; // here sign is holding the ascii code for the first digit from __ftoa_engine output. this has nothing to do with sign
	if ((vType & FTOA_CARRY) && (sign == '1' )) // this is too check for a very special case. if val was 0.999 and you call __ftoa_engine with mxdigit = 2, the data buffer returned is going to store [1][0] instead of [1][0][0]. take note that 2 things has happened, 1) the number has been rounded up (FTOA_CARRY) 2) the output does not have 3 digits in the output, it has only 2, that is why we subtract 1
		nDigits -= 1; // not sure why "nDigits--" is not used"


	if ((signed char)nDigits < 1) // this condition is checking if nDigits is 0 or more than 128 all in one shot. I am not sure why zero become 1 and also not sure why higher numbers are promoted to 1 instead of zero. would be glad if someone can point it out
		// take note that if prec < 60, nDigits = prec + 1 + exponent. this only become more than 128 if exponent is a negative number and "prec + 1 + exponent" become negative (since nDigits is unsigned) or if exponent is a very big positive number. the original programmer has documented "//exp is resticted approx. -40 .. +40" which probably is done through this
		nDigits = 1; // at this point nDigits shows number of digits which are printed from the __ftoa_engine output
	else if (nDigits > 8) // of course we cannot have more than 8 digits because __ftoa_engine is limited to 8 only so here we make sure the numbers higher become 8 which is the actual maximum
		nDigits = 8; // at this point nDigits shows number of digits which are printed from the __ftoa_engine output
	// if nDigits was between 1 and 8 it would be untouched. 


	printedNumberSize_index = exponent > 0 ? exponent : 0; // printedNumberSize_index from here is used as an index for printing. index is actually power of 10. ex. decimal point is printed at "printedNumberSize_index = -1" where the decimal point has to be printed before digit.
	do {
        if( (printedNumberSize_index == -1) && (nBytesPrinted < size) ) // printing the decimal point when the right time comes
        {
            *s++ = '.'; // printing the decimal point
            nBytesPrinted++; // increamenting number of bytes printed in buffer
        }

        flags = (printedNumberSize_index <= exponent && printedNumberSize_index > exponent - nDigits) ? buf[exponent - printedNumberSize_index + 1] : '0';
        // placing the digit that has to be printed in flags
        if ( --printedNumberSize_index < -prec ) // checking if this should be the last digit, take note last digit is printed outside do while
            break;

        if(nBytesPrinted < size)
        {
        	*s++ = flags; // placing the digit
        	nBytesPrinted++; // increamenting number of bytes printed in buffer
        }

     } while (1);

     if ( printedNumberSize_index == exponent && (sign > '5' || (sign == '5' && !(vType & FTOA_CARRY))) ) // checking for a special case where the first digit from __ftoa_engine is right after when printing finish. for example consider 0.005 and lets say the user choose 2 for prec. that prints as "0.00" however if correctly rounded it should be 0.01. this if take care of similar conditions. FTOA_CARRY also make sure it has not been rounded up to become 5.
        flags = '1';

     if(nBytesPrinted < size) *s++ = flags;

	while(width) { // This condition is correct only if it has to be adjust from left and minimum width is not reached and it is making sure enough number of space are printed
		*s++ = ' '; // printing a space
		width--; // decrement the number of the spaces printed
		// incrementing++; // increamenting number of bytes printed in buffer but it does not matter any more
	}

	if(nBytesPrinted <= size) *s++ = 0; // placing the last null character. Due to special case we check for "nBytesPrinted < 0". the special case is when size is equal to zero

	return 0;
     
}

int8_t* myDtostrf (double val, int8_t width, uint8_t prec, int8_t *sout, const uint8_t size)
{
	bool dummy;
	return myDtostrf(val, width, prec, sout, size, dummy);
}


int8_t* myDtostrf (double val, int8_t width, uint8_t prec, int8_t *sout, const uint8_t size, bool& didItFit)
{
	uint8_t flags;

	/* DTOA_UPPER: for compatibility with avr-libc <= 1.4 with NaNs   */
	flags = width < 0 ? DTOA_LEFT | DTOA_UPPER : DTOA_UPPER;
	myDtoa_prf (val, sout, abs(width), prec, flags, size, didItFit);
	return sout;
}

 

 

myDtoa_prf part of this code is exactly same as dtoa_prf and buy reading the comments you can easily understand the original dtoa_prf too.

 

There is only one part which I did not understand and that is 

if ((signed char)nDigits < 1) 
	nDigits = 1;
else if (nDigits > 8) 
	nDigits = 8;

I tested with

if ((signed char)nDigits < 0) 
	nDigits = 1;
else if (nDigits > 8) 
	nDigits = 8;

and it produce exact same results after testing with random float number, but I think it is faster because it can be branch if minus. (I am not very familiar with AVR assembly instruction yet but from what I recall from 6800 instruction set, branch if minus was faster than comparing and then branching)
I would be happy if anyone can point out why there is a 1 there instead of 0. I am afraid it is for a very special case and the random number generator has not generated that.

 

Moreover you may share your opinion about how the testing is carried out (in the first code posted)

 

 

And lastly,
https://www.mikrocontroller.net/topic/301125
 shows folks who solved the same problem. However my version has one difference. Their version does not print the number at all if it does not fit. My version it prints as much as it can and then set the bool argument which passed by reference to false.

 

 

I would like to thank you all for the support, suggestions and your efforts.