GPS based clock missed seconds

Go To Last Post
21 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Not sure the best forum to post this, but since it is an Arduino sketch I'll try it here.

 

Background:  I've always wanted to build one of those self setting clocks and when I came across a simple gps module thought I would give it a try.

The project consists of an arduino UNO, a serial gps module, and a serial 7seg serial display module.

The time works great, turn it on and after the gps acquires a fix the nmea position fix contains UTC time.

 

To parse the nmea string, I'm using the adafruit_gps lib to give me the hours minutes and seconds.

I've set the gps to output two fixes a second.

 

The problem:  the time is displayed on a 4 digit 7seg display, so hour and minutes.

I wanted to flash the colons between the hours and minutes, one second on / off.

What I'm seeing is the flashing is not consistent, on one sec, off two seconds, on two seconds, off one second .....

 

Looking at the raw nmea strings, I can see the data I want and it looks ok, so..

digging into the lib by adafruit I find this to parse out the HH MM SS and miliseconds

 

  uint8_t hour, minute, seconds, year, month, day;
  uint16_t milliseconds;
  
  // look for a few common sentences
  if (strstr(nmea, "$GPGGA")) {
    // found GGA
    char *p = nmea;
    // get time
    p = strchr(p, ',')+1;
    float timef = atof(p);
    uint32_t time = timef;
    hour = time / 10000;
    minute = (time % 10000) / 100;
    seconds = (time % 100);

    milliseconds = fmod(timef, 1.0) * 1000;
    

It looks to me like the string containing the time, looks like a floating point number so they use atof() to convert to a float.

Then pick out the integer portion and parse out the hours / minutes and seconds.

Then go back to the float to get the decimal portion containing the miliseconds.

 

The problem, I think is the float value does not have enough digits to hold a valid HHMMSS value and gets truncated in the seconds value.  At least not during hours where I'm awake enough to care! smiley

printing the seconds value to the serial console confirms it misses seconds, although they do appear in the raw data stream.

I'll report the error to adafruit, but thought I would toss this out to the freaks for the best way to parse this data correctly.

 

A typical data line looks like this:

$GPGGA,172814.500000000,3723.46587704,N,12202.26957864,W,2,6,1.2,18.893,M,-25.669,M,2.0,0031*4F

 

time above is 17 hrs, 28 min, 14.5 seconds

 

TIA

Jim

 

This topic has a solution.

Click Link: Get Free Stock: Retire early! PM for strategy

share.robinhood.com/jamesc3274
get $5 free gold/silver https://www.onegold.com/join/713...

 

 

 

 

Last Edited: Wed. Mar 28, 2018 - 12:34 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ki0bk wrote:
The problem:  the time is displayed on a 4 digit 7seg display, so hour and minutes.

I wanted to flash the colons between the hours and minutes, one second on / off.

What I'm seeing is the flashing is not consistent, on one sec, off two seconds, on two seconds, off one second .....

if you make it display Min:secs instead, how do the Sec behave ?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Couldn't you parse out the time yourself instead of using the Arduino library? Which model GPS are you using by the way? Also, Does the GPS module have a 1 second pulse output that you could use to blink the colon?

East Coast Jim

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0
uint8_t hour, minute, seconds, year, month, day;
uint16_t milliseconds;

// look for a few common sentences
if (strstr(nmea, "$GPGGA")) {
  // found GGA
  char *p = nmea;
  // get time
  p = strchr(p, ',')+1;
  float timef = atof(p);
  uint32_t time = timef;
  sscanf(p, "%u" &time)
  hour = time / 10000;
  minute = (time % 10000) / 100;
  seconds = (time % 100);

  milliseconds = fmod(timef, 1.0) * 1000;

Milliseconds will still suffer from truncation.

 

Assuming fractional seconds are always reported to nanosecond resolution:

uint8_t hour, minute, seconds, year, month, day;
uint16_t milliseconds;

// look for a few common sentences
if (strstr(nmea, "$GPGGA")) {
  // found GGA
  char *p = nmea;
  // get time
  p = strchr(p, ',')+1;
  float timef = atof(p);
  uint32_t time ns  = timef;
  sscanf(p, "%u.%u" &time, &ns)
  hour = time / 10000;
  minute = (time % 10000) / 100;
  seconds = (time % 100);

  milliseconds = ns / 1000000 fmod(timef, 1.0) * 1000;

EDIT:  typo ms should have been ns

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

Last Edited: Tue. Mar 27, 2018 - 08:39 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Who-me wrote:
if you make it display Min:secs instead, how do the Sec behave ?

The seconds will pause.   Although the raw nmea stream show them changing.

 

 

Click Link: Get Free Stock: Retire early! PM for strategy

share.robinhood.com/jamesc3274
get $5 free gold/silver https://www.onegold.com/join/713...

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

jgmdesign wrote:
Couldn't you parse out the time yourself

I think I'll have to, just looking for ways to do that.

 

joeymorin wrote:
Milliseconds will still suffer from truncation.

I think the problem is before that, in the atof(),  I think I need to scan the incoming string fragment.

 

Jim

 

Edit: Note the same code appears later in the lib to parse another nmea string, RMC

so the code will have to be fixed twice.

 

Click Link: Get Free Stock: Retire early! PM for strategy

share.robinhood.com/jamesc3274
get $5 free gold/silver https://www.onegold.com/join/713...

 

 

 

 

Last Edited: Tue. Mar 27, 2018 - 08:11 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ki0bk wrote:

Who-me wrote:
if you make it display Min:secs instead, how do the Sec behave ?

The seconds will pause.   Although the raw nmea stream show them changing.

Hmm .. Is all of the code executed in doing this simply too slow, or is there some beat effect with another timer going on ?

What SysCLK is this using ?

Or is some code doing floats inside an interrupt ?

Last Edited: Tue. Mar 27, 2018 - 08:14 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think the problem is before that, in the atof()

Yes.  I didn't care because you don't need milliseconds ;-)

 

I think I need to scan the incoming string fragment.

Which is what I did for HHMMSS, using sscanf.

 

However, I edited my post (a few times) while you were posting.  Have a look again.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

Last Edited: Tue. Mar 27, 2018 - 08:40 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Who-me wrote:
Is all of the code executed in doing this simply too slow

I don't think so, the RX is buffered via interrupt, but the parsing is done in loop, will a GCC float hold 7 digit numbers with out truncation?

 

Jim

 

Click Link: Get Free Stock: Retire early! PM for strategy

share.robinhood.com/jamesc3274
get $5 free gold/silver https://www.onegold.com/join/713...

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

For this I would probably dump a bunch of NMEA strings to a UART and catch them with a PC.

Then take JoeyMorins modifications and decode those strings on a PC untill you have a debugged routine to your liking.

 

Using atof() and then backward calculating minutes and seconds is redicilous.

You have accurate data in hours, minutes, seconds, milli seconds ..... in an ascii string.

Converting that to a number, and converting back to HH:mm:ss is redicilous.

How static is the format of the nmea string?

Are the dots and comma's always in the same place?

It is pretty trivial to write a function to take 2 digits out of an ascii string and convert them to a packed BCD or uint8_t.

You can also use sscanf to read an exact number of ascii characters from the string and put those directly in hour, minute, second, milli second ... variables.

 

Are you also relying on a constant data stream from your GPS device?

I would not be surprised that if reception is bad, it just resends the last known combination of timestamp and position it knows.

This sounds very likely.

GPS modules are mainly for getting location information, and the timestamp is not the "actual time" but the time at which the location was calculated.

 

ki0bk wrote:
Who-me wrote: Is all of the code executed in doing this simply too slow
As always: It depends.

If you want accurate nano seconds on your clock, I wish you good luck with an AVR.

But an AVR can probably decode a nmea sting  in several hundred micro seconds.

 

My advise is to take a different approach.

Write a separate time keeping function for your AVR and use that for the display.

The DCF clock I used long ago was based around a 1ms or 10ms time tick and used overflows to count seconds, minutes, days etc.

Then you can compare & synchronize with the data from your NMEA strings by only allowing the clock to run a minute amount faster or slower.

(So update seconds after 999 or 1001 1ms ticks).

This will always result in regular intervals for the display update.

 

Your NMEA data probably often has old timestamps. if you compare them with the internal AVR clock, and spit out the difference to a UART (LCD, whatever) you get some data on how old the NMEA string is.

If you want to get fancy and as accurate as you can get. The NMEA strings probably always have a timestamp somewhere in the history and never in the future.

If your NMEA strings are at 9600baud then the time to send the string is probably much longer than the time to decode it.

The time in the NMEA string is probaby older than the first start bit of that string.

 

Conclusions:

1). Your biggest error is probably because your GSM sends old NMEA strings.

2). You may have rounding errors in the float conversion and backwards calculation.

3). Building a Sub ms accurate clock is fun and you can get temperature & drift information, out of your AVR's F_CPU crystal oscillator.

Doing magic with a USD 7 Logic Analyser: https://www.avrfreaks.net/comment/2421756#comment-2421756

Bunch of old projects with AVR's: http://www.hoevendesign.com

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ki0bk wrote:

Who-me wrote:
Is all of the code executed in doing this simply too slow

I don't think so, the RX is buffered via interrupt, but the parsing is done in loop, will a GCC float hold 7 digit numbers with out truncation?

If the NMEA strings really are 100% ok, as you report, something is dropping seconds in the AVR processing.... 

 

The largest number seems to be 245959, which can resolve inside 18 binary bits, whilst floats are 23-24 bit mantissa

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I thought you might like the time keeping routine I wrote long ago.

I haven't looked at it for years, It is based on a 10ms tick, but easily adjustable.

It is also pretty compact code.

//====================================================== Function definitions.
// Correction for the lowest nibble of a packed BCD number after an addition.
// The high nibble needs no correction because it never overflows for the clock.
static void Bcd(uint8_t *Number)
{
	if(((*Number) & 0x0F) > 9)
		*Number += 6;
}

//===========================================================================
static void CheckAlarms( void) {
	uint8_t Cnt;

	for( Cnt = INDEX_ALARM_01; Cnt <= INDEX_ALARM_LAST; Cnt++) {
		if(  (Time[INDEX_TIME].Minutes	== Time[Cnt].Minutes)
		  && (Time[INDEX_TIME].Hours	== Time[Cnt].Hours)
		  && (Time[INDEX_TIME].WeekDay	& Time[Cnt].WeekDay) ) {
			SendAlarm |= 1ul << (Cnt - 1);
		}
	}
}

//===========================================================================
// Increment one of the time variables: Hundredths, Seconds, Minutes, Hours.
static uint8_t TimeCarry( uint8_t *Number, uint8_t Top) {
	(*Number)++;
	Bcd(Number);
	if((*Number) >= Top)
	{
		*Number = 0;
		return true;		// Overflow.
	}
	return false;			// No overflow.
}

//===========================================================================
// Some housekeeping to do each second.
static void SecondsTick(void) {
	secondTick = true;
	if( ResyncTimer > 0)
		ResyncTimer--;

	if( DarkTimer > 0)
		DarkTimer--;

	if( lastSyncSeconds >= 0)
		lastSyncSeconds++;
}

//===========================================================================
// This thread define's the speed of the clock and must be called every 10ms.
void ThreadTimeTick(void) {
    static int8_t calibration;        // Small corrections in bcd format.
//---------------------------------------------------- Update the actual time.
    if( TimeCarry( &Time[INDEX_TIME].Hundredths , 0xA0 - calibration)) {
//-------------------------------------------- If a whole Second has elapsed.
        calibration = 0;    // Reset calibration correction after it's used once.
        SecondsTick( );        // Do the seconds handler.
        if( TimeCarry( &Time[0].Seconds, 0x60) ) {
//-------------------------------------------- If a whole Minute has elapsed.
            calibration = CALIBRATE_167_PPM;
            if( TimeCarry( &Time[0].Minutes, 0x60) ) {
//---------------------------------------------- If a whole Hour has elapsed.
                calibration = CALIBRATE_3_PPM;
                if( TimeCarry( &Time[0].Hours, 0x24) ) {
//---------------------------------------------- If a whole Day has elapsed.
                    Time[0].WeekDay <<= 1;
                    if((Time[0].WeekDay & 0x80) || (Time[0].WeekDay == 0))
                        Time[0].WeekDay = 1;
                }
            }
            // Check the alarms every minute AFTER the time is updated.
            CheckAlarms( );
        }
    }
}

The "calibration" variable is for very small corrections every minute and hour.

It is supposed to keep the time accurate to almost a single 10ms tick.

These were once hand calibrated, but temperature variations and drift of a cheap microcontroller crystal are too big to make this really usefull.

Doing magic with a USD 7 Logic Analyser: https://www.avrfreaks.net/comment/2421756#comment-2421756

Bunch of old projects with AVR's: http://www.hoevendesign.com

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ki0bk wrote:
I've set the gps to output two fixes a second. The problem: the time is displayed on a 4 digit 7seg display, so hour and minutes. I wanted to flash the colons between the hours and minutes, one second on / off. What I'm seeing is the flashing is not consistent, on one sec, off two seconds, on two seconds, off one second .....
This very likely has nothing to do with your NMEA decoding. It is a synchronisation problem.

How are you updating the colon flashes? You may miss the nmea string by a milli second or so and then show the wrong colon value for a whole second.

Doing magic with a USD 7 Logic Analyser: https://www.avrfreaks.net/comment/2421756#comment-2421756

Bunch of old projects with AVR's: http://www.hoevendesign.com

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

jgmdesign wrote:
Which model GPS are you using by the way?

Also, Does the GPS module have a 1 second pulse output that you could use to blink the colon?

 

Fist question is my curiosity, second question was meant as written.

 

ECJ

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

"I was told there would be no math!"

 

#include <inttypes.h>
.
.
.
  uint8_t hour, minute, seconds;
  uint16_t milliseconds;
.
.
.
  if (strstr(nmea, "$GPGGA")) {
    char *p = nmea;
    p = strchr(p, ',')+1;
    sscanf(p, "%2" SCNu8 "%2" SCNu8 "%2" SCNu8 ".%3" SCNu16, &hour, &minute, &seconds, &milliseconds);

Presumes that the time in the NMEA string is always in the format HHMMSS.ssssss

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

Last Edited: Wed. Mar 28, 2018 - 03:00 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

... plus a trailing "}"

 

Ross McKenzie ValuSoft Melbourne Australia

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ross, I was only keeping to the skeletal form of the excerpted adafruit code in #1.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

This reply has been marked as the solution. 
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

And anyway, no need for the format string to take up precious SRAM:

#include <inttypes.h>
#include <avr/pgmspace.h>
.
.
.
  uint8_t hour, minute, seconds;
  uint16_t milliseconds;
.
.
.
  if (strstr(nmea, "$GPGGA")) {
    char *p = nmea;
    p = strchr(p, ',')+1;
    sscanf_P(p, PSTR("%2" SCNu8 "%2" SCNu8 "%2" SCNu8 ".%3" SCNu16), &hour, &minute, &seconds, &milliseconds);

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ki0bk wrote:

 

  uint8_t hour, minute, seconds, year, month, day;
  uint16_t milliseconds;
  
  // look for a few common sentences
  if (strstr(nmea, "$GPGGA")) {
    // found GGA
    char *p = nmea;
    // get time
    p = strchr(p, ',')+1;
    float timef = atof(p);
    uint32_t time = timef;
    hour = time / 10000;
    minute = (time % 10000) / 100;
    seconds = (time % 100);

    milliseconds = fmod(timef, 1.0) * 1000;
    

$GPGGA,172814.500000000,3723.46587704,N,12202.26957864,W,2,6,1.2,18.893,M,-25.669,M,2.0,0031*4F

 

time above is 17 hrs, 28 min, 14.5 seconds

 

TIA

Jim

 

 

 Hmm... Looks like both conversion and sync issues. You need neither strchr() nor atof() to parse a GGA:

 

char * nmea;
int hr;

hr = (* ( nmea + 7 )) - '0';
hr *= 10;
hr += ((* ( nmea + 8 )) - '0' );

 `Fix taken at' field of GGA is not a number, it's a string in hhmmss[.(x)] format. Take the 7th char of the string, convert it to int (substracting '0' works just fine for ASCII), that will give you tens of hours. Same conversion for hours, 8th char of GGA. Then proceed with minutes and seconds.

 But you that only once., after the first fix. then, as East Coast Jim suggested, you need to use 1 Hz pulse (1PPS) generated by GPS receiver to run your clock. Receiver does not guarantee that it will form and send NMEA sentences strictly pertiodically. And its send routine can be interrupted by receiver's internal events. Thus, you need to trck time offset between 1PPS and change of minutes in GGA that you receive.

 

PS

 Check your receiver's docs. It may make your life easier if it supports $GPZDA

 


Qoitech AB, The Home of Otii Arc, an SMU for every developer

Check out Otii solution at www.qoitech.com

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

joeymorin wrote:

Ross, I was only keeping to the skeletal form of the excerpted adafruit code in #1.

 

Joey, I was just trying to avoid the, almost inevitable, "the code does not work" response.

 

Ross McKenzie ValuSoft Melbourne Australia

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Fantastic guys! Thanks for all of the ideas, I'll mark joey's as the solution for now, let you know how it works soon.

 

You are the all the best!

Jim

Click Link: Get Free Stock: Retire early! PM for strategy

share.robinhood.com/jamesc3274
get $5 free gold/silver https://www.onegold.com/join/713...