looking for moving average code

Go To Last Post
35 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi freaks,

  I have searched for some code posted here for a moving average that used a single float and an int new sample returning an updated float value,

instead of storing an array of integer samples, I think was posted by david.prentice but have not been able to find it. 

If you have a copy or can find and post the link, Thanks

 

Jim

 

This topic has a solution.

Mission: Improving the readiness of hams world wide : flinthillsradioinc.com

Interests: Ham Radio, Solar power, futures & currency trading - whats yours?

 

Last Edited: Fri. Jan 1, 2016 - 08:46 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0
Last Edited: Wed. Dec 30, 2015 - 06:46 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks I did find a clue there, and came up with this:

 

#define N = 16;

float ema( float average, uint16_t sample)
{
    average = (average*(N-1) + sample)/N; // running average

    return average;
}

Does that look like it would work?

 

My project is using a T25, so ram space is tight for a large array of samples, but I have space for the FP routines in flash.

Actually, I need to do several averages, so will modify the above to pass a pointer to float for each one. 

Thanks

Jim

 

Mission: Improving the readiness of hams world wide : flinthillsradioinc.com

Interests: Ham Radio, Solar power, futures & currency trading - whats yours?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That's not a moving window average.  Assuming you retain the returned value and feed it back into the next call as the first argument, it's a IIR filter.  Similar, but not identical, effect.  Specifically, it's a type of low-pass filter.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"Read a lot.  Write a lot."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Your right, for this app, a low pass filter is what I'm wanting, so I should change the name of the function to reflect that.

What changes would I need to make it a moving window average?

 

Jim

 

Mission: Improving the readiness of hams world wide : flinthillsradioinc.com

Interests: Ham Radio, Solar power, futures & currency trading - whats yours?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ki0bk wrote:
Your right, for this app, a low pass filter is what I'm wanting, so I should change the name of the function to reflect that.

What changes would I need to make it a moving window average?

You already know ;-) :

storing an array of integer samples

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"Read a lot.  Write a lot."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If you want a moving average, just maintain N samples in a ring buffer. Together with the current total.
When you add a new sample, you adjust the total by subtracting the previous entry and adding the new entry to the total.
The average is always available as total/N.
You choose the algorithm that is most suitable for your requirements. Looking at the historic thread, I gave an example for a cycle computer. i.e. you want to see the speed after one revolution. But an average of N samples will give you a steadier display.
David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Do you really need to keep the ring buffer and its contents? Surely, the previous sample value, the running total (providing it comprises the N samples) and the newest sample are all that are required.

Ross McKenzie ValuSoft Melbourne Australia

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You can certainly do a "windowed" average without a ring buffer, but its not a "moving window" average.  The idea is of the later is that you can get an average at each and every sample.  If you are not storing N values in a buffer, you must start a new window each N values, and the average is only valid every N values, instead of at every sample.

 

The problem is that the subtraction is of the value at the head of the ring buffer, where the insertion is at the tail.  That is, you subtract the Nth previous sample and add the most recent.

 

Martin Jay McKee

As with most things in engineering, the answer is an unabashed, "It depends."

Last Edited: Thu. Dec 31, 2015 - 06:01 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Absolutely correct Martin... I should not type while the head is suffering in +39C temperatures. What on earth was I thinking!

 

Ross McKenzie ValuSoft Melbourne Australia

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

+39C temperatures

Ouch.

 

Anyway, something like:

#define MOVING_WINDOW_SIZE 16

int moving_window_average(int dat) {

  static int window[MOVING_WINDOW_SIZE];
  static int window_idx;
  static long int window_total;

  window_total -= window[window_idx];
  window_total += dat;
  window[window_idx] = dat;
  window_idx = (window_idx < (MOVING_WINDOW_SIZE - 1)) ? (window_idx + 1) : 0;

  return (window_total / MOVING_WINDOW_SIZE);

}

EDIT: typo in code

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"Read a lot.  Write a lot."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

Last Edited: Fri. Jan 1, 2016 - 07:25 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

joeymorin wrote:

+39C temperatures

Ouch.

 

Yep... and our air conditioner is broken.

 

joeymorin wrote:

Anyway, something like:

#define MOVING_WINDOW_SIZE 16

int moving_window_average(int dat) {

  static int window[MOVING_WINDOW_SIZE];
  static int window_idx;
  static long int window_total;

  window_total -= window[window_idx];
  window_total += dat;
  window[window_idx] += dat;
  window_idx = (window_idx < (MOVING_WINDOW_SIZE - 1)) ? (window_idx + 1) : 0;

  return (window_total / MOVING_WINDOW_SIZE);

}

 

Yeah probably correct, but my fried brain is going to seek an iceblock out of the fridge before attempting validation.

 

Happy New Year!

 

Ross McKenzie ValuSoft Melbourne Australia

Last Edited: Thu. Dec 31, 2015 - 06:53 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

joeymorin wrote:
  Similar, but not identical, effect.  

To see the different effects in your application, gather some sample data, load it into your favourite spreadsheet, apply the two "processes" and graph the results.

 

You could also generate some "simulated" input data showing the kind(s) of artefact(s) you're trying to deal with - and how the two "processes" respond ...

 

Of course, if you have something like Matlab, it's ideal for this kind of thing...

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks guys,

   I have a simple moving average function that uses a ring buffer, the problem with it is the limited ram available on the t25.

What I'm looking for is an ema function, the one I found before used a single float value.   The IIR filter I posted above, after some testing is close, but does not work for what I need.  The function I'm looking for is similar to that....  I keep digging.

Happy New Year to all!

Jim

 

Mission: Improving the readiness of hams world wide : flinthillsradioinc.com

Interests: Ham Radio, Solar power, futures & currency trading - whats yours?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Go on. A ring buffer with 4 elements is hardly going to be expensive. 8 will be adequate for most things in life.

If your app really needs more SRAM, go and buy a tiny85.

David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

david.prentice wrote:
A ring buffer with 4 elements is hardly going to be expensive. 8 will be adequate for most things in life. If your app really needs more SRAM, go and buy a tiny85.

Indeed.  With an AVR8, one speculates 10-bit ADC values, or maybe some timer capture values which are again fairly small numbers.

 

With a little bit of thought, and a signal that doesn't make drastic changes (one wouldn't apply something like this to a sampled sine wave), I've used 8-bit signed differential samples in the past, and it can work well in many instances.

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

This reply has been marked as the solution. 
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think this is what I'm looking for:

 

#define N 16
float ema(float ave, unsigned int sample)
{
	float alpha = 2.0/(N+1);
	ave = alpha * sample + (1.0-alpha) * ave;
	return ave;
}

to get an exponential moving average without needing a ring buffer. 

 

If you have something similar, let me know.

TIA

Jim

 

Mission: Improving the readiness of hams world wide : flinthillsradioinc.com

Interests: Ham Radio, Solar power, futures & currency trading - whats yours?

 

Last Edited: Fri. Jan 1, 2016 - 06:44 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ki0bk wrote:
If you have something similar, let me know.

Not "similar", but if you make N = 15 then with a bit of scaling it becomes an integer situation.  Alpha is then 1/8.  1-alpha is 7/8.  And now aren't you right back to where you started from?

 

I don't know how you call it "exponential", without any exponents in there.

 

Start here

https://www.avrfreaks.net/comment...

https://www.avrfreaks.net/comment...

 

 

 

 

 

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

Last Edited: Fri. Jan 1, 2016 - 07:03 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think your right, I was just starting to verify the code posted, when I read your reply!

I'll keep looking, I know its here somewhere.

 

Jim

 

Mission: Improving the readiness of hams world wide : flinthillsradioinc.com

Interests: Ham Radio, Solar power, futures & currency trading - whats yours?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

After some testing, the code above seems to work fine.  So I have marked as the solution. 

Thanks for the feedback!

Jim

 

Mission: Improving the readiness of hams world wide : flinthillsradioinc.com

Interests: Ham Radio, Solar power, futures & currency trading - whats yours?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I don't know how you call it "exponential", without any exponents in there.

 

The weighted filter will give a simple exponential RC time (lowpass) response   ex:  Ynew= 0.1*Ysample +  0.9*Yold

With a 0 to 1 step input, this will rise to 0.707 after approx 12 samples

using constant 0.2 (instead of 0.1) will take approx 6 samples

 

So you could also use this to generate an exponential shape.

 

When in the dark remember-the future looks brighter than ever.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Isn't that inverse exponential i.e. logarithmic?

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"Read a lot.  Write a lot."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Isn't that inverse exponential i.e. logarithmic?

 

I'm sorry that it is not.  It follows e to a negative power (but that is not an "inverse exponential").

 

However, it is true that the inverse of an exponential is a log function (& vice-versa).

=================================================================

 

Unlike some other smoothing methods, such as the simple moving average, this technique does not require any minimum number of observations to be made before it begins to produce results. In practice, however, a “good average” will not be achieved until several samples have been averaged together; for example, a constant signal will take approximately 3/α stages to reach 95% of the actual value.[citation needed] To accurately reconstruct the original signal without information loss all stages of the exponential moving average must also be available, because older samples decay in weight exponentially. This is in contrast to a simple moving average, in which some samples can be skipped without as much loss of information due to the constant weighting of samples within the average. If a known number of samples will be missed, one can adjust a weighted average for this as well, by giving equal weight to the new sample and all those to be skipped.

This simple form of exponential smoothing is also known as an exponentially weighted moving average (EWMA). Technically it can also be classified as an Autoregressive integrated moving average (ARIMA) (0,1,1) model with no constant term.[6]

Time Constant[edit]

The time constant of an exponential moving average is the amount of time for the smoothed response of a unit set function to reach 1-1/e \approx 63.2\,\% of the original signal. The relationship between this time constant,  \tau , and the smoothing factor,  \alpha , is given by the formula:

\alpha = 1 - e^{-\Delta T \over \tau}

Where \Delta T is the sampling time interval of the discrete time implementation. If the sampling time is fast compared to the time constant then

\alpha \approx {\Delta T \over \tau}

When in the dark remember-the future looks brighter than ever.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

One thing I have done in the past is a "delta" (don't know a name but work like a delta modulator), variable that just follow the ADC value, one up if ADC is higher and one down if not, (perhaps make a zero band), but if this is a 16 bit variable and only the top 8 bit get's showed it works fine, with a min. of calculations.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It follows e to a negative power (but that is not an "inverse exponential").

Quite right.  My mistake.  That Wikipedia page is a good link to quote from.  Here's the actual link
https://en.wikipedia.org/wiki/Exponential_smoothing

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"Read a lot.  Write a lot."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

Last Edited: Sat. Jan 2, 2016 - 01:42 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

David Prentice wrote  :

"A ring buffer with 4 elements is hardly going to be expensive. 8 will be adequate for most things in life."

 

Well, what seems puzzling is the use of 2's exponents (4, 8, - 16 in JoeMorin's code-). They are likely to be optimized out (in the late 80's, Fortran optimizers did it: I suppose optimizers have improved) into shifts.... making float arithmetic (which eats flash) useless .....

 

BTW, I see some drawbacks in IIR filters:

Suppose you have an outlier (ex : 1, 2, 3, 1, 988, 5, 1, 3, 4 ...) : a finite window filter will forget it soon (4, 8 or 16 samples later) . An infinite response will take "a lot of " time to forget it.

Sometimes, an analog RC filter is added before the ADC. A 1rst order software IIR lowpass filter just simulates such a RC filter , making it redundant (if an analog input filter exists) .

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 

One thing I have done in the past is a "delta" (don't know a name but work like a delta modulator), variable that just follow the ADC value, one up if ADC is higher and one down if not, (perhaps make a zero band), but if this is a 16 bit variable and only the top 8 bit get's showed it works fine, with a min. of calculations

 

very interesting and can be rather useful (slew rate limiting filter)

 

https://books.google.com/books?id=Wtp5lg6UVygC&pg=PA81&lpg=PA81&dq=%22maximum+slew+rate%22+filter&source=bl&ots=4AcHMP6C3o&sig=3ffToIDixL8ynSqhFM4SLenyrTk&hl=en&sa=X&ved=0ahUKEwiZ1av_9YzKAhXFez4KHf27CNgQ6AEINjAF#v=onepage&q=%22maximum%20slew%20rate%22%20filter&f=false

 

http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0ahUKEwidveTP-4zKAhUCZD4KHbNqBZUQFggcMAA&url=http%3A%2F%2Fntrs.nasa.gov%2Farchive%2Fnasa%2Fcasi.ntrs.nasa.gov%2F19780012199.pdf&usg=AFQjCNFohVsD2ooHVsaWTE-62BtlKjNzWg

 

 

When in the dark remember-the future looks brighter than ever.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I actually had to use it because there was to many errors in the signal, and I did not have time to use a median filter.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

(ex : 1, 2, 3, 1, 988, 5, 1, 3, 4 ...)

If you have that kind of errors one should use a median filter. (so 988 never get used.)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Where would you like us to move your average?

 

277,232,917 -1 The largest known Mersenne Prime

Measure twice, cry, go back to the hardware store

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

"If you have that kind of errors one should use a median filter. (so 988 never get used.)"

 

I fully agree... but, with a limited RAM tiny CPU, a median filter is somewhat greedy (one needs to copy the cyclic buffer -not the original, to keep temporal consistency- , and sort the copy: I never taught of a more clever algorithm). With a finite response, the outlier  is somewhat quickly forgotten and it is -at least theoretically- impossible with an infinite response. 

 

Edited : I did read D.Prentice's post and realized  it is possible (through tedious) to compute the median of 3 elements http://stackoverflow.com/questio... without copies (5 elements would be more tedious, but one can notice the indexes of the min and the max.. and compute the median from the remaining). I am accustomed to great arrays which are sorted before computing the median https://stat.ethz.ch/R-manual/R-....

Last Edited: Mon. Jan 4, 2016 - 09:58 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Think about it. A median filter that is only 5 wide would have resulted in 3. Even if you had only 3 wide, the result would have been 3 or 5.
So I do not accept the argument about a Tiny being too small. Surely you could afford an 8 wide buffer.
Of course, everything depends on your particular noise environment. You use the type of filter and sampling that is suitable for your situation.
And yes, you can cope with individual glutches or burst glitches.
David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I was in a situation like getting 988 , and since I didn't had time for a 3 point median (or 5, perhaps 2 wrong in a row), I made my delta filter, then at least the 988 only count 1 up, 

but what is best depends of the situation.

 

Add:

And I will add what look nice in C don't always make a nice ASM code. (And the GCC sometimes make some very bad code, but in general it's ok)

Last Edited: Mon. Jan 4, 2016 - 11:12 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Surely you could afford an 8 wide buffer.

Who say that!

That isn't always the case. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks for the discussion, and I would normally agree that 8 to 16 samples would normally be enough for an average. 

In this case, I was playing with building a power meter for my solar panel, which feeds a PWM type charge controller to charge a SLA battery.

The PWM controller pulses the charge, so the panel voltage swings from max voltage (around 20 volts) to battery voltage ( around 13.5v), current

works in a similar manor, going from 0 to max running opposite to the voltage, these two swings I needed to smooth in order to get average power and a small number of samples

just was not cutting it.  My other goal was to rebuild my library of useful routines that I have accumulated from this site over the years, that I lost when my HD crashed.

The ema() function, has worked well for me in the past when I needed relatively large # of samples, or have many items I'm tracking, while the sma() function with a ring buffer is my go to routine when I only need a few samples. 

 

There may be a better way to do what I'm attempting and am open to suggestions.

Thanks again, for the help!

 

Jim

 

 

 

 

Mission: Improving the readiness of hams world wide : flinthillsradioinc.com

Interests: Ham Radio, Solar power, futures & currency trading - whats yours?