Non-linear regression algorithm

Go To Last Post
12 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Freaks,

Does anyone know a site that details how to implement a non-linear (i.e curved graph) regression algorithm - also known as a "line-of-best-fit" algorithm? My devices requires the user to input 7 points of interest along a graph (mL versus degress celcius) so the device can then calculate a value based on the closest projected value possible.

For example, a curved graph is shown (in ASCII) below, with the seven points marked as "X" 's.
| _X______________.
mL's | ____X____X/ \X
/se | ..X...''''X'''' \X
----------------------------------------------------------------
Degrees Celcius

The device should be able ot pinpoint with the best possible accuracy any value on the graph, using only the seven given data points. Any ideas?

- Dean :twisted:

Make Atmel Studio better with my free extensions. Open source and feedback welcome!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

hello dean,
would be interested too!
google didn't find any usefull links.

greetings
gerhard

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi,

it depends on what mathematical function you want.

with "linear" do you mean: y = ax + b ??
--> Something like least square error? I will look for it.

or do you mean any function.
--> there are many solutions

or a specific function?
--> for example: if you know that it has to be logarithmic...

With 7 values you will (always) find a function like:
y = ax^6+bx^5+cx^4+dx^3+ex^2+fx+g
that fits your values.

Klaus

Klaus
********************************
Look at: www.megausb.de (German)
********************************

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi

I would choose Microsoft Excel.
Plenty of chart tools in it.

But then again someone else may have a better solution.

Cheers

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Sorry Freaks, I was in the library when I posted my last message, thus I was brief. I shall try to explain the situation better below.

I am 15 years old and my hobbies are electronics, microcontrollers and programming. I have written a documentation for a commercial microcontroller product, and my dad's friend who owns a small chemical mixing company has asked me to create a device, called the "(Company Name) Programmable Volumetric Liquid Dispenser". It should allow the user to program it with the liquids being pumped through six channels - giving it the name of each chemical/liquid and the ml's pumped at different temperatures. The device should then use a non-linear regression algorithm to predict how long the pumps need to be on to pump a certain amount of the liquid, as enterd by the user.

The device should be as accurate as possible, but also as cheap as possible to manafacture/produce. Temperature sensors are placed inside each of the six tanks. The user enters the amount of liquid, channel/pump and type of liquid into the device and presses "dispense". The device shoudl then calculate and pump the correct amount. Since most liquids have a non-linear viscosity response to temperature change, the device must use a non-liner regression algorithm. Because the calculations must be performed inside the device, Excell or other computer programs are no appropriate.

After searching my brothers year 12 textbooks, my parents old college mathematics textbooks, my dad's memory (it's a bit dusty in there!) and the web, I have determined that the best algorithm is the "Least Squares" function as it is accurate and reponds reasonable well to outliers. The web turned up a limited few pages with the formula - most were company's trying to sell algorithm DLL's and such - but it is very complex, using many symbols that I have not encountered before in my year 10 mathematics. Although my dad can explain it to me, can anyone provide a sample of C, BASIC or other easy to understand language that I can use?

The liquid's data is in the form of two arrays holding seven items each. The first element of the Y array corresponds to the first element on the X array.

Any help would be GREATLY appriciated.

- Dean :twisted:

Make Atmel Studio better with my free extensions. Open source and feedback welcome!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Oh oh oh oh oh! (Jumps up and down excitedly) I think i've found it!

http://www.bjmath.com/bjmath/lea...

Has a clear representation of the formula! Can anyone verify that this is correct?

- Dean :twisted:

Make Atmel Studio better with my free extensions. Open source and feedback welcome!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ah you are in luck Dean, I just had a test over the least sum of squared errors method. It is very simple to do by hand, but it creates alot of nasty decimal values. Anyway I will give you an example. Say you are given a set of equations that are "overdetermined," ie, there are more equations than unknowns. Using the least sum of squared errors, you can find a line of best fit for these equations. Take for example:

3x1 + x2 = 3
4x1 + x2 = 5
5x1 + x2 = 5
x1 + x2 = 1

those variables are x1 and x2, they could just as easily be x and y or a and b, it just fits better with the example. First form a matrix from the coefficients of x1 and x2 like so and name this matrix A, like so:

A =
3, 1
4, 1
5, 1
1, 1

If we were writing by hand this matrix would have brackets around it. Now form a matrix called A^T (thats A superscript T) that will be matrix A "transposed" ie, all colums become rows and all rows become columns, like so:

A^T=
3, 4, 5, 1
1, 1, 1, 1

Again A^T would have brackets around it if it were writen by hand. Now multiply A^T times A like so: (periods added for spacing, brackets added for "clarity")

[3, 4, 5, 1][3, 1]
[1, 1, 1, 1][4, 1]
.................[5, 1]
.................[1, 1]

Look up matrix multiplication and you will see that the order of these matrices is important, A^T must come first followed by A in the multiplication. Your result with be a matrix of size 2x2:

51, 13
13, 4

Now you must invert this matrix (I lieave researching of inverting up to you, its not difficult with 2x2 matrix):

4/35, -13/35
-13/35, 51/35

Okay, hold on to that little 2x2 matrix, we will need it in a minute. Now take out matrix A^T formed earlier, and multiply it by the matrix we will call y, which is formed from the right hand side of the original set of equations, like so:

y =
3
5
5
1

Now multiply A^T times y, like so:

[3, 4, 5, 1][3]
[1, 1, 1, 1][5]
................[5]
................[1]

The result of that is a matrix of 2 rows and one column, like so:

55
14

Now, we are almost done! Multiply the matrix found earlier that has all the messy fractions in it by the matrix just calculated, like so (periods added for a bit of spacing):

[4/35, -13/35]....[55]
[-13/35, 51/35]..[14]

The result being a final matrix of two rows and one column

38/35
-1/35

those values correspond to the "best fit" values for our original variables x1 and x2:

x1 = 38/35
x2 = -1/35

We call the above matrix simply x, and we can give an equation for the least sum squared error values of x like so:

x = ((A^T * A) ^-1)(A^T * y)

solving this equation is the exact process we just went through. keep in mind the ^-1 in there represents inverting the matrix as I talked about above. So basically as you can see, its pretty involved for a little microcontroller to do all those calculations. If you want an example that finds values for the equation y = mx + b I can give you that as well. I hope that helps, good luck!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi Dean,

I hope that i understood your problem. For me it seems it´s more usefull for you to use a lookup table. The variable is the viscosity in relation to the temperature. You have several fluids with independent temperature and viscosity.

Make a table with temperature and the dependent viscosity. Do linear interpolation between the points. If this is not exact enough (because of non linearity) calculate more points.

I do not think you have to calculate the hole function at the time you want to dispense the fluid. The function is given by your table. All you have to do is read your temperature and look into your table.

Liquids: Mind that many fluids have a change in viscosity depending on its speed. This could be positive and negative. (thixotrophy).

Hopes this makes it easier.
Klaus

Klaus
********************************
Look at: www.megausb.de (German)
********************************

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hello Dean,
I just edited my post. I was going to agree with Klaus but penny just dropped. The curve changes shape depending on the 7 points of interest (correct?) in which case a fixed look up table is no help, as is this post , sorry. Although , if the curve is generally logarithmic or exponential couldn’t a table for that function be written , and then applied in a similar way to the linear least squares method ?

Michael

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Dean I also agree with making a lookup table. I am working on an altimeter and the altitude calcuations are similarly complicated so I have made one gigantic lookup table for all possible altutitudes, using my trusty calculator to come up with the values beforehand. Works pretty well so far.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Dean,

Working from viscosity seems like a bit of a roundabout way to do this. Would it not be possible to do one of the following?

1. Use a pump that pumps volume directly, something like a persitaltic pump. (e.g. http://www.michael-smith-engineers.co.uk/ismatec%20docs/ecoline_series.htm) or maybe even a syringe driven by a screw thread for really small volumes. [How small, roughly, and how often are you thinking of?] Then, you need only count the revolutions of the pump to determine how much fluid has been dispensed.

Edit: just thought about this. You could connect the syringe up through a pair of one-way valves. Compressing the plunger dispenses the liquid. Returning it to its initial position sucks liquid from a much larger reservoir. As long as the syringe is larger than the largest dispensed volume, then you can do this in one stroke. Drive each syringe with a stepper motor, and off you go!

2. Weigh the output, pump one component at a time. Volume is then weight change/density. The densities of the liquids are unlikely to change much over normal temperature ranges, certainly less than their viscosity can. If the output of the scale is non-linear, a single lookup table will be enough to compensate. This is better for bigger values.

Sean.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks all for your help. In response:

The device is commercial and it's design was given to me by by employers (oh, it's great to say that when your 15!). I can't change it - I just get paid to do the best job I can.

Lookup tables would be innapropriate, as the device should be as accurate as possible. 7 data points was chosen as it gives a high accuracy when calculated on a ti-83 calculator (using the QuadReg function) and it requires less setting up by the customers. It's better to implement complex mathematics now than risk angry customers who don't want to enter 400,000 different values into the device. As shown on my trust ti-83, a quadratic regression algorithm provides the best accuracy with the lest amount of setting up.

Sellis: I partially agree. There are several scientific devices on the market that do use a syringe and valve system, but these are slow and cannot dispense at any great speed (biger syrynges give less accuracy). This is for commercial/industrial use, and so efficiency is prized above all else.

The device can control up to six channels, and store 12 different liquids. The user enters the data into the device, it should calculate the A, B and C values of the formula and store these in the EEPROM. Typical amounts of liquids are anything from 20mL's to several hundred mL's/several litres.

The devide does not need to be perfectly accurate, but using a linear regression algorithm would produce larger errors than a quadratic regression algorithm - the errors are magnified when multipled with the amount being pumped. In addition, my employers (that word again :lol: ) specifically asked for a non-linear algorithm.

I agree that the design is far from optimal, but it's not my decision and my employers ( :lol: ) money is as good as anyone elses. Lets just ignore this and get back to the problem at hand.

BASCOM cannot use matrixes, only arrays. Also, I only have about 1-1.5k of program space left, so can anyone help me out with some basic or c code that uses less than that?

- Dean :twisted:

Make Atmel Studio better with my free extensions. Open source and feedback welcome!