Capturing, storing, modifying and retransmission of a data stream.

Go To Last Post
68 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi All,

 

Another bike project.

 

I want to capture a data stream as below (I/P on a digital pin). I want to store it in RAM, modify 1 byte and then send it out again (on another digital pin). I need to do this 10 time a second.

 

Data stream (as I can fathom so far) has the following details.

 

10 messages/second @ 100ms apart. Start to start.
Each message is 8 bytes @ 5ms apart. Start to start.

Last byte is Checksum

It will have ~60ms to modify and retransmit.

7800 baud. This will probably require SoftwareSerial.

This data is from a motorcycle ECU to the dash cluster.

There is one byte I want to change (I don't know which one yet). 

 

Can anyone suggest the best/easiest way to achieve this?

Last Edited: Tue. Aug 18, 2020 - 01:19 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You can do a bit bang serial rcv,  put the data in a buffer, modify the xx byte of the queue & bitbang it back out.  The advantage is the timebase (xtal, rc, etc) isn't as critical, doesn't need cal'd, since the same timing is used for rx & tx. 

 

You can use the 100ms gap (say use 90ms) to sync the queue to a new batch of incoming (or trash any partials if a gap of 100ms goes by without a full load)

 

Or better yet:

 

byte in---> byte out, no queue used

The 100ms gap lets you sync & count off & intercept the desired byte.  

So the process becomes close to zero time lag. 

 

Maybe you can take it down to bit in--> bit out

With the sync, jump in & intercept/mod the desired bits of the chosen byte.

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

Last Edited: Tue. Aug 18, 2020 - 01:30 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks for the reply. 

 

At the moment I think I really only need to look at 3 bits. I think I will keep it simple and do the whole byte but will see how I go. 

 

I was going to try and keep it to RC timing. I don't know how critical it will be at this stage.

 

Looks like a bit more reading on bitbang and byte in byte out.....what ever they are.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

All good ideas above.

 

It is likely worth noting, however, that 7800 baud is a nice number for a 4 MHz / 8 MHz / or 16 MHz clock for the micro.

(Baud rate error is 0.16% which is very good)

Are you using an external resonator for the clock source, (perhaps an 8 MHz one?).

 

If you have a stable clock signal, across the operating temperature range, and you pick 4, 8, or 16 MHz, then you could make it easy on yourself and just select a micro with two hardware USARTs.

In that manner you will not need to write a SW USART, (either 1/2 of it, anyways).

With 60 mSec for the re-transmission you won't even need to use interrupts for this program, if that is all the micro is doing, (although you could if you wanted to).

 

Since a micro like the Mega168 data sheet says that the USART supports "Full Duplex Operation (Independent Serial Receive and Transmit Registers), you could likely do this project with just ONE USART, as both of your data streams are uni-directional, (one input, and one output).

 

That makes chip selection even less of an issue, and just the clock source being the key design parameter for the project.

(That and a clean power supply, and filtering on the input and output signals...)

 

JC

 

Edit: Typo

Last Edited: Tue. Aug 18, 2020 - 06:08 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0


This is the data stream I have managed to get so far.

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks for the suggestions. 

 

I was planing on using an ATTiny3217 as I am using them for another project it will be a commonality thing.

 

A big problem at the moment is a lot of this is way beyond my abilities. I am slowly learning and everything I have done in coding so far has been do it now type stuff. Loading into a buffer/register then modifying it and pumping it out again is all new.

 

I am sure that by the end of the month I am going to need a new brain.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Bugger. 3217 only has one usart. Mega168/328 might be the go.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Mega168/328 might be the go.

They also have 1 USART, will need to go with a new small new AVR 0/1 or Xmega (e5 series), otherwise a 40 pin Mega 164 (etc) chip.

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I didn't suggest Uart, since wasn't sure if his signal was even uart-compatible (such as having start and stop bits), though it would be tough if there were no startbit.

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

Last Edited: Tue. Aug 18, 2020 - 03:56 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I posted that before I checked.

 

I might have to ask DocJC for a little assistance.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

168/328 only have one UART.

 

Jim

 

Until Black Lives Matter, we do not have "All Lives Matter"!

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

A more "normal" chip with 2 USART would be a mega328pb.

 

The biggest problem I see is :

Last byte is Checksum

Do you know how it's calculated ?

If not that could be a show stopper, the rest is easy (meaning just hard work) 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

>Bugger. 3217 only has one usart.

 

Why do you need 2 usarts? You are receiving one stream of data, and then transmitting the modified stream. The rx and tx don't care about each other, although they share the same baud rate which will be the same if your are intercepting/modifying/sending.

 

Maybe you need another for something else?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sparrow2 wrote:

A more "normal" chip with 2 USART would be a mega328pb.

 

The biggest problem I see is :

Last byte is Checksum

Do you know how it's calculated ?

If not that could be a show stopper, the rest is easy (meaning just hard work) 

 

Fortunately I do know how it is calculated so that should not be a problem........I hope.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

curtvm wrote:

>Bugger. 3217 only has one usart.

 

Why do you need 2 usarts? You are receiving one stream of data, and then transmitting the modified stream. The rx and tx don't care about each other, although they share the same baud rate which will be the same if your are intercepting/modifying/sending.

 

Maybe you need another for something else?

 

Since posting that and doing a little research what you have stated is what I found. Thank you. 

 

As far as I know I don't need another one.

 

It should make it a little easier.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Now after all that could someone please get me started on some code. 

 

As I don't have direct access to the data stream at the moment and only have about a minutes worth of capture using Saleae Logic would having some code on a seperate Nano looping the data as a signal generator be the easiest?

 

I don't want people to write everything for me otherwise I will never learn but some starting code I can build on would be greatly appreciated.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

How would you do it on Arduino? Then translate that to whichever cpu you use.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Start simple, then work your way up. For testing, you can have some other mcu generate the packets, and maybe at some normal rate like 9600 baud so you could view the output on a pc. You don't even need to modify the packet at first, just see if you can echo it. Once you can echo something, then its just a matter of tweaking little bits at a time until you finally get what you want. You may eventually end up using interrupts, etc., but if there is nothing else going on it may not even be necessary.

 

This is not a flawless/reliable outline, but it will probably get data moving in/out

 

uint8_t buf[8];
uint8_t bufIdx;

 

int16_t read_rx():
    if( rx data not available ) return -1
    return rx

 

write_tx(uint8_t):
    while( tx not ready );
    tx = dara

 

read_packet():
    bufIdx = 0;
    while( bufIdx < 8 ){ 
        int16_t c = read_rx 
        if( read rx == -1 ) continue
        buf[bufIdx++] = c;
    }

 

write_packet:
    bufIdx = 0;
    while( bufIdx < 8 ) write_tx  buf[bufIdx++]   

 

setup tx/rx pins - rx input/pullup on, tx output
setup usart- 7800 baud, 8N1 (default), rx enable, tx enable

loop:
    wait for 30ms of no activity (10-50) to find the start of 'packet'
    for( uint8_t i = 0; i < 30; i++, _delay_ms(1) ){
        if( read_rx >= 0 } i = 0 //start over if rx data
    }
    we now have 30ms of silence

    read_packet
    modify
    write_packet

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Perhaps I'm too old but I would use a terminal program on a PC, and use canned good (and bad) packets to test the code. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sparrow2 wrote:

Perhaps I'm too old but I would use a terminal program on a PC, and use canned good (and bad) packets to test the code. 

 

I would too but can you set a terminal to odd baud rates?

 

PS. Different profile as I am having major trouble login in on my real one.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I would too but can you set a terminal to odd baud rates?

Yes some do, but you may have to also call Bill:

 

Realterm will pass any requested baud rate through to Windows. Many other applications have a list of baud rates, but Realterm does not, it will request anything.

Mostly, if a baud rate is not accepted, there is no error or warning - it just does not work. Whether a baud rate will be work depends on two things Reaterm has no control over:

  • Will the driver and/or Windows accept a requested but possible rate?

  • Can the hardware divide its clock down to the requested rate?

Microsoft says this : "For all other cases, as long as the requested baud rate is within 1 percent of the nearest baud rate that can be found with an integer divisor, the baud rate request will succeed"

The most basic PC uart has a maximum baud rate of 115,200. Any frequency of 115,200 / N, can usually be requested. More modern PC's and laptops, usually have a higher maximum baud rate of 230,400, or 460,800, or 921,600. Actual serial ports usually have a maxium that is a multiple of 115,200. Once you have found the maximum, any baud rate of <Max Baud Rate>/N should work.

 

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I normally use FTDI usb to RS232 chips and as I remember the can do everything that can be divided 3MHz (the 16x clk run from the 48MHz USB clk) 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

XR21B1420 - MaxLinear USB UART has a FBRG with a low latency mode for low baud by MaxLinear's driver instead of Microsoft's USB CDC ACM driver.

XR21B1420 Data Sheet (page 8)

XR21x14x Universal Async Receiver Transmitters - MaxLinear | Mouser

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sparrow2 wrote:
A more "normal" chip with 2 USART would be a mega328pb.
and 3 USART in mega4808; both of these are in Microchip's automotive catalog.

Automotive 8-bit MCUs | Microchip Technology

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Windows??? What's Windows???

 

I only use Windows at work because I have to. Linux for me, soooooo much quicker and much more stable.

 

Been away on holidays for a while, will get the brain back into this on the weekend. Doing some CNC stuff until then.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Attiny1634 has two UARTs... (though now Id probably go for one of the Mega0 or AVR-Dx chips.)
ATtiny1634 is supported by Dr Azzy's ATTinyCore for Arduino.
(only relevant if the protocol is actually UART-compatible.)

 

I created this design for this sort of application.  https://hackaday.io/project/1976... [
Alas, not actually been tested yet; it just seemed like a useful thing to have aroun

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

curtvm wrote:

Start simple, then work your way up. For testing, you can have some other mcu generate the packets, and maybe at some normal rate like 9600 baud so you could view the output on a pc. You don't even need to modify the packet at first, just see if you can echo it. Once you can echo something, then its just a matter of tweaking little bits at a time until you finally get what you want. You may eventually end up using interrupts, etc., but if there is nothing else going on it may not even be necessary.

 

This is not a flawless/reliable outline, but it will probably get data moving in/out

 

uint8_t buf[8];
uint8_t bufIdx;

 

int16_t read_rx():
    if( rx data not available ) return -1
    return rx

 

write_tx(uint8_t):
    while( tx not ready );
    tx = dara

 

read_packet():
    bufIdx = 0;
    while( bufIdx < 8 ){ 
        int16_t c = read_rx 
        if( read rx == -1 ) continue
        buf[bufIdx++] = c;
    }

 

write_packet:
    bufIdx = 0;
    while( bufIdx < 8 ) write_tx  buf[bufIdx++]   

 

setup tx/rx pins - rx input/pullup on, tx output
setup usart- 7800 baud, 8N1 (default), rx enable, tx enable

loop:
    wait for 30ms of no activity (10-50) to find the start of 'packet'
    for( uint8_t i = 0; i < 30; i++, _delay_ms(1) ){
        if( read_rx >= 0 } i = 0 //start over if rx data
    }
    we now have 30ms of silence

    read_packet
    modify
    write_packet

 

Hey Curtvm,

 

Were you on the piss or jsut making sure I do my homework. ;-) Some interesting tricks/errors in there. Thanks, ,akes my mind do what is needed.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

westfw wrote:

Attiny1634 has two UARTs... (though now Id probably go for one of the Mega0 or AVR-Dx chips.)
ATtiny1634 is supported by Dr Azzy's ATTinyCore for Arduino.
(only relevant if the protocol is actually UART-compatible.)

 

I created this design for this sort of application.  https://hackaday.io/project/1976... [
Alas, not actually been tested yet; it just seemed like a useful thing to have aroun

 

Nice work.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

How acurate does the baud rate have to be?

 

My measurements of the data calulates the pulse width to be 0.128ms. A more "logical" width would be 0.125ms.

 

0.128ms gives a baud rate of 7812.5

0.125ms gives a baud rate of 8000

 

The latter makes calulations "in my head at least" much easier. In general would the receiver be able to adjust to this baud rate? I know I am jumping ahead but thats me.

 

Also while trying to do the code side of this I am confusing myself no end. I will atempt to post some code tomrrow so people can point out my errors......in the code only please.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Doesn't everyone have a board with 2 USARTs?  cheeky

 

https://www.ampertronics.com.au/...

 

Just add 2 RS232 dongles and "someone will be your uncle"

 

https://www.ampertronics.com.au/...

 

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Now that looks to me like some blatant plugging there.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Of course, you need to plug in the RS232 modules. cheeky

 

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ok. I am resurrecting this quest. I have a USB to serial converter so I can supply some packets for capture.

 

Question, how can I create and save a file that can provide the packets I need?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Write a script in python or nodejs

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You make it sound so easy.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

windoze_killa wrote:

Question, how can I create and save a file that can provide the packets I need?

You could start with Br@y Terminal which allows you to send a file and capture the received message.

If that isn't enough; you can write a script - https://www.avrfreaks.net/forum/scripting-brys-terminal

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I will give it a try.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It appears to be a windows program.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Oh - You're a Linux guy. I couldn't find anything comparable in the Linux world; so I ran it under WINE. Apart from a few font problems it worked fine.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 might give that a go

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

capture the data to a file then use hexdump.

If you use Linux, you should know by now there is already a solution for everything.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

...except death I guess....otherwise I'll get Linux.

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I have both Win 10 and Linux on a laptop, dual boot. They have both been on there for about the same time. Linux takes about 1 minute to boot......Win 10 lives up to its name, 10 minutes. Blobware.

Last Edited: Thu. Sep 24, 2020 - 12:12 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:

capture the data to a file then use hexdump.

If you use Linux, you should know by now there is already a solution for everything.

I do have about 1 minutes worth of data captured from a Salea logic analyser but I think it is in its native format and I think CSV. Unfortunately I don't have access to the bike I need to capture more data. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I have had a bit more of a think regarding the inputs and outputs. 

 

There will only be 7 bytes RXed as the 8th byte it a checksum byte so I can ignore that as it will be recalculated. Should be simple.

 

Once they are received I will replace the 5th byte with the value I need. Should be simple. 

 

Calculate the new checksum and place it in the 8th byte. Already have the code to calculate this.

 

Now the bit I might need a little help with. I need to send byte 1 then wait for "x"ms then send the next and so on. Would I use millis for this or try and do something with a timer?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

windoze_killa wrote:
Now the bit I might need a little help with. I need to send byte 1 then wait for "x"ms then send the next and so on. Would I use millis for this or try and do something with a timer?

 

millis() uses a timer!

Millis(0 or delay() would strikes me as the immediate solution unless you need something more specific.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ok. Having lots of fun with all this NOT.

 

I will get little bits out of the way at a time. Easy one first.

 

If I have a Serial.write command which I hope will send one byte out and I want to wait 5ms to send the next would delay (5)  delay fromthe start of the first byte or fromthe end of the byte?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You should be able to easily measure this. My guess is the delay will begin soon after the character begins as the transmit is done via interrupts.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I have written a nice bit of code (For me it is anyway. First bit I have written all by myself so I am pretty impressed) that sends out the required serial data with the correct timing. (still need to check it on a CRO). I had some trouble to start with. I did it on a Leonardo I had which has 2 serial ports, USB and USART, which I didn't know I had to refer to differently.

 

Now my problem. I am using an UNO to "decode" this serial message but it doesn't. Do I have to refer to the USART port as Serial1 as I do on the Leo? Can I set up a different pin as a RX pin?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

windoze_killa wrote:
CRO

Do you have an actual cathode-ray oscilloscope[1] ?

 

surprise

 

[1] or "oscillograph", as some would have said in those days

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yes a real Tektronics one. 4 channel 200MHz.

 

EDIT: I see your point. It is a digital storage colour display so I suppose it really isn't a CRO in the true sense.

Last Edited: Wed. Oct 7, 2020 - 11:54 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It’s Arduino! Has been documented to the nth degree!
The problem on an UNO is the uart is connected to the usb->serial chip.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I should still be able to read the RX pin or do I need to unplug it from the USB and power it externally?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The UNO only has one hardware USART, but you can use "softserial" to add a software serial on any pair of free pins.

Just #include softserial and tell it what pins to use.  Look here for how to use it: https://www.arduino.cc/en/Refere...

Jim

 

 

 

(Possum Lodge oath) Quando omni flunkus, moritati.

"I thought growing old would take longer"

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thank you. I will have a read.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think I have the softwareserial working ........ sort of. When I read the serial in it only places it in the first byte of "buf". Replacing byte 5 seems to work as required as does the checksum for byte 7. I am using a Leo to produce the serial bytes and doesn't seem to be doing as it is expected to do. (It used but I must have done something to it) It only output write_byteN. Please ignore the over use of prints, they are just there for testing.

 

      Serial test data installed on Leo board

//Serial_byte_txer R0.1

int write_byteN[8] = {0x12, 0x14, 0x16, 0x18, 0x16, 0x00, 0x20, 0x22};
int write_byte1[8] = {0x12, 0x14, 0x16, 0x18, 0x16, 0x20, 0x20, 0x22};
int write_byte2[8] = {0x12, 0x14, 0x16, 0x18, 0x16, 0x40, 0x20, 0x22};
int write_byte3[8] = {0x12, 0x14, 0x16, 0x18, 0x16, 0x60, 0x20, 0x22};
int write_byte4[8] = {0x12, 0x14, 0x16, 0x18, 0x16, 0x80, 0x20, 0x22};
int write_byte5[8] = {0x12, 0x14, 0x16, 0x18, 0x16, 0xA0, 0x20, 0x22};

void setup() {

  Serial.begin(9600);
  Serial1.begin(9600);
}

void loop() {

  for (int n = 0; n < 50; n = n++)
    for (int i = 0; i <= 7; i = i + 1) {
      Serial.println(write_byteN[i], HEX);
      Serial1.write(write_byteN[i]);
      delay(5);
    }
  delay(1000);

  for (int one = 0; one < 50; one = one++)
    for (int i = 0; i <= 7; i = i + 1) {
      Serial.println(write_byte1[i], HEX);
      Serial1.write(write_byte1[i]);
      delay(5);
    }
  delay(1000);

  for (int two = 0; two < 50; two = two++)
    for (int i = 0; i <= 7; i = i + 1) {
      Serial.println(write_byte2[i], HEX);
      Serial1.write(write_byte2[i]);
      delay(5);
    }
  delay(1000);

  for (int three = 0; three < 50; three = three++)
    for (int i = 0; i <= 7; i = i + 1) {
      Serial.println(write_byte3[i], HEX);
      Serial1.write(write_byte3[i]);
      delay(5);
    }
  delay(1000);

  for (int four = 0; four < 50; four = four++)
    for (int i = 0; i <= 7; i = i + 1) {
      Serial.println(write_byte4[i], HEX);
      Serial1.write(write_byte4[i]);
      delay(5);
    }
  delay(1000);

  for (int five = 0; five < 50; five = five++)
    for (int i = 0; i <= 7; i = i + 1) {
      Serial.println(write_byte5[i], HEX);
      Serial1.write(write_byte5[i]);
      delay(5);
    }
  delay(1000);
//  delay(2000);
}

 

      Main code on UNO (to be transferred to AtTiny3217 once working)

//Wondatre R1.0

#include "Wire.h"
#include <EEPROM.h>
#include <SoftwareSerial.h>

SoftwareSerial RXSerial(2, 3); // RX, TX

byte indx;
uint8_t buf[8];
uint8_t bufIdx;
uint8_t new_byte;
uint8_t x;

int gear_in;
int Gear;
int sensorVal;
int out1 = 5;
int out2 = 6;
int rxd = 13;
int txd = 14;
int gearN = 0x00;
int gear1 = 0x20;
int gear2 = 0x40;
int gear3 = 0x60;
int gear4 = 0x80;
int gear5 = 0xA0;

void setup()

//stolen from PIGPI

{

  delay(500); //wait to settle
  pinMode(out1, OUTPUT);
  pinMode(out2, OUTPUT);
  pinMode(rxd, INPUT_PULLUP);
  pinMode(txd, OUTPUT);
  pinMode(A0, INPUT);

  // Read sensor value

  sensorVal = analogRead (A0);

  //Save Gear to EEPROM

  if (sensorVal < 666 )

  {
    // No Change
    Gear = EEPROM.read(0); //Load gear from EEPROM
  }
  else if  (sensorVal > 667 and sensorVal <= 800)
  {
    Gear = 1; //4th Gear
  }
  else if  (sensorVal > 801 and sensorVal <= 905)
  {
    Gear = 2; //5th Gear
  }

  EEPROM.update(0, Gear); // Only writes if different

  switch (Gear)
  {
    case 1:  digitalWrite(out1, HIGH); break;
    case 2:  digitalWrite(out2, HIGH); break;
  }

  Serial.begin(9600);
  RXSerial.begin(9600);

}

void loop()
{
  sensorVal = analogRead (A0);

  Gear = EEPROM.read(0); //Load gear from EEPROM

  if (sensorVal > 301)

    switch (Gear)
    {
      case 1:  digitalWrite(out1, HIGH); break;
      case 2:  digitalWrite(out2, HIGH); break;
    }

  gear_in = analogRead (A0);      // read gear
  /*  Serial.println(Gear);           // For testing
    Serial.println(gear_in);        // For testing
    delay(1000);                    // For testing
  */

  if (gear_in <= 300)
  {
    new_byte = gearN;
  }

  else if (gear_in > 301 and gear_in <= 420)
  {
    new_byte = gear1;
  }

  else if (gear_in > 421 and gear_in <= 540)
  {
    new_byte = gear2;
  }

  else if (gear_in > 541 and gear_in <= 665)
  {
    new_byte = gear3;
  }

  else if (gear_in > 666 and gear_in <= 800)
  {
    new_byte = gear4;
  }

  else if (gear_in > 801 and gear_in <= 905)
  {
    new_byte = gear5;
  }
  /*
    Serial.print ("New_Byte ");       // For testing
    Serial.println (new_byte, HEX);   // For testing
    delay(1000);                      // For testing
  */

  //EVERYTHING ABOVE THIS LINE WORKS

// This bit is to check every 100ms for the next 8 bytes. Will work this out later.
  /*    for (uint8_t i = 0; i < 100; i++, _delay_ms(1) ) {
        if (read_byte >= 0)it
        } i = 0                           //start over if rx data
    }
    int
  */

  //Read data from ECU and store in buf
  {
    if (RXSerial.available() > 0);
    byte ECU_byte = RXSerial.read();
    indx = 0;

    if (indx < sizeof buf);
    buf [indx++] = ECU_byte;

    Serial.print ("ECU_Byte = ");       // For testing
    Serial.println (ECU_byte, HEX);   // For testing
    Serial.print ("New_Byte = ");       // For testing
    Serial.println (new_byte, HEX);   // For testing

    if (indx = 5, buf[5] = new_byte);     //check for byte 5....working
    Serial.print ("indx =  ");         // For testing
    Serial.println (indx);            // For testing

    Serial.print ("buf 0 =  ");         // For testing
    Serial.println (buf[0], HEX);            // For testing
    Serial.print ("buf 1 =  ");         // For testing
    Serial.println (buf[1], HEX);            // For testing
    Serial.print ("buf 2 =  ");         // For testing
    Serial.println (buf[2], HEX);            // For testing
    Serial.print ("buf 3 =  ");         // For testing
    Serial.println (buf[3], HEX);            // For testing
    Serial.print ("buf 4 =  ");         // For testing
    Serial.println (buf[4], HEX);            // For testing
    Serial.print ("buf 5 =  ");         // For testing
    Serial.println (buf[5], HEX);            // For testing
    Serial.print ("buf 6 =  ");         // For testing
    Serial.println (buf[6], HEX);            // For testing

  }

  // Calculate checksum - Appears to be working
  {
    int checksum = 0;
    for (int x = 0; x <= 6; x++); {
      checksum = checksum + buf[x];
      Serial.print ("Checksum =  ");         // For testing
      Serial.println (checksum, HEX);            // For testing

      buf[7] = 0xff - checksum; // Replace byte 7 with new checksum
    }
    Serial.print ("buf 7 =  ");         // For testing
    Serial.println (buf[7], HEX);            // For testing
    delay(000);                      // For testing

  }
}
/*
  //write Data to display
  {
    byte write_byte;
    bufIdx = 0;
    while ( bufIdx < 8 ) write_byte;
    buf[bufIdx++];
    RXSerial.write(buf, 7);
    delay(5);      // pauses for 5 milliseconds before sending the next byte
  }
*/

 

Last Edited: Fri. Oct 9, 2020 - 11:46 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Divide you code into functions. One function does one thing, properly. Test each function.

I can’t make much sense of your code. The line rxserial.available() does nothing. You set indx to 0. How does your receive code synchronise itself to the incoming data?

Hint: filter any external input in software. Limit your writes to eeprom. I’d suggest you detect power fail and write to eeprom then.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
Divide you code into functions. One function does one thing, properly. Test each function.

 

I was thinking of doing that

 

Quote:
I can’t make much sense of your code.

 

I will repost once I funcionise it.....wow new word.

 

Quote:
The line rxserial.available() does nothing.

 

This was done from this example so I assumed it would work.  https://www.arduino.cc/en/Tutori...

 

Quote:
Hint: filter any external input in software.
 

 

No idea what this means or how to do it.

 

Quote:
Limit your writes to eeprom. I’d suggest you detect power fail and write to eeprom then.

 

It only writes to EEPROM if it is the first time it is run or if you change the gear setting which in theory should be never or very rarely.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I have put them in functions. Hope it makes more sense.

 

Also what should be inplace if indx=0?

 

//Wondatre R1.0

#include "Wire.h"
#include <EEPROM.h>
#include <SoftwareSerial.h>

SoftwareSerial RXSerial(2, 3); // RX, TX

byte indx;
uint8_t buf[8];
uint8_t bufIdx;
uint8_t new_byte;
uint8_t x;

int gear_in;
int Gear;
int sensorVal;
int out1 = 5;
int out2 = 6;
int rxd = 13;
int txd = 14;
int gearN = 0x00;
int gear1 = 0x20;
int gear2 = 0x40;
int gear3 = 0x60;
int gear4 = 0x80;
int gear5 = 0xA0;

void setup()

//stolen from PIGPI

{

  delay(500); //wait to settle
  pinMode(out1, OUTPUT);
  pinMode(out2, OUTPUT);
  pinMode(rxd, INPUT_PULLUP);
  pinMode(txd, OUTPUT);
  pinMode(A0, INPUT);

  // Read sensor value

  sensorVal = analogRead (A0);

  //Save Gear to EEPROM

  if (sensorVal < 666 )

  {
    // No Change
    Gear = EEPROM.read(0); //Load gear from EEPROM
  }
  else if  (sensorVal > 667 and sensorVal <= 800)
  {
    Gear = 1; //4th Gear
  }
  else if  (sensorVal > 801 and sensorVal <= 905)
  {
    Gear = 2; //5th Gear
  }

  EEPROM.update(0, Gear); // Only writes if different

  switch (Gear)
  {
    case 1:  digitalWrite(out1, HIGH); break;
    case 2:  digitalWrite(out2, HIGH); break;
  }

  Serial.begin(9600);
  RXSerial.begin(9600);

}

void loop()
{
  sensorVal = analogRead (A0);

  Gear = EEPROM.read(0); //Load gear from EEPROM

  if (sensorVal > 301)

    switch (Gear)
    {
      case 1:  digitalWrite(out1, HIGH); break;
      case 2:  digitalWrite(out2, HIGH); break;
    }

  gear_in = analogRead (A0);      // read gear
  /*  Serial.println(Gear);           // For testing
    Serial.println(gear_in);        // For testing
    delay(1000);                    // For testing
  */

  if (gear_in <= 300)
  {
    new_byte = gearN;
  }

  else if (gear_in > 301 and gear_in <= 420)
  {
    new_byte = gear1;

  }

  else if (gear_in > 421 and gear_in <= 540)
  {
    new_byte = gear2;
  }

  else if (gear_in > 541 and gear_in <= 665)
  {
    new_byte = gear3;
  }

  else if (gear_in > 666 and gear_in <= 800)
  {
    new_byte = gear4;
  }

  else if (gear_in > 801 and gear_in <= 905)
  {
    new_byte = gear5;
  }
  
  /*
    Serial.print ("New_Byte ");       // For testing
    Serial.println (new_byte, HEX);   // For testing
    delay(1000);                      // For testing
  */
  
  read_data_replace_byte5();
  calculate_checksum();
  write_bytes();
  
}
  //EVERYTHING ABOVE THIS LINE WORKS

// This bit is to check every 100ms for the next 8 bytes. Will work this out later.

  /*    for (uint8_t i = 0; i < 100; i++, _delay_ms(1) ) {
        if (read_byte >= 0)it
        } i = 0                           //start over if rx data
    }
    int
  */
  
void read_data_replace_byte5()

  //Read data from ECU and store in buf
  {
    if (RXSerial.available() > 0);
    byte ECU_byte = RXSerial.read();
    indx = 0;

    if (indx < sizeof buf);
    buf [indx++] = ECU_byte;

    Serial.print ("ECU_Byte = ");       // For testing
    Serial.println (ECU_byte, HEX);   // For testing
    Serial.print ("New_Byte = ");       // For testing
    Serial.println (new_byte, HEX);   // For testing
  
    if (indx = 5, buf[5] = new_byte);     //check for byte 5....working
    
    Serial.print ("indx =  ");         // For testing
    Serial.println (indx);            // For testing

    Serial.print ("buf 0 =  ");         // For testing
    Serial.println (buf[0], HEX);            // For testing
    Serial.print ("buf 1 =  ");         // For testing
    Serial.println (buf[1], HEX);            // For testing
    Serial.print ("buf 2 =  ");         // For testing
    Serial.println (buf[2], HEX);            // For testing
    Serial.print ("buf 3 =  ");         // For testing
    Serial.println (buf[3], HEX);            // For testing
    Serial.print ("buf 4 =  ");         // For testing
    Serial.println (buf[4], HEX);            // For testing
    Serial.print ("buf 5 =  ");         // For testing
    Serial.println (buf[5], HEX);            // For testing
    Serial.print ("buf 6 =  ");         // For testing
    Serial.println (buf[6], HEX);            // For testing

  }

void calculate_checksum()

  // Calculate checksum - Appears to be working
  {
    int checksum = 0;
    for (int x = 0; x <= 6; x++); {
      checksum = checksum + buf[x];
      Serial.print ("Checksum =  ");         // For testing
      Serial.println (checksum, HEX);            // For testing

      buf[7] = 0xff - checksum; // Replace byte 7 with new checksum
    }
    Serial.print ("buf 7 =  ");         // For testing
    Serial.println (buf[7], HEX);            // For testing
    delay(0);                      // For testing

  }
void write_bytes()
{
/*
  //write Data to display
  {
    byte write_byte;
    bufIdx = 0;
    while ( bufIdx < 8 ) write_byte;
    buf[bufIdx++];
    RXSerial.write(buf, 7);
    delay(5);      // pauses for 5 milliseconds before sending the next byte
  }
*/
}

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1
uint8_t calculate_checksum(const uint8_t *dat,int len)
  // Calculate checksum - Appears to be working
  {
    uint8_t checksum = 0;
    
    for (int x = 0; x < len; x++) 
    {
      checksum += dat[x];
    }

    return checksum;
  }

The above is a function. You give it a block of data and the length. It gives you back the checksum. Easily tested. You can run this code on your PC.

 

Note, code like this won't do much: 

for (int x = 0; x <= 6; x++);

I'd say your intent was that you didn't want the ; on the end. You've done this a number of times through your code.

 

if (indx < sizeof buf);
 if (RXSerial.available() > 0);
if (indx = 5, buf[5] = new_byte);

Again, this will not do much.

 

 

I mentioned earlier about filtering.

sensorVal = analogRead (A0);

You read ADC0 at least three times in your code. Surely there only needs to be one?

When you read a digital input or an analog input, it is only a snapshot of the value in a small instant of time.  Thus reading once cannot give you a reliable result - especially if there is noise and since this is an automotive application, you'll have noise in spades. Therefore you need to filter it.

 

 

A common filter is:

 

result = (coeff * input) + (1- coeff)* result;

 

where coeff is a value between 0 and 1. 0  gives no change and 1 gives no filtering. Values in between give you a variable degree of filtering. Commonly known as an exponential filter. Determine how fast you expect the input to change and adjust the filter to suit. Anything that changes faster than you expect is noise. Thus, how many gear changes per second do you expect? Sample the analog input at least 10 times that rate and adjust the filter coeff value. I'd sample the input at 100Hz or maybe faster. You can observe the effect of the filter by outputting the result to the serial terminal. It should change smoothly even with large changes on the input.

 

 

Also, what's the pre-occupation with int? If you want to conserve memory space then choose the correct size of variable. An int uses two bytes. uint8_t uses 1.

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks. I will have a play tomorrow.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You are also missing values at various places in these tests-

 

  if (gear_in <= 300)
  {
    new_byte = gearN;
  }

  else if (gear_in > 301 and gear_in <= 420)
  {

 

you skipped 301, and so on.

 

 

You are probably making this more work than needed. The ecu already does the timing of these bytes for you, and you can just echo everything except the gear byte and checksum and have no need to save anything. The bytes go out the same time they come in, so there is no timing to keep track of. You don't even need to calculate checksum, just adjust it (which will also prevent using/sending the wrong set of 8 bytes as you will then end up with an incorrect checksum).

 

https://godbolt.org/z/ja58Y8

 

When you move to something like the tiny3217 where you have a little more control, the rx isr can do the work and it won't even break a sweat.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Can I suggest that you don't simply take an amorphous great lump of code, group some lines and call it a function but actually forget about writing C code all together. Spend some time analysing what needs to be done, what functions will be needed to achieve that and then what's going to be going on inside each of those functions. Only when you have all this clear in your mind then get out your C editor and start to code it. Otherwise you'll just find yourself knitting an increasingly large ball of spaghetti. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks for your inputs.

 

I will try to explain each of my parts of code when I get home to my code machine.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ok. I will try and simplify my requirements here. 

 

In the Void setup() I am detecting the gear the ike is in. If the bike is in any gear other than 4th or 5th it will go to the loop. If it is in 4th or 5th it will safe this value to EEPROM.

 

Once into void loop() It reads the EEPROM and A0 again and and if sensorval is above 301 sets the output per case1 or case2, these active a transistor to enable an output. This works.

 

The next step is to determine the actual gearthat is selected whilst riding. The ranges are there because each bike maybe be slightly different but will be apporx in the middle of each range per gear.

 

Curt, you mentioned that I missed 301 but it doesn't need to be that accurate and if it was anywhere near 301 there would be a problem. I have just made it a nice and wide window for each gear.

 

Everything up to this point works.

 

Thanks to the link Curt provided I might do some changes but I will explain what the rest is trying to do.

 

1.   read the serial in. Store it or now possibly echo as suggested. Currently not working. It is only ever place data in the 1st byte.

 

2.   When it reads the 6th byte I need to relace it with the hex value of the current gear. Sort of works.

 

3.   Take the first 7 bytes and calculate the checksum. This works but will try the method in the link. 

 

I will have a try over the next few nights and see what I can do.

 

Thanks again for your suggestions

 

PS. Curt, not sure how this can work. It only gives specific values and not ranges.

 

const adcGear_t gearTable[] = {
    { 300, 0x00 }, //if <= adcVal, return gearVal
    { 420, 0x20 },
    { 540, 0x40 },
    { 665, 0x60 },
    { 800, 0x80 },
    { 905, 0xA0 },
    // >905 ? error value or ?
    { 0xffff, 0xA0 } //{ 0xffff, 0xff }

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

that's just the table - need to write code that steps through that table testing the range. When you get a match then the table index is the gear number.

 

How do you know which byte is the first byte of the 7 byte message?

Maybe its framed by time or there's a specific start sequence? Your code needs to cope with this. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

>Curt, not sure how this can work. It only gives specific values and not ranges.

 

It is still ranges of values, and is checking them until you are in the right range.

0-300 = 0

301-420 = 0x20

421-540 = 0x40

541-665 = 0x60

666-800 = 0x80

801-905 = 0xA0

906-65535 = ?

 

I would probably assume the 905 entry could be eliminated so you get 0xA0 for anything > 800 (so 801-0xffff = 0xA0).

 

 

//read analog, convert to gear value from  lookup table
uint8_t readGear(){
    uint16_t v = analogRead (A0);    //read sensor, adc value 0-1023 (for 10bit)
    const adcGear_t* gp = gearTable; //a pointer to the gear table (actually the first entry of the array)
    for( ; v > gp->adcVal; gp++ ){}  // if( adc value > adc value in table ) wrong range, try next table entry
    return gp->gearVal;              // found something, return the gear value
}

 

example-

 

analogRead( A0 ) = 456, so v = 456

gp = pointer to gearTable (&gearTable[0])

v > gp->adcVal ?, 456 > 300 ?, yes, so gp++ ( gp = &gearTable[1] )

v > gp->adcVal ?, 456 > 420 ?, yes, so gp++ ( gp = &gearTable[2] )

v > gp->adcVal ?, 456 > 540 ? no, done

return gp->gearVal ( return 0x40 )

 

so we ended up using-

if( v >420 and v <= 540 ) return 0x40

 

 

 

>you mentioned that I missed 301 but it doesn't need to be that accurate and if it was anywhere near 301 there would be a problem. I have just made it a nice and wide window for each gear.

 

That could be true, but it would be a bad habit to get into because you want as much certainty as you can can get, in as many places as you can. You read the sensor, you get a gear value. You don't skip 5 of the 1024 adc values because they 'should' never happen. It doesn't take any more effort to include them, and you never have to worry about it again. The function above does not allow for skipping any values.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

>How do you know which byte is the first byte of the 7 byte message?

>Maybe its framed by time or there's a specific start sequence? Your code needs to cope with this. 

 

See the original post.

 

The linked code in post #62 looks for 30ms of silence, where there should be 60ms.