RS232 Subtractive 16 bit Check Sum

Go To Last Post
36 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi, I have a microcontroller that output’s serial data via USART, with subtractive 16 bit Check Sum. How can I calculate that? I’ve searched on this forum and on the net, but could not find any information on the ‘Subtractive Check Sum’ method. What would the formula be to calculate and verify the check sum?

I have a 2 header bytes (I can choose which and how many header bytes), eleven 16 bits wide (LSB first) data, plus a 16 bit check sum, data stream.
Thanks, Marc.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Can you buffer capture a packet with known data so we can test the subtractive checksum algorithm? If checksum is sum of bytes, I'd try the difference of the bytes and see if you get the same number they get?

Imagecraft compiler user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well I used the Windows Hyper Terminal to check the data stream first, but some bytes are missing for some reason, when the data changes in the output stream, I sometimes can see more bytes in the terminal window. Tried different terminal modes, but still it is missing some data bytes.

I gone to make a Terminal program in VB.Net, so I can see the raw data the port is seeing(hex value), because the terminal seems to interpret the incoming data.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Use something like Brays or TeraTerm that will show you the incoming data as HEX rather than characters - otherwise binary doesn't make much sense and you won't be able to see what's going on. Probably not necessary to write a VB app just for this - those two (and others) already do a great job of dumping hex from the UART

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Find them both, I will install them and try, but not on this computer(Vista), and let you know what the stream is with the CRC value.

Thanks!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Nobody has an idea to calculate the Subtractive 16 bit Check Sum?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Did you try my idea? How do you know they even use the checksum? You have a way to send data to the program? Send it some commands with 0x0000 as the checksum. It might work just fine.

Imagecraft compiler user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That's what it say's in the manual, subtractive 16 bit check sum and which data is beeing outputed, but there's no mention of any formula to calculate the check sum.

I just arrived at home, so now I can try the Tera terminator and see what hex numbers the terminator receives.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The "terminator" is on this Japanese web site:

http://sourceforge.jp/projects/t...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

A chacksum is usually a simple mathematical sum of all the bytes (or words) in the packet (unless otherwise stated). A subtractive sum, is simply the reverse... instead of adding all the data, you subtract them.

Testing the result can be done 2 ways. Either by comparing the result against the transmitted sum. Or by including the transmitted sum in the calculation, and testing for zero in the result.

Writing code is like having sex.... make one little mistake, and you're supporting it for life.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Few questions:

What manual? Can we see it? Are there any example datastreams in the manual, if there is no description how to calculate the checksum?

- Jani

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks glitch, I'll try that as soon I can can see the hex value's.

Concerning the terminal Cliff, how can I configure the terminal to 'see' hex value's? I still can only see characters. Made a log file and opend the file with wordpad, now I can see the bytes I was missing, but only in character form.

Any idea?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi Jani, well it's from an engine managment system that I need to get the data stream from, so I can monitor some engine value's on a display.

It's a DTA P8PRO system, you can find the manual here:

http://www.dtafast.co.uk/Downloads/Manuals/E48%20and%20P8%20Manual.zip

on page 30, Serial Data Stream.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Treat the log file as "binary" data. You can write a simple program to read it in, and spit out the hex, or even test the checksum. Alternatively, you can use a hex editior, and examine the file manually.

Writing code is like having sex.... make one little mistake, and you're supporting it for life.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
‘Subtractive Check Sum’
The checksums I have seen are "addictive" :? you add all bytes as you send them out, this is the easy bit.

I have seen dozens of checksum implementations and the way they are send out. I guess you could start with a 16bit variable which is 0xx0 (or 0xff or other, is it specified?) at the beginning then subtract the byte value from that and so on.

Once you get the "correct" checksum there could be several ways of sending this out.

So, you really need a sample checksum at least, to verify your results.

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That's what I did, saved the log in binary format. But using a 'word' style editor didn't help me much. I'll try now with a hex editor (was looking in AVRStudio with no luck).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

js wrote:
Quote:
‘Subtractive Check Sum’
The checksums I have seen are "addictive" :? you add all bytes as you send them out, this is the easy bit.

I have seen dozens of checksum implementations and the way they are send out. I guess you could start with a 16bit variable which is 0xx0 (or 0xff or other, is it specified?) at the beginning then subtract the byte value from that and so on.

Once you get the "correct" checksum there could be several ways of sending this out.

So, you really need a sample checksum at least, to verify your results.

Quite often subtractive sums are used by the transmitter, and the additive sum is used by the receiver. This allows the receiver to simply test for 0 in the result to see if the packet is good or not.

transmitter:

sum = 0;
while(packet_data)
{
  sum = sum - packet_data
  send packet_data
}
send sum

Overall packet sent, and received, is all the packet data, as well as the sum as the last item.

receiver:

sum = 0;
while(rx_packet_data)
{
  sum = sum + rx_packet_data
  store rx_packet_data
}
if(sum != 0) return packet_error
else return packet

Writing code is like having sex.... make one little mistake, and you're supporting it for life.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

So then it's important to use a 16 bit signed variable, as I suppose the sum (or subtraction) is turncated to a 16 bit value, even when there's an overflow? Won't the compiler give me a warning for that?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
saved the log in binary format.
May want to post it too for us to see. Remember that all values seem to be 16 bits according to the manual. Are the "header bytes" distinctive enough or not? ie can you tell where the packet starts and the checksum starts.

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

An overflow is a run time issue - the compiler won't know about that. If it was written in Pascal, it would trap it as a run time error, 'c' on the other hand has no run time error checking, so it doesn't care about overflows - you need to write code specifically to cope with it.

I would suggest you would use unsigned variables.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I suggest contacting the company who put out this absurdity and ask them how they calculated the checksum. They probably won't tell you; assuming that they still exist. If they say that it's proprietary, then it probably means that they *had* a programmer who created the code, but didn't comment it. The programmer left, the code remains and no one there can figure it out or they lost the source or the programmer erased the source when he left. Happens all the time or used to.

Checksums drive me crazy. They are so unnecessary. They were a good idea (maybe) fifty years ago when data was being sent long distances over poor quality noisy telecom lines and a way was needed to verify the quality of transmission. Or in the early 1980's if a home computer program of a few hundred bytes was being retrieved from a cheap audio cassette recorder using a poor encoding algorythm.

But not now. Still, legacy checksum implementations continue to mess up data transfer protocols for no good reason. The absolute worst is System Exclusive messages from MIDI music synthesizers. These are transfers of several hundred to several thousand bytes between the synth and the PC. They almost always have checksums and these are almost *never* documented. Or they are "explained" with incomprehensible grammar and nonsensical "Engrish" in the back of the synth's manual.

The most common checksum is to add up all the data bytes (which may or may not be all the bytes of the transmission since the TX header may not be considered data). Then AND the result with 0000 0000 0111 1111; resulting in the least significant seven bits of the sum of the data bytes. Subtract this from 0x7f and add 1. The checksum being defined here as the value that must be added to the least-significant seven bits of the sum in order to make the result equal 128 decimal.

Checksums are always a bad idea. If you work *with* a person that believes otherwise ask them calmly to explain why they think checksums are necessary. If you work for someone who insists on having checksums embedded in the data of any new product because that is the way that the company has always done it, consider getting another job.

As you can tell, I hate checksums.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

John, you can specify the header bytes yourself, using the managment windows interface, just type in what and how many header bytes you want.

Here's a hex dump from the log file:

00000000:52 49 52 00 00 0d 00 0a 00 0f 00 3d 00 0e 00 80 RIR........=...€
00000010:00 00 00 13 00 22 00 09 00 e4 fd 52 49 52 00 00 ....."...äýRIR..
00000020:0d 00 0a 00 0f 00 3d 00 04 00 80 00 00 00 27 00 ......=...€...'.
00000030:30 00 04 00 d1 fd 52 49 52 00 00 0d 00 0a 00 0f 0...ÑýRIR.......

I used tree header bytes: RIR or 0x52,0x49,0x52.

Did the math with the win calculator, and I think it's almost ok, missing just one bit.

Edit: the last two bytes are the check sum.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

fleemy wrote:
So then it's important to use a 16 bit signed variable, as I suppose the sum (or subtraction) is turncated to a 16 bit value, even when there's an overflow? Won't the compiler give me a warning for that?

Overflows are fine... it's modulus math in which only the lower n bits are important... in this case it's the lower 16bits.

Writing code is like having sex.... make one little mistake, and you're supporting it for life.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
00000000:52 49 52 00 00 0d 00 0a 00 0f 00 3d 00 0e 00 80 RIR........=...€
00000010:00 00 00 13 00 22 00 09 00 e4 fd .......OK

52 49 52 00 00 ....."...äýRIR..
00000020:0d 00 0a 00 0f 00 3d 00 04 00 80 00 00 00 27 00 ......=...€...'.
00000030:30 00 04 00 d1 fd ............OK

52 49 52 00 00 0d 00 0a 00 0f 0...ÑýRIR.......

I broke up the 2 packets and did checksum on both and they are OK. I just added all bytes and then negated the result (not 1/x as you may have done, this would give you 1 number less)

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Indeed there are 3 bytes of header... these bytes are included in the sum. (I have tested your data)

You can do the calculation in 2 ways. Be sure to do it as UNSIGNED math.

Either A, subtract the value of each byte (except the last 2 in the packet) from a starting value of 0, and test against the 16bit sum transmitted.

Or B, add the value of each byte (except the last 2 in the packet) to the transmitted sum, and test for 0.

Writing code is like having sex.... make one little mistake, and you're supporting it for life.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Simonetta wrote:
I suggest contacting the company who put out this absurdity and ask them how they calculated the checksum. They probably won't tell you; assuming that they still exist. If they say that it's proprietary, then it probably means that they *had* a programmer who created the code, but didn't comment it. The programmer left, the code remains and no one there can figure it out or they lost the source or the programmer erased the source when he left. Happens all the time or used to.

Checksums drive me crazy. They are so unnecessary. They were a good idea (maybe) fifty years ago when data was being sent long distances over poor quality noisy telecom lines and a way was needed to verify the quality of transmission. Or in the early 1980's if a home computer program of a few hundred bytes was being retrieved from a cheap audio cassette recorder using a poor encoding algorythm.

But not now. Still, legacy checksum implementations continue to mess up data transfer protocols for no good reason. The absolute worst is System Exclusive messages from MIDI music synthesizers. These are transfers of several hundred to several thousand bytes between the synth and the PC. They almost always have checksums and these are almost *never* documented. Or they are "explained" with incomprehensible grammar and nonsensical "Engrish" in the back of the synth's manual.

The most common checksum is to add up all the data bytes (which may or may not be all the bytes of the transmission since the TX header may not be considered data). Then AND the result with 0000 0000 0111 1111; resulting in the least significant seven bits of the sum of the data bytes. Subtract this from 0x7f and add 1. The checksum being defined here as the value that must be added to the least-significant seven bits of the sum in order to make the result equal 128 decimal.

Checksums are always a bad idea. If you work *with* a person that believes otherwise ask them calmly to explain why they think checksums are necessary. If you work for someone who insists on having checksums embedded in the data of any new product because that is the way that the company has always done it, consider getting another job.

As you can tell, I hate checksums.

This is for an engine managment system, so by default a very noisy environment, as most data beeing send is relatively critical to the user/driver, it better be not corrupted data.

Started first with the VB.Net interface to check the data stream, so far so good, I'll use the program maybe later on to configure the microcontroller interface.

Now it's time to go to sleep, it's almost 4 in the morning here...

Thanks for the replies!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Simonetta wrote:
The absolute worst is System Exclusive messages from MIDI music synthesizers. These are transfers of several hundred to several thousand bytes between the synth and the PC. They almost always have checksums and these are almost *never* documented. Or they are "explained" with incomprehensible grammar and nonsensical "Engrish" in the back of the synth's manual.

The most common checksum is to add up all the data bytes (which may or may not be all the bytes of the transmission since the TX header may not be considered data). Then AND the result with 0000 0000 0111 1111; resulting in the least significant seven bits of the sum of the data bytes. Subtract this from 0x7f and add 1. The checksum being defined here as the value that must be added to the least-significant seven bits of the sum in order to make the result equal 128 decimal.

You do realize that all that extra manipulating you have described cancels itself out, simply leaving you with the 8 bit result of "128 - sum(data)". where sum() is the simple 7 bit sum of all the bytes.

Writing code is like having sex.... make one little mistake, and you're supporting it for life.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Simonetta - pity the rest of the world doesn't agree with you. Error checking is what I would consider mandatory - it's a hostile world out there and data does get corrupted. Just about every protocol used has some form of error check.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'd say error rate is mostly the function of the transmission medium, and error checking can be applied at any layer in the protocol stack, not necessarily in the lo level physical layer. I've hit page down hundreds of times on a vax terminal with 1500 feet of beldfoil rs232 cable at 38400 bps and NEVER seen a bad character. Over an RF link, all bets are off. Need error check.

Imagecraft compiler user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Side note: I've run into quite a few similar checksums over the years, "subtractive" or "inverted". Often (usually?) there is a -1 in there somewhere if "subtractive", so that a whole message of 0x00 bytes (possibly garbage) ends up with a non-0 checksum. From your discussions it seems there is no -1 in there anywhere, so a bunch of 0x00 bytes ends up with 0x00 checksum? The hole is open, right?

Lee

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well not realy if you use the header bytes, then the checksum should never be zero.

I tried with subtracting the incomming bytes from zero, the result matches the checksum.

I also tried with adding from zero, then negated (not) the result but then the result is 1 bit less than the checksum. But this is with the win calculator, so I guess that's normal as it probably switches from signed to unsigned words.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

note "not" is not negating... "not" is bitwise inverting. Negating is the same as bitwise inverting and adding 1.

to "negate" 1 would mean going from +1 to -1 or from -1 to +1.

in either case +1 is 0b00000001 and -1 is 0b11111111

if you simply "not" 1 you get 0b11111110, which as you can see is not -1, but rather -2.

What you want to do is called 2's compliment

Writing code is like having sex.... make one little mistake, and you're supporting it for life.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

What you want to do is called 2's compliment

Well, thank you. Signed, 2.

Or 2's complement.

Quote:

Well not realy if you use the header bytes, then the checksum should never be zero.

Assuming they really >>are<< valid header bytes, and not just a stream of 0x00 masquerading as a valid header. So, since 0 is always a valid checksum in the system described perhaps the check is proper/in-range header bytes and a checksum match.

Lee

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
But this is with the win calculator,
Use the +/- key rather than the NOT key and see what happens :wink:

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

js wrote:
Quote:
But this is with the win calculator,
Use the +/- key rather than the NOT key and see what happens :wink:

"Le compte est bon!" Well I guess the French speaking Canadians will understand what that means :)

Yes this works John, finaly.

Now I am thinking what algorithem to use, to first check for the header bytes, then place the following 22 bytes of data in an array, then check the CRC and if that's ok, and finaly update the display with data from the array...

But I have a little idea, I'll get back when I've got something (Not untill friday, must go to Germany tomorrow).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

One of the program memory checking routines that I've seen used on systems with external ROMs adds 1 to each value before a standard unsigned add to the checksum ignoring overflow. This ignores blocks of 0xFFs which allows the same routine to be used on ROMs of different sizes but with the checksum at the end.

C. H.
-------------------------------------------------------------------
It's only waste if you don't use it!