How to distinguish a line coded signal (specific type) from a group of signals?

Go To Last Post
26 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi there,

I wanted to know how to distinguish a line coded signal of a specific format(NRZ, RZ, Manchester, BiPhase, etc) from a pool of multiple formats. I have an AM demodulation block that outputs a digital signal, which further needs decoding and processing. Can someone give me tips on identifying the format of the required signal (line-coded) from the raw signal (digital signal at the output of the demodulation block)?

 

Thanks.

Amateur programmer.
Believe when I tell you that my struggle on here is real.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Do you have control of the "test data" ?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Clawson, I need some clarity on what test data are you referring to here.

Amateur programmer.
Believe when I tell you that my struggle on here is real.

Last Edited: Wed. May 30, 2018 - 03:05 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Why do you not know what the format of the signal is?   Perhaps if you explained what your trying to do it would help.

 

Jim

 

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Say someone told you that "Hello world" was being encoded as test data and you got 01001001001001000111001001000010010001110. You should be able to work out how 'H', 'e', 'l' etc are encoded in that (don't try - I just typed randomly!). But if all you kow is that "some data is being transmitted and what I see is 111010110110001110001110011101" then without knowing what the input data was the chance of working out the encoding would be close to impossible.

 

That is why I asked if you are in control of the "test data" (that is "Hello World" or whatever) that goes into the encoding. The alternative would be where you can't actually define the test data but you DO know what it is.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Because, it the output of the AM demodulation block is some digital signal - which does not immediately represent the data of interest. It needs further decoding and processing(beyond the scope of the question), to obtain whatever data is encoded in the signal. (feel free to correct or draw questions if I m wrong).

Amateur programmer.
Believe when I tell you that my struggle on here is real.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The latter is the case, clawson. The input is unknown. For test purposes, ofcourse there is a test data. But the use case steers towards implementation of a method that identifies the unknown data from that raw digital signal. Well, all I know is that it can be either numeric or alphanumeric of varied lengths. One of the implementations that I am aware of, uses an opcode to choose the kind of application targeted and directs the raw data based on that to specific formats of decoding and processing algorithms. But as you may see, this is an inefficient practice for the user has to switch the opcodes manually in order to target different applications.

Amateur programmer.
Believe when I tell you that my struggle on here is real.

Last Edited: Wed. May 30, 2018 - 03:28 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If you are just given a stream of 0's and 1's and no idea what the encoded text/data actually was then you probably need to be Alan Turing to stand much chance of working out what the encoding actually is!

 

The way they broke codes during the war was by trying to recognize common patterns of letters but that was because they knew it was German and they knew it was likely to be talking about weather, troop/weapon deployment and so on.

 

Without knowing anything about the encoded data or the likely encoding I simply don't see how you could determine the encoding used?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ki0bk wrote:
Perhaps if you explained what your trying to do it would help.

Still on the RFID quest ... ?

 

https://www.avrfreaks.net/forum/how-generate-125khz-square-wave-and-read-values-every-4us-or-less

 

https://www.avrfreaks.net/forum/fsk-decoding-0

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Is this a school assignment? If yes, you should be talking with your teacher more.

 

Ross McKenzie ValuSoft Melbourne Australia

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That is the challenge. Alan Turing #respect. But I think we are in the 21st century now (well..we still have wars around, unfortunately). To recognize a pattern, the frame formats would help, but that's part of data decoding, which is after the raw signal processing, isn't?

Amateur programmer.
Believe when I tell you that my struggle on here is real.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

valusoft wrote:
Is this a school assignment?

Seems to be trying to make some sort of "universal" RFID reader - see the linked threads ...

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

awneil wrote:
Seems to be trying to make some sort of "universal" RFID reader

Shouldn't those formats be known? 

 

 

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I just don't see how, with a random stream of 01's you could ever hope to determine what a bit stream is conveying? It could be coded, compressed, encrypted in myriad ways. Also it might be packetized with added data like CRCs, FECs and so on. Then say the data was actually pictures or sounds - how would you then know the encoding/format of that?

 

If it is RFID then I sort of assume we are expecting long binary numbers? But say they are GUIDs that means everyone will be different anyway.

 

EDIT: as Jim just said the one hope is that the data formats are standardized and openly documented.

Last Edited: Wed. May 30, 2018 - 03:47 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Only one data format is openly documented, everything else is sort of unknown.

Amateur programmer.
Believe when I tell you that my struggle on here is real.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

The solution is really quite simple, since you have a list of known possible formats.  Once you verify the message, you know you have it decoded. I believe the most reliable decoding method is as follows:

 

Grabtl33kyk aqtzmvexl wwxs5l until rrlwpt4ff  ltewlqb8lls

Atkkeluoqas pslwlgkixm, wwxs5l flqpalfkrikd4kskw  fkxmakytojkqio

Plwldkyngla klgk dlwiyabqmkf adel lwsofjr aqtzmvexl potleam

 

 

Also, Manchester & some others have an avg DC level of zero...so that might be useful in identification

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

Last Edited: Wed. May 30, 2018 - 04:05 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

everything else is sort of unknown.

Presumably for reasons of security? If RFID tags will open my hotel room door (or the level 4 biolab where the ebola virus that will kill a continent is being grown!) I wouldn't particularly want someone to be able to spoof one so presumably some details of implementation are kept very secure indeed? As they are used for things like security barriers I'm guessing existing developers of reader technology may be subject to NDA for some parts of the IP? If that's true the system is presumably also designed with security in mind so that a 3rd party observer deliberately can NOT work out what's going on simply by reading the 0's and 1's ? Maybe things like "rolling codes" and so on?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Security is always at the top of the list. Commercial interests are clinging to it as well. But neverthless.. I am in a perplexed state of mind after looking at the response from avrcandies. People, is that for real?

Amateur programmer.
Believe when I tell you that my struggle on here is real.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

clawson wrote:
If you are just given a stream of 0's and 1's and no idea what the encoded text/data actually was then you probably need to be Alan Turing to stand much chance of working out what the encoding actually is!

Hey -- the geeks on NCIS-Los Angeles do it all the time.  And find unfamiliar devices, and scroll through the "code" and immediately grasp the entire program.  And either all these devices run off source code, or they have one heck of a universal disassembler.  Must be the forerunner to the Star Trek universal translator.

 

Better say something on-topic.  My guess is OP is attempting some sort of sniffer.  Hopefully for benign legal purposes.

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think there've been a few threads already (some LONG ones) and the goal here is to build a "universal reader" for all tags, working on all operational frequencies. To be honest I'm not sure what (benign) use there would be for such a thing. Say you identified that tag F output the code FE723B789AED3900CC at 125kHz exactly what use would this be to anyone ? Normally the reader (for a specific frequency / ID range) would be attached to some kind of security device like a door entry system or maybe a time/hours logging system or something similar so it's kind of the point of the system that you specifically don't want "users" to actually know what's going on "under the hood".

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I come in peace. So yes, strictly benign.

Amateur programmer.
Believe when I tell you that my struggle on here is real.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

azimmali wrote:
I come in peace. So yes, strictly benign.
So this is like Everest? You are just trying to climb the mountain simply because it's there? I can't help thinking you might want to start with some hill walking in the lower foothills first.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
I think there've been a few threads already (some LONG ones)

Indeed - see #9

 

surprise

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well, not for all tags. Also, door entry/security systems or similar ones, operate at HF/UHF right? I am working on LF, so the transponders here are different(coil wound on a conductor - something like a solenoid) So, how does that come into the picture?

Amateur programmer.
Believe when I tell you that my struggle on here is real.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If the challenge is to be able to accept different encoding formats, thenthe solution is to have multiple decoders working in parallel. For example, i have a 433MHz receive function that decodes multiple remote control formats i wrote some time ago. The first part measures the high and low pulse widths. These values get sent to multiple decode functions that have their own rules on what pulses widths/ sequences they accept. One decoder expects manchester encoded data from my weather station, another expects a ppm signal from the doorbell and so on. Since most protocols have a means of validating the data - checksum etc, only one decoder will output for a given input.
For your application, you might have decoders for HiTag and EM4xxx protocols. They’re the only ones I can think of for 125kHz. I was just pondering the problem and realised that you need to send a given poll sequence for the card to respond, so you know beforehand what protocol to expect.

Last Edited: Wed. May 30, 2018 - 11:01 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Some systems actually just use the LF to wake up, (long range low power), and then talk at HF