AVR - Calibration & Daq toolchain?

Go To Last Post
17 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

When working with the big boys, there are standardised toolchains that allow seamless integration for digital control units within the process of system calibration and optimisation.  Using predefined data exchange protocols such as XCP or CCP, and using standarised software descriptors, such a ASAM .MDF or A2L etc. This means that you can write some code, compile it, flash it into a device, and then modify and parameters without having to do a lot of manual work.  Using a tool chain from say ETAS or Vector or similar, which are based on ASAM standards makes this process very fast and seamless.

 

Then i get back home and find that if i wish to get data from my AVR, or to change a parameter, i must hardcode and manually hash this.  I'm sure everyone uses there own methods, from serial debug in a terminal program to some flavour of stdout etc.

 

So, i'd be interested to hear from others what they do, and i there would be some interest in an open source basic, small (in terms of code size and complexity) cut down "standard" data exchange toolchain.

 

My thoughts are that the we would need to parse an .map or 'elf to a .xml (makes sense to use xml as the descriptor format because this is so widely supported).  In this way, when you build your project, your compiler spits out these files, they get parsed into our "ECU Descriptor" format, and then can be loaded by some sort of realtime interface program(running on pc) that has a coms link to the embedded AVR (serial, CAN, ethernet etc, but start with basic serial!).  That coms link needs a standard protocol and scope of actions, i guess equivalents of the old PEAK and POKE commands etc.  It could then allow easy calibration and Data Aquisition from the target device, no matter what code is running on that device.

 

The key will be to keep the code overhead on the target device low and very simple imo.  Simple RAM read and write would provide a lot of functionality for a very low overhead.

 

This is no small task, but would be a very useful addition to small embedded development i think?

 

What do people think?  Worth pursuing?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

max_torque_2008 wrote:
get data from my AVR,
What "data" are you talking about? Do you mean the non-volatile contents of the EEPROM or actually dynamic variable in RAM during run-time?

For the EEPROM stuff why can't you just read it out using the ISP programmer any time you need it? It's be easy to write some kind of decoder/interpreter for it as long as you know the struct layout of the data in the EEPROM.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm talking about being able to 'read' and 'modify' live RAM (as a start) during run time, to/from a serially connected pc or similar.

 

For example, imagine you want to tune a PID loop>  You need to be able to see/graoh/log the PID parameters, and change the setpoints in real time to make this an easy task.

 

If one writes a generic interface protocol and front end, and use an auto generated set up from the compiler output then it becomes a very powerful flexible tool!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

But if you were writing such code surely you would just include a UART interface and a menu/tweaking system in your design (perhaps just enabled during development with key parameters in modifiable EEPROM later?). Why would you need a specific "protocol" to achieve this. Surely you just do it on a per app basis:

Enter new values for P, I and D: 13, 23, 47

or whatever it is?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

max_torque_2008 wrote:
I'm talking about being able to 'read' and 'modify' live RAM (as a start) during run time, to/from a serially connected pc or similar.
For SAM with Atmel-ICE, EDBG, or mEDBG :

Data Visualizer

External Connection

Data Gateway Interface (DGI)

Code Profiling

configuration example

http://www.atmel.com/webdoc/GUID-F897CF19-8EAC-457A-BE11-86BDAC9B59CF/index.html?GUID-9D27AA1D-2FAF-4F33-BD7D-3210B883D31C

...

(bottom of the page)

The slider is now in control of the frame_comparator variable in the application code. Drag the slider, and notice that the LED blink frequency changes. Any change in the slider position will be sent to the target device through the debug interface, and a new value stored to the variable. At the same time the value is also read back from the target, and displayed on the segment display.

...

For AVR with Atmel-ICE, EDBG, or mEDBG :

Data Visualizer

Example Code snippets

Dashboard example code

http://www.atmel.com/webdoc/GUID-F897CF19-8EAC-457A-BE11-86BDAC9B59CF/index.html?GUID-710FB7D6-3BDA-483E-BC16-4951A08CC4AD

...

(at about 3/4)

The code samples the ADC continuously and sends the data over the SPI interface to the EDBG (Embedded Debugger) on the ATMEGA256RFR2 Xplained Pro board. The EDBG then sends the SPI data over DGI to the host computer. The ATmega256RFR2 ADC is 10 bit but only the lower 8 bits contain useful data in this example.

In addition the code sets up the CDC USART and sends the state of the night mode switch as a single byte. The received data on the CDC USART is parsed as a double value and is used as threshold for the night mode switch.

...

For ARM Cortex-M or Cortex-R :

Logo

QSPY: QSpyView User Interface

Poke Command

http://state-machine.com/qspy/qspyview_ui.html#qspyview_poke

The "Poke Address..." command allows you to write data to the specified address in Target's RAM ...

...

 


Quantum Leaps

QP™ Frameworks Feature Comparison

http://www.state-machine.com/products/index.html#QP-Comp

 

Edit : strike-thru (Atmel-ICE does not have a UART interface)

 

"Dare to be naïve." - Buckminster Fuller

Last Edited: Thu. Mar 9, 2017 - 02:29 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

max_torque_2008 wrote:
If one writes a generic interface protocol and ...

http://www.atmel.com/tools/atatmel-ice.aspx?tab=documents

DGILib User Guide
(file size: 263KB, 17 pages, revision A, updated: 09/2016)

DGILib is a Dynamic-Link Library (DLL) to help software applications communicate with Data Gateway Interface (DGI) devices. It handles the basic low-level USB communication and adds a level of buffering for minimizing the chance of overflows.
http://www.atmel.com/Images/Atmel-42771-DGILib_UserGuide.pdf

(page 3)

1. Description

DGILib is a Dynamic-Link Library (DLL) to help software applications communicate with Data Gateway Interface (DGI) devices.

...

DGILib handles the low-level USB communication and adds a level of buffering for minimizing the chance of overflows.

The library helps parse data streams of high complexity. The timestamp interface is parsed and split into separate buffers for each data source. The power interface is optionally parsed and calibrated using an auxiliary API.

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yes, plenty of existing systems exist but they tend to be either expensive, bloatware (for a small limited memory 8bitter), or have poor / non standard PC software.

 

Of course you can use a proprietry solution, i already do, you already do (the aformentioned usart to serial link and some terminal s/w on pc) but each and every one of those needs to be hard coded into your individual project, and then you need to configure the pc software to read / write those specific values.

 

The beauty of a pointer based PEAK/POKE type data exchange is that the pointer address is pulled from the .elf or .map file,  (ideally automatically parsed into an .xml ecu descriptor file) and then when ever you build a project you can easily remotely access any memory parameter. (the same as JTAG, and various other serial debugging tools, but should be device and compiler agnostic)

 

I work in automotive development and we spend our days optimising multiple embedded control systems using industry std tool sets like AutoSars and Etas INCA etc, and these (very expensive) tool sets bring a very flexible and seamless data integration to those projects (which simply couldn't work otherwise)  Now i'm not suggesting an AVR type afair needs to be anything like as complex but a basic, open source implementation could be used in the same fashion we already use open source bootloaders etc

 

Currently, if you are debugging or calibrating, if you haven't written the specific serial coms to say read a parameter you are stuck, you have to recode, recompile and reflash.  With a universal solution you can just add that parameter to your logging software (pulled from list generated from ecu descriptor file) and away you go!

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

max_torque_2008 wrote:
but each and every one of those needs to be hard coded into your individual project,
hardly - surely every one working with micros has a basic kind of UART lib and menu system for driving test?

 

As noted in other recent threads I prefer a one character per function style menu so for you PID I might use P to increase the "p" value and Q to decrease it or something. I guess I could implement "P nnnnn" but then I need to parse that which probably complicates things.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

max_torque_2008 wrote:
(the same as JTAG, and various other serial debugging tools, but should be device and compiler agnostic)
sigrok has JTAG, does not have AVR JTAG, but does have AVR ISP and AVR PDI (XMEGA)

AVR ISP can read and write EEPROM.

AVR PDI can read and write all XMEGA data space and via NVMC program space and EEPROM.

Likewise for tinyX AVR UPDI (one-wire PDI, tinyX has unified memory); there's Python for UPDI.

 

In-lieu of sigrok AVR ISP is LUFA AVRISP2 (ISP, TPI, PDI)

LUFA is FOSS and the AVRISP2 protocol is documented.

AVRDUDE has an interactive mode but it does not directly access XMEGA RAM; a possible is indirect via raw PDI instruction codes.

Might create an AVRDUDE driver specific to peek and poke, or, go direct via USB (vendor class, full speed)

 

PDI is via a USART with common transmit and receive (UPDI is via a UART)

PDI could go through a USB USART bridge.

Create Python PDI akin to Python UPDI.

 


http://sigrok.org/wiki/Protocol_decoders (avr_isp, avr_pdi)

http://sigrok.org/wiki/Protocol_decoder:Avr_pdi

https://www.sigrok.org/blog/libsigrokdecode-041-released

https://github.com/mraardvark/pyupdi (Python UPDI driver for programming "new" tinyAVR devices)

https://github.com/abcminiuser/lufa/blob/master/Projects/AVRISP-MKII/AVRISP-MKII.txt

http://download.savannah.gnu.org/releases/avrdude/avrdude-doc-6.3.pdf

(page 25)

3 Terminal Mode Operation
AVRDUDE has an interactive mode called terminal mode that is enabled by the `-t' option.
This mode allows one to enter interactive commands to display and modify the various device memories, perform a chip erase, display the device signature bytes and part parameters, and to send raw programming commands.

...

send b1 b2 b3 b4

Send raw instruction codes to the AVR device. ...

...

 

Edits : LUFA, Python PDI, AVRDUDE terminal mode, strikethru

 

"Dare to be naïve." - Buckminster Fuller

Last Edited: Sat. Mar 11, 2017 - 11:25 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

max_torque_2008 wrote:
Worth pursuing?
Yes

IIRC steve17 has posted a screen capture of a Windows app for XMEGA IO registers.

https://www.avrfreaks.net/forum/can-i-run-usb-using-pll-internal-rc2mhz#comment-2100926

 

 

"Dare to be naïve." - Buckminster Fuller

Last Edited: Fri. Mar 10, 2017 - 04:05 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

max_torque_2008 wrote:
Then i get back home and find that if i wish to get data from my AVR, or to change a parameter, i must hardcode and manually hash this.  I'm sure everyone uses there own methods, from serial debug in a terminal program to some flavour of stdout etc.
MPLAB X Data Monitor and Control Interface (DMCI) is available via MPLAB REAL-ICE for reading, or, a serial port to read or write (RTDM, added library and interrupts, PIC24H, PIC24EP, dsPIC33F)

 

http://microchipdeveloper.com/mplabx:dmci

via https://plus.google.com/+MicrochipTech/posts/5UmPumEZQCz

 

"Dare to be naïve." - Buckminster Fuller

Last Edited: Wed. Jan 17, 2018 - 10:43 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Who is the audience for this interface ? Yourself or the end user ?

 

In my previous employment (until redundancy in December) we were occasionally asked to provide the protocols for adjustment of offsets or indeed for complete re-calibration.

These commands existed (for factory use of course) but we/I never wanted to formalise them as an API for end users to use (or abuse).

 

Writing an common API Conforming to industry standards would have no doubt been a big undertaking, and without much benefit either.

 

Does your industry/market expect such an interface to be available on your devices ?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

max_torque_2008 wrote:
So, i'd be interested to hear from others what they do,

I have a device that needs to have a unique ID put into it by the end user, and it is stored in EEPROM.  There is a button on the side of the unit that when held and power applied places the unit in "Configuration mode" that you can access through a terminal program and a USB/485 converter.  The user enters the unique ID and the AVR places it in EEPROM, then reboots.

 

Simple

 

max_torque_2008 wrote:
The key will be to keep the code overhead on the target device low and very simple imo.

The above uses very little code...existing USART routines and a handful of assembler code.

 

Jim

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

My first micro, 70's vintage had a "monitor" program in rom that was used to load code, access, change and view the contents of ram in a live system, used a set of simple commands, L(oad), S(tore), D(ump), R(ead paper tape), W(rite paper tape), etc.   I have a similar monitor program I captured from a freak for the AVR (wonder where Bob is these days).

 

Jim

 

 

(Possum Lodge oath) Quando omni flunkus, moritati.

"I thought growing old would take longer"

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

N.Winterbottom wrote:
Who is the audience for this interface ?
Engineers - system characterization

Technicians - system diagnosis and repair

PCBA manufacturing technician - (possibly) final test

Logistics technician - system provisioning

N.Winterbottom wrote:
Yourself ...
Maybe

N.Winterbottom wrote:
... or the end user ?
For most operators, no but there are savvy operators (additional instruments via OBD)

N.Winterbottom wrote:
Does your industry/market expect such an interface to be available on your devices ?
No though it's typical for the engineers to add such.

For automotive, the answer is yes :

https://www.avrfreaks.net/forum/avr-calibration-daq-toolchain#comment-2107791

...

I work in automotive development and we spend our days optimising multiple embedded control systems using industry std tool sets like AutoSars and Etas INCA etc, and these (very expensive) tool sets bring a very flexible and seamless data integration to those projects (which simply couldn't work otherwise)  

...

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ki0bk wrote:
My first micro, 70's vintage had a "monitor" program in rom that was used to load code, access, change and view the contents of ram in a live system, ...
That can be implemented in a finite state machine that's attached to the memory arbiter in some MCU.

Such is in XMEGA AVR (Bus Matrix, Prog/Debug Controller, NVM Conroller) and tinyAVR 1-series (Bus Matrix, UPDI, NVMCTRL)

Likely will also be in 0-series AVR (mega, tiny)

ki0bk wrote:
... used a set of simple commands, L(oad), S(tore), D(ump), R(ead paper tape), W(rite paper tape), etc.
XMEGA PDI instructions :

LDS - Load Data, Direct Addressing

STS - Store Data, Direct Addressing

LD - Load Data, Indirect Addressing

ST - Store Data, Indirect Addressing

LDCS - Load Data, Control and Status

STCS - Store Data, Control and Status

REPEAT - Set Instruction Repeat Counter

KEY - Set Activation Key

 

tinyAVR 1-series UPDI instructions :

same as XMEGA PDI instructions

 

Alan might consider adding a monitor function to his PDI ISP and its GUI; IIRC, Alan is considering UPDI.

 


http://ww1.microchip.com/downloads/en/DeviceDoc/40001893B.pdf (datasheet, page 13, Block Diagram) via http://www.microchip.com/wwwproducts/en/attiny1617

https://www.avrfreaks.net/forum/megaavr-0-series

http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-8331-8-and-16-bit-AVR-Microcontroller-XMEGA-AU_Manual.pdf (page 401, 32.5.6 [PDI Controller] Instruction Set)

via http://www.microchip.com/wwwproducts/en/atxmega128a1u

(16KB tinyAVR 1-series datasheet, page 395, UPDI Instruction Set)

https://www.avrfreaks.net/forum/wts-standalone-pdiisp-auto-programmers

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Microchip Technology Inc

Microchip

Using the Real-time Data Monitor (RTDM) in the MPLAB X IDE

by Jeffrey O'Keefe (Microchip, staff engineer)

http://www.microchip.com/webinars.microchip.com/WebinarDetails.aspx?dDocName=en558828

...

Duration: 4 min

...

via https://plus.google.com/+MicrochipTech/posts/MrHset5s4cH

 

"Dare to be naïve." - Buckminster Fuller