Brief Update: AVR Debugger

Go To Last Post
38 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

Hello,

 

it's been a while since I've posted the last update on my project (almost a year ago), and I hope it is ok that I post again (if not, tell me ^^).

 

Anyways, much has changed, and here is what it looks like today:

 

 

 

So what exactly is new:

 

- Backstepping is possible

- Stepping over calls is possible although not yet published by the time of writing this

- UI has changed completely

- Disassembler is generating labels and comments

- Syntax Highlight

- You can simulate up to 4 files simultaneously

- Rewritten in C

 

Disassembler Mode:

You can use the disassembler as a standalone feature

 

Headless Mode:

This mode will step through the specified file without showing you the internals or the steps in-between. When stepping over illegal opcodes (or max. 5 seconds runtime) the program will terminate and print the latest internal memory-data in form of a json-file to stdout.

This might be useful when fuzzing AVR programs or when you just need a quick result.

 

And what is next:

 

- Graph view of the disassembled source code

- Support for peripherals like timers, etc.

- Completing the AVR instruction set

- and many more...

 

So that's it. I know there are some official tools for this purpose, and I am not trying to compete with them (and I can not ^^). I just enjoy the development :)

 

For those who are interested: https://github.com/Milo-D/MDX-As...

 

Thanks for reading and have a nice day :)

 

- Milo

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Wouldn't it have made more sense to have updated in the existing thread?

 

https://www.avrfreaks.net/commen...

 

 

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Update: 8-bit timer support.

Timer0 is now working (timer1, timer2 will follow). Currently only normal mode is available. Timer overflow (TOV0) plus interrupt works fine too. The overflow frequency seems to be accurate.

Supported Prescaler: 1/8/64/256/1024
External clock source: Currently not implemented.

CTC, PWM, etc. will follow. More details at https://github.com/Milo-D/MDX-Assembly-Debugger/releases

awneil wrote:

Wouldn't it have made more sense to have updated in the existing thread?

 

https://www.avrfreaks.net/comment/2822571#comment-2822571

 

 

Yeah you are right, but I've rewritten the whole program so the old thread contains a lot of outdatet information. I would delete the old post but I can not.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

You, as the op, can delete the text in your original post and leave a link to this thread.

Last Edited: Mon. Sep 28, 2020 - 06:51 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

RickB wrote:

You, as the op, can delete the text in your original post and leave a link to this thread.

Done. Thank you :)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Strike-through may be preferred as history is in a post instead of in a change set.

Web archival exists due to the typically limited lifetime of a web site.

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

Next time, edit your original post to make clear the thread is out dated, do not delete anything. Put a link to the new thread and all is good.

Then a moderator can lock it and all history is preserved.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

 

Update (v.0.4.3)

 

- All ~133 Instructions are now supported and can be disassembled

- EEPROM including: Write, Read, Erase, ERDY IRQs

- Timer0 Support (partial) including: CTC, Normal, IRQs, FOCs, ...

- Cycle accurate simulation of the program and the peripherals

- Jump Cycles (jc): Fast forwarding simulation based on cycle count

- New type of memory examination (xdb - examine data bitmap)

 

Long term goals:

 

- Support for other peripherals like WDT, USART, I2C, ADC, etc.

- Disassembler should reconstruct IO Registers based on the address

- Memory Maps and Hexdump

- Binary Patching

- More advanced interaction with disassembly (references, searching, ...)

- Graph View of disassembly

- UI Enhancements (more flexibility)

- support for other MCUs

- support for external hardware like LCDs, Keypads, ...

- many more...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 

Update (v.0.5.0)

 

- Performance: Reached 17 MHz execution speed. It is now possible to simulate default Arduino in realtime.

- Added: Analyzer Module.
    - The Analyzer Module consists of multiple (future) submodules.
    - Each submodule will be covering exactly one aspect of static code analysis.
    - Planned submodules:
        - ISR analysis (explore details about ISRs in your code)
        - SFR analysis (annotate addresses which target special function registers)
        - Function analysis (analyse function calls and maybe even their arguments/return-values)
        - many more...
    - The analyzer is a work in progress

- Refactor of toplevel structures
    - table_t class is now only for user-defined entries like breakpoints, etc.
    - system_t class does not contain table_t class anymore.
    - report_t class is the report of the analyzer after decoding and disassembling hexfile.

- Displaying hexadecimal addresses instead of line numbers in sidepanel.
- Table commands (like break, unbreak) are now requiring a hexadecimal address

- Added: Diagram of the analyzer dataflow
- Added: New screenshot

- DebugWindow Refactor (II)
    - panel properties are now calculated seperately in <g,s,f,d,e,o,r> pnl_ctor
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I did wonder where you were going with this project - but:

 

- Performance: Reached 17 MHz execution speed. It is now possible to simulate default Arduino in realtime.

 

Now that is a notable achievement. Congratulations.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thank you for your feedback :) Next step would be to reach 20 MHz and reduce the total memory usage.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Is it your plan to do all AVR's or only 328 (and 16 8 that more or less is a subset) ? (I read the comment that DES need work, and that instruction is not in those AVR's)

 

The 17 MHz is that with all HW running in the background ? (timers, UART WDT .........).

 

For the instructions only it should take something like 20 clk (modern x86, perhaps less if the hole AVR is inside the cache) for each instruction (actually AVR clk), so that would give about 200 MHz on a modern PC.

Does it really take about 10 times more for all the debugging history ? 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I am planning to implement the m328(p) first, then its family (16, 8). Implementing other AVRs might take a while, but maybe I will get some help ;)

17 MHz without using peripherals, but they do not affect execution speed too much. Maybe I could test the execution speed while having peripherals activated.

So yeah, there is definetely room for improvement.

Only 20 Cycles to simulate an instruction could be a tough challenge.

The virtual system is structured like a tree of submodules. So a system access has to be redirected to its corresponding submodule.

For example writing to an IO register triggers sys_write_data(...) -> data_write(...) -> io_write(...). Then the access information (whether it was a write or read access) has to be stored (so that it can be colorized later).

So there is unnecessary overhead but this helps keeping the codebase clean and structured.

I have already some ideas to boost the execution speed a little bit more. But for now there are other spots I want to improve and work on.

Thank you for your feedback :)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 2

I looked through your code a bit.  Nice.  I looks like you have data space and register file separated.  That is good.  I wrote a CPU simulator and had to go back and rework for the new 0-series.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

#14 Yes I have the same problem, a lot of my "real" code (ASM) can't run on 0-series (actually from x-mega and on), that was not a problem on bit chips just some waste (bloated code) but not good on small chips, the only explanation I have heard was that it was because of the DMA on x-mega's but yes it's a waste on all the never chips with no DMA :( 

 

About 20 clk for each AVR clk should not be a problem if coded for speed, (I made some simulator code in delphi long time ago 10+ years, and that was about the speed, and IPC on a PC has gone up).

I was also started on a AVR emulator running on an AVR, (so it can load it's own code, or run from external memory), and that was also about 20 clk (or a bit less) as I remember when it emulated from it's own flash, some faster and some slower.

A long time ago I remember that one of the C64 emulators ran about 10 times faster than a real C64 (it had a mode to run real speed), and it had to emulate all the graphic aswell.   

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks :) Although having the GPR (and SREG) seperated from the Dataspace forces me to take an extra branch in case a read/write operation targets mapped GPR (or SREG) ^^

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I am quiet sure that a much higher execution speed is possible. Hopefully I don't have to mess with the code structure too much. I am always trying to keep a balance between performance and architecture.

One thing I could do to improve the performance is to preprocess opcodes and extract every operand and store it in a struct. Currently I am extracting operands whenever executing an instruction even if it was already executed before. So that might boost the performance too.

Offtopic: Currently I am working on some submodules for the analyzer. Every submodule should cover exactly one aspect of static code analysis, for example a submodule that analyzes labels and its callers/callees. If you guys have some more ideas regarding future submodules, please let me know. I would be happy to work on them :)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

This GUI got me thinking about a related matter... Did you know the linux toolchain command line debugger has an avr simulator built in?, it is command line, and not really documented, but I have played around with it and got simple things done with it. A gui for that would be nice, it supports the new seriesTinyAvr1/0... but someone has to either reverse engineer it to figure out its capabilities or get the source from microchip (link is broken on their website for source).
Another needed tool, the ability to read opcodes from an isr and calculate the cycles that isr can take, worst and best case scenario. I feel like someone (or I) need to write a tool for that. I find myself always doing that calculation manually.
And lastly a tool to calc max stack usage by analyzing the lss or elf file. ezstack is an older tool that exists but sometimes has trouble newer avr. Maybe another approach to the problem is in order.

Last Edited: Tue. Jan 5, 2021 - 04:19 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

GUI is the single thing in my project I don't like to work on ^^ so I am really not a fan of UI development :)

Do you mean gdb ? Isn't it core only simulation (without peripherals) ?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I hear you, GUI development is not my fav either. The gdb might only be core simulation, I've only done a little with it. I edited my previous post with a couple tool ideas, things I probably will never have time to tackle, but seem useful.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Really interesting suggestions. I really like them ^^ Especially the idea to analyze ISR cycles. I will add your ideas to my "list".

Thank you for your suggestions ;)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I used to used a nice GUI front end for gdb .... Ah, this is it: DDD.  (hmm, looks like it has not been touched for a while -- maybe it still works)

 

Also, there is a protocol for hooking up simulators to gdb.  It's in the appendix of the gdb manual I believe.

Last Edited: Tue. Jan 5, 2021 - 01:32 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

MattRW wrote:
Ah, this is it: DDD
When I first started to use Linux the first GUI interface for GDB I used was DDD. I still have fond memories. Sure the interface is a bit "clunky" but it was pretty easy to get most things done (and unlike a lot of modern IDE it didn't have 5,000 additional functions to dig through to  find the core stuff you actually wanted/needed.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 

New Feature: Opcode view in disassembly. It is now possible to see opcodes to each instruction disassembled by MDX.

 

Update (v.0.5.1)

- Systemprinter refactored
    - Rename sytemprinter.* to debugprinter.*
    - Seperated printer functionality into six files:
        - gpnl_printer.* (printer for the gpr panel)
        - spnl_printer.* (printer for the sreg panel)
        - fpnl_printer.* (printer for the flash panel)
        - dpnl_printer.* (printer for the data panel)
        - epnl_printer.* (printer for the eeprom panel)
        - rpnl_printer.* (printer for the right panel)
    - functionality stayed the same

- Added credits for the architecture diagram.
- Fixed comments for push/pop mnemonic.

 

Update (v.0.5.2)

- This update reduces heap allocations for an overall better memory performance
- Further improvements regarding memory consumption are coming soon

- Now using sstream in mnemonics generator and debugprinter
    - avoiding unnecessary heap allocations

- Added: stringstream collection (sstream.c, sstream.h)
    - will be replacing queue collection (queue.c, queue.h)

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi Milo,

 

Since you are going for speed, would you mind running a benchmark on your machine, assuming you are developing on Linux?  I think your simulator is faster than mine.  The following sysbench on my machine takes 10.0s.  I'm getting like 7 MHz on my simulator, but I run with an approach different to most simulators.

 

sysbench --test=cpu --cpu-max-prime=20000 run

 

The approach I use is to use time as the independent variable, versus instruction count.  As time goes it generates oscillator events, which generate cpu-exec events.  I tried running with "-pg" arguments to gcc (compile and load) to trace down the bottlenecks but that isn't working right now.  Apparently, it's since I'm dynamically loading in the MCU device.  I am sticking with the time-based sim, since I want to integrate spice and run multiple MCUs with oscillators that drift (e.g., my oscillators have a parameter for Allan deviation).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Sure, these are the results of my benchmark

 

 

That is an interesting approach. I think a timebased simulation is possibly more accurate especially when it comes to simulating peripherals. Do you have a link to your simulator ? :)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

 

Edit: It seems like I've posted this log with my second account, maybe someone can delete this post, so that I can re-post this with my update-log only account (milo-d). Thanks.

 

# v.0.5.4

- Added first Submodule for the Analyzer: Static SFR Analysis
  - This submodule explores special function registers in disassembly
  - once an address to a special function register has been detected
    by the module, an annotation will be created with its corresponding name

  - example: out 0x3f, r1 ; SREG

- Minor refactor of timer8.
- New Debugcommand: xeb - examine EEPROM as bitmap
- Cleanup: unused Headers, unused variables
Last Edited: Thu. Jan 28, 2021 - 10:27 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0


If the red text is something you are adding:

then perhaps change " R21[0] = 0...."  to "R21[0] == 0....."

 

just keep in line with C annotation as now it at first glance looks like you are setting a bit to 0 instead of actually checking if it is 0

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yep you're right, thanks. ' ==' might be better here.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I like the SFR annotation, very handy to have this information in the same view.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The layout reminds me of some PLC programming I did in the DOS days (on a FESTO 405 in statement list it was a Z80 computer).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Update Log (v.0.5.5)

 

- Separated the simulation engine (engine/) from the GUI (debugger/). These two main components are now in different directories.

 

This is the only change I've made. I will try to create a small statically linked library, so that the engine can be used in different projects. Maybe with bindings for higher level languages, if I find the time for it.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thank you ccrause ^^

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Update (v.0.6.0)

 

- Massive reduction of heap usage (Issue #47)

 
- Added: New Analyzer Submodule: Label-Analyzer
  - This module analyzes jumps, branches and calls in order to
    find possible labels in disassembly.
  - it also discovers callers of labels
 
- Generated Labels are now sorted (ascending)
- Fixed Bug in Disassembler (Issue #46)

- Performance Enhancement (runtime) of Disassembler and Analyzer

- Added new example program (test/m328p/misc/delay.hex)
- removed include path
- renamed function sys_read_instr to sys_read_progmem

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Update (v.0.7.0)

 

-> Reached 20 MHz simulation speed.

 

-> I don't like to work on user interfaces so I kicked out the GUI. MDX is now a library for static and dynamic analysis of AVR binaries, called libvmcu. I want to focus purely on the engine.

 

The library can be statically linked and its potential usecases are:

 

    - creating (regression) tests for embedded systems in order to validate functionalities

    - binary analysis of AVR programs

    - other people can create tools out of it, like simulators (probably even with a much better UI than MDX)

 

Library Features:

 

    - static analysis (find potential labels and their jumpers/callers, find SFRs, filter instructions, analyze operands, etc.)

    - dynamic analysis (cycle accurate realtime simulation of the ATmega328 family including peripherals and backstepping)

    - decoding intel hex (more formats are coming soon), disassembling, decomposing and analyzing

    - easy to use

 

Future Plans:

 

    - codeflow graph

    - decompiler

    - cycle analysis, ISR analysis, etc.

    - bindings for python

 

I would appreciate feedback, since this is the only way to develop an user-friendly library ^^

 

New link: https://github.com/Milo-D/libvmc...

 

What happens to MDX ?

 

MDX will be moved to another repository. It will be open for PRs, but personally I have no further interest in developing UIs (especially not with ncurses lol).

 

Last Edited: Sat. Feb 20, 2021 - 07:04 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

cycle analysis, ISR analysis via standalone command line on an elf or hex would make for a very useful tool indeed. Outputting text analysis to command line. Maybe even a minimum and max possible cycles, and infinite cycle detection (forever loop scenarios). I have an isr that can do into a forever loop to force watchdog reset, but also has min and max possible cycles under other conditions.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Update (v.0.7.1)

 

-> Added Java Bindings (thanks to pointbazaar) for libvmcu. Java bindings support

 

    - microcontroller simulation

    - static analysis of binary files (intel hex)

 

-> Added an example of a possible firmware unit test with libvmcu (driver/led/)

 

    - driver/led/ledtest.c is using libvmcu to verify interrupt frequency of timer0 on the ATmega328P

#define PORTB     0x0025
#define PB5       0x05

#define TESTFILE  "../../driver/led/led.hex"

#define bit(v, b) ((v & (1 << b)) >> b)

int main(const int argc, const char **argv) {

    uint8_t led;

    /* ignoring checks for this example */
    vmcu_report_t *report = vmcu_analyze_ihex(TESTFILE);
    vmcu_system_t *sys    = vmcu_system_ctor(report);

    do {

        vmcu_system_step(sys);
        led = vmcu_system_read_data(sys, PORTB);

    } while(bit(led, PB5) == 0x00);

    const double f    = 16000000U;
    const double c    = sys->cycles;
    const double time = (c / f);

    printf("Time between LED toggle: %lf [s]\n", time);
    assert((0.95 <= time) && (time <= 1.05));

    vmcu_report_dtor(report);
    vmcu_system_dtor(sys);

    return EXIT_SUCCESS;
}
// Time between LED toggle: 1.000021 [s]
// Expected Result: 0.95 [s] <= time <= 1.05 [s]

 

-> Added option to build libvmcu as a dynamic library

Last Edited: Mon. Feb 22, 2021 - 02:42 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

A couple of suggestions:

1) show symbol types (global, local, or weak)

2) show symbol names for lds/sts addresses

3) decode multi-byte register loads i.e.:

 ldi r24, 0x39

 ldi r25, 0x30    ; r25:24 = 12345

 

I have no special talents.  I am only passionately curious. - Albert Einstein