Problems with avrdude and ATxmega256A3

Go To Last Post
50 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi,
 

This is my first post here, and I bust in with a question right away. My sincere apologies for that.
 

I'm currently trying to port a firmware installer from Windows to Mac OS X.
The device is based on an ATxmega256A3, connected to serial via USB and FT232R UART.

The windows version ist based on AVROSP.exe, but since there is no Mac version of it, I've been trying for 2 days now to do this using AVRDUDE.
Unfortunately, still no joy.

I'm sorry if this is a n00b question, while I secretly hope it is and someone can give me a hint, as I've done a lot of search on it but to no avail so far.
It has been holding me up for days now.

This is what the XML file for AVROSP looks like:
 

<AVRPART>
<MEMORY>
<PROG_FLASH>270336</PROG_FLASH>
<EEPROM>4096</EEPROM>
<BOOT_CONFIG>
<PAGESIZE>512</PAGESIZE>
</BOOT_CONFIG>
</MEMORY>
<FUSE></FUSE>
<ADMIN>
<SIGNATURE><ADDR000>$1E</ADDR000><ADDR001>$98</ADDR001><ADDR002>$42</ADDR002></SIGNATURE>
</ADMIN>
</AVRPART>

As far as I can see, it matches exactly the definitions in avrdude.conf.
When I start AVROSP.exe from the windows command line:

 

mode COM5 Data=8 Parity=n Baud=38400 DTR=OFF RTS=OFF
AVROSP -dATxmega256A3 -e -if"updatefile.hex" -pf

everything works as expected.

However, AVRDUDE (6.3) does this:
 

./avrdude -p x256a3 -C ./avrdude.conf -c avr911 -P /dev/cu.usbserial-A504ZISZ -b 38400 -U flash:w:updatefile.hex -e -v
avrdude: Version 6.3, compiled on Sep 12 2016 at 17:22:25
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "./avrdude.conf"
         User configuration file is "/Users/Tozzi/.avrduderc"
         User configuration file does not exist or is not a regular file, skipping

         Using Port                    : /dev/cu.usbserial-A504ZISZ
         Using Programmer              : avr911
         Overriding Baud Rate          : 38400
         AVR Part                      : ATxmega256A3
         Chip Erase delay              : 0 us
         PAGEL                         : P00
         BS2                           : P00
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 0
         StabDelay                     : 0
         CmdexeDelay                   : 0
         SyncLoops                     : 0
         ByteDelay                     : 0
         PollIndex                     : 0
         PollValue                     : 0x00
         Memory Detail                 :

                                  Block Poll               Page                       Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00
           prodsig        0     0     0    0 no         50   50      0     0     0 0x00 0x00
           fuse1          0     0     0    0 no          1    0      0     0     0 0x00 0x00
           fuse2          0     0     0    0 no          1    0      0     0     0 0x00 0x00
           fuse4          0     0     0    0 no          1    0      0     0     0 0x00 0x00
           fuse5          0     0     0    0 no          1    0      0     0     0 0x00 0x00
           lock           0     0     0    0 no          1    0      0     0     0 0x00 0x00
           data           0     0     0    0 no          0    0      0     0     0 0x00 0x00
           eeprom         0     0     0    0 no       4096   32      0     0     0 0x00 0x00
           application    0     0     0    0 no     262144  512      0     0     0 0x00 0x00
           apptable       0     0     0    0 no       8192  512      0     0     0 0x00 0x00
           boot           0     0     0    0 no       8192  512      0     0     0 0x00 0x00
           flash          0     0     0    0 no     270336  512      0     0     0 0x00 0x00
           usersig        0     0     0    0 no        512  512      0     0     0 0x00 0x00
           fuse0          0     0     0    0 no          1    0      0     0     0 0x00 0x00

         Programmer Type : butterfly
         Description     : Atmel AppNote AVR911 AVROSP

Connecting to programmer: .
Found programmer: Id = "AVRBOOT"; type = S
    Software Version = 1.7; No Hardware Version given.
Programmer supports auto addr increment.
Programmer supports buffered memory access with buffersize=128 bytes.

Programmer supports the following devices:
    Device code: 0xfa

avrdude: devcode selected: 0xfffffffa
avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.01s

avrdude: Device signature = 0x1e9842 (probably x256a3u)
avrdude: erasing chip
avrdude: reading input file "updatefile.hex"
avrdude: input file updatefile.hex auto detected as Intel Hex
avrdude: writing flash (142442 bytes):

Writing | ################################################## | 100% 57.92s

avrdude: 142442 bytes of flash written
avrdude: verifying flash memory against updatefile.hex:
avrdude: load data flash data from input file updatefile.hex:
avrdude: input file updatefile.hex auto detected as Intel Hex
avrdude: input file updatefile.hex contains 142442 bytes
avrdude: reading on-chip flash data:

Reading | ################################################## | 100% 57.95s

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0000
         0x00 != 0x0c
avrdude: verification error; content mismatch

avrdude done.  Thank you.

 

The Update is much faster than with AVROSP.exe, but it doesn't work. The device won't boot anymore until I reflash it under Windows, using AVROSP.exe.
Verify always fails. I've tried slowing it down using the -i option, but that did not help either.

I really hope it's just a stupid mistake on my end and someone can help me with this.

 

Best regards,
Stephan

This topic has a solution.

Greetings from Munich, Germany
Stephan

Last Edited: Tue. May 23, 2017 - 10:17 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 

I don't know.  It appears there is something wrong with avrdude, or the way you are using it.

 

You could try a few things.  I would try the verify only thing to see if it can read back the flash memory you put on there with avrosp.  I do that by replacing the :w: with :v:.

You could try a different version of avrdude.  I've been using version 6.1 and I think I've also used 6.2.  I don't know if I used 6.3.

 

Different versions might need slightly different options.  I've never used the -v option.  I don't know what that does.   I don't use the -e option either, but maybe that is required for your avr911 bootloader.  I use a modified version.

 

I guess it's possible there is an avrdude bug when writing more than 64k bytes of flash.  I've only used less than 64k.

 

I haven't used the serial connection for a long time either.   

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

A verify-only with the same hex file fails, too.

The -v option is just for verbosity...
If i omit -e, writing fails. (***failed;).

I'm running out of ideas. :(
 

EDIT:
I've tried with avrdude 5.9, and with that version, I get the following error:
avrdude: ERROR: address 0x100010 out of range at line 4098 of updatefile.hex

Greetings from Munich, Germany
Stephan

Last Edited: Wed. May 17, 2017 - 06:21 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Tozzi wrote:

I've tried with avrdude 5.9, and with that version, I get the following error:
avrdude: ERROR: address 0x100010 out of range at line 4098 of updatefile.hex

Avrdude 5.9 is probably fairly old, so I don't know if that means anything if you use a newer edition, but it might.

 

I would try programming the flash with a hex file that gives less than 64k.  You might be able to just remove the last half of your hex file.  I'd make a copy first.  The hex file is a text file you can look at and edit with any text editor.  You should have it end with the "end of file" entry.  You'll see it at the end of the hex file. 

 

It will look like this:

:00000001FF

Last Edited: Wed. May 17, 2017 - 06:54 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Problem is, I need to get this to work with the hex file I was provided with in the first place.

I can't just cut it in half...
But I'm beginning to think the problem indeed lies within the hex file.

However, I do not have access to the source code of the firmware I want to flash (yet).
I'm just trying to get avrdude to do the same thing avrosp.exe does...
After all, all I want to do is port a firmware updater from Windows to Mac OS X.

Interestingly, I'm also not able to verify with avrosp.exe. It writes correctly, but won't verify, so my "black box" is apparently write-only.
Also, it takes about 6 minutes to complete writing, whereas avrdude is done after just 59 seconds.

Greetings from Munich, Germany
Stephan

Last Edited: Wed. May 17, 2017 - 07:15 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I understand that you need the whole file.  But if it works with a small hex file, that tells you where the problem lies, so you could at least stop looking for the problem.

 

I think most people use DFU these days.  That can be a bit of a headache too.   Can you make a USB connection between the xmega and the computer?

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The fact that avrosp can't verify is a bad sign.  Do you know it programs the flash correctly?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yes, that's the weird part.  avrosp.exe does indeed program the flash correctly, while avrdude won't.
Problem is, it's kind of a black box i'm dealing with here.
It's a device connected to a proprietary WiFi Module, which is required to do the firmware update and features a USB port (FT232R). 
AVROSP.exe communicates successfully through a standard serial device (COM5:) however.
Also, avrdude 6.3 will do its thing without complaining, only the device I'm trying to flash is dead after that and can only be rescued by re-flashing it with AVROSP.exe.

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Maybe there is something peculiar about the particular bootloader you are using.

 

I'm using an allegedly avr911 bootloader.  Avrdude 6.1 seems to be able to work okay even with more than 64k.  My programs are only around 53k but I tinkered with a hex file and got avrdude to load 69k.  I've modified mine to use USB CDC and it's a hell of a lot faster than yours.  It could flash your hex file in about 4 seconds.  DFU is faster yet.

 

The bootloader I modified was originally for a serial port.  You can find the original on github and I have it around here somewhere.

https://github.com/bandtank/Xmega_Bootloader

 

 

Last Edited: Wed. May 17, 2017 - 10:03 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yes, there must be something peculiar about it...
I wish I could send you the hex file in question, but it's under NDA.
I appreciate your help a lot, but I guess we are getting nowhere with this without me having access to the source code that has produced this dreaded .hex file...

I'll try 6.1 and then give up for now... too many unknown variables... wasted 4 days now with this in trial/error mode.

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Tozzi wrote:

 too many unknown variables... wasted 4 days now with this in trial/error mode.

I know the feeling.  I just spent 5 days in trial and error mode migrating my PC programs from Visual Studio 2010 to Visual Studio 2017.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Maybe it's the way the Intel hex file addresses go above 16 bits.  The format has room for 2 bytes.  Above that, there are various ways to specify the higher order bits.  The whole thing is somewhat a mess, because when the format was invented, no addresses went that high.

 

I tried the "Extended segment address", code 02, and that didn't work.  It's possible I entered it wrong.  Then I tried "Extended linear address", code 04, and it worked.

 

My line looks like this:

:020000040001F9

 

If you tinker with it, keep in mind the last two bytes are the check sum.  Fortunately my editor has a command to set it.

 

https://en.wikipedia.org/wiki/Intel_HEX
 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

AVR911 is a binary protocol so the hex decode is surely done within avrdude itself? It seems unlikely it would get the >64K addressing wrong?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I was thinking avrdude might get the >64k addresses wrong in the Intel Hex file.  The file format is as perverted as the Intel chips are.

 

I don't think the avr911 bootloaders I've seen handle >64k at all.  The reason they work in this case is the avr911 addresses, just like all the AVR bootloader addresses I've seen, are word addresses.  Or you could look at it as byte addresses right shifted one bit.

Last Edited: Thu. May 18, 2017 - 06:37 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks for all your help!
My guess is, there's something weird about this device (which basically still is a black box to me) and avrdude cannot handle this peculiarity (yet).

Trying to find the needle in the haystack in the dark won't get me any closer to my goal however, so I'll ask the client to send me the firmware sources plus some more tech info about the device and meanwhile I'll debug this issue low-level.
Good thing both avrdude and avrosp come with source code.
Something I probably should have done in the first place, and it's something I was trying to avoid, but who could have known it's going to be this tricky.
I was hoping for a hint regarding some stupid mistake of mine, but it looks like we're all just making wild guesses here after all.

I mean, a tool written back in 2004 can do the trick, and avrdude 6.3 can't? This in itself is something I find strange.
But I think I've tried every possible combination of parameters now that remotely makes sense.
Most of you guys will likely compile their own firmware, which is why probably none of you have run into this issue.

There's a good chance the hex file itself is the culprit, but seeing the differences in timing, it might also be buffersizes, wait cycles, or any combination of that...

I'll post here once I found out.
Meanwhile, I still appreciate any ideas where I should focus on or if there's something I should try.

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You might try doing a search if you haven't already.

I search for "avr911" and got this plus many other pages that might help:

http://www.avrfreaks.net/forum/a...

 

Just guessing if it will be good information since I know very little about avr911 or avrdude.

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yea, that one did come up in one of my searches, and it did point me towards this forum.
I've bookmarked it and will go through it in-depth next week. So far, no eye-openers there...
Will need to debug this on bits and bytes level...

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'd still like to see the line where the addresses wrap from FFFx to 000x.  There must be a line there that sets the seventeenth address bit to 1.  There are various ways to do it.

 

I think it would be a simple search for a text editor.  Probably you could just look for :02 or :04.  The next four bytes are the address which isn't used and is usually set to 0000.

 

I'm also curious to see how many payload bytes per line.  Gnu uses 16 so the data lines, which are almost all of the lines, start with :10.  10 being hex for 16.  When I use Atmel's tools, a programmer/debugger and PDI, to pull the flash from a chip and save it as a hex file, it uses 32 bytes per line.  So those lines start with :20.  Very thoughtful eh?  It makes it so easy to compare the data pulled from flash with the hex file produced with GCC, not.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Steve17, you might be right onto something here...

:04 doesn't occur.

:02 occurs twice, lines 4097 and 8194.

Lines 4096-4098:
 

:10FFF000ED81FE813196E49180E090E0A0E0B0E8F0
:020000021000EC
:100000008093B1389093B238A093B338B093B4389A

Lines 8192-8196:

:10FFE000E092F33AF092F43A80912B2A90912C2AE5
:10FFF00003960E9445D8E0912B2AF0912C2A87A1E4
:020000022000DC
:1000000090A580932B2A90932C2AE091292AF09195
:100010002A2A0F5F202F30E083A194A18217930733

So, I should try and convert this to "Extended linear address"?

What would be the best way to do that?

 

Greetings from Munich, Germany
Stephan

Last Edited: Fri. May 19, 2017 - 01:30 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Tozzi wrote:

So, I should try and convert this to "Extended linear address"?

I don't think that will work.  We can try it.  I can give you the line to put in there.  Unfortunately I don't think that's the problem.

 

For one thing, I tried that form of "set address command" here and it worked.

 

But there is a more serious problem, the avr911 protocol.  The first "set address command" adds a 1 in front of the address.  So the next line with address zero would be actually 10000.  Okay that should work because the address is right shifted one bit when sending to the bootloader in the Xmega.  Atmel sez it's a word address.

 

The second "set address command" puts a 2 in front of the address so address 0000 is really 20000.  Now we have a problem.  Right shifting that results in 10000 which won't fit in a 16 bit address which avr911 and probably every other AVR bootloader uses.  What is needed is a "set high address bits" command avrdude would send to the bootloader and which the bootloader would recognize and stick the included high address bits in front of the usual 16 bit address.  So the questions are, does avrdude send such a command and does your bootloader recognize it.   I'm guessing avrdude sends it but avr911 doesn't know what to do with it.

 

When I was working on my avr911 bootloader I realized it had no such command, and no way to handle more than 128k of flash.    I was curious and looked through my collection of bootloaders and I found such a command.  I think it was in a bootloader called Xboot. 

 

So if avrdude sends the command, a fairly simple fix to the avr911 bootloader should fix the problem.

 

When avrdude said 142442 bytes were written, that should have told me there could be trouble, but my brain failed me.  In hex that's 0x22c6a. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

Oh wait a minute.  Your avrosp works, so somehow your avr911 can handle it.  That is, if you are sure the avrosp really writes to flash above 128k.

Last Edited: Fri. May 19, 2017 - 03:23 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0000
         0x00 != 0x0c
avrdude: verification error; content mismatch

avrdude done.  Thank you.

 

I haven't used avrdude before. Is verify complaining that it sees 0x00 where is should actually be 0x0c?

It seems like it's failing to verify quickly at the very first flash location at 0x0000 and it's expecting to see 0x0c which is a 'typical' jump value for the reset vector.

Can you show your ihex file at the beginning of the file at location 0x0000 to see what byte is there. Is it 0x0c or is your program doing something different with the vector jump table?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

These are the first 2 lines:

:100000000C9456760C9476760C9476760C947676E0
:100010000C9476760C9476760C9476760C947676B0

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That looks right, I guess for some reason it's writing 0x00 to that first location instead of the correct 0x0c.

It would be 0xff if the location was erased without a write.

So it's writing incorrect bytes.

Maybe check the baud rate, usb adapter and avrdude command line parameters?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Unfortunately, there's no way to tell, since read (verify) always returns 0x00 regardless.
I did sniff out the serial communication by using "Advanced Serial Port Monitor" under Windows, and found out some differences.

This is what happens with AVRDude (first lines, including system events):
 

 0x00000 43 4F 4D 20 69 73 74 20 6F 66 66 65 6E 20 53 63  COM ist offen Sc 
 0x00010 68 6C 61 6E 67 65 6E 67 72 F6 DF 65 20 49 6E 2F  hlangengröße In/ 
 0x00020 4F 75 74 20 31 30 32 34 2F 31 30 32 34 20 42 61  Out 1024/1024 Ba 
 0x00030 75 64 2D 52 61 74 65 20 33 38 34 30 30 20 52 54  ud-Rate 38400 RT 
 0x00040 53 20 61 75 73 20 44 54 52 20 61 75 73 20 44 61  S aus DTR aus Da 
 0x00050 74 61 20 42 69 74 73 3D 38 2C 20 53 74 6F 70 20  ta Bits=8, Stop  
 0x00060 42 69 74 73 3D 31 2C 20 50 61 72 69 74 79 3D 4E  Bits=1, Parity=N 
 0x00070 6F 6E 65 20 5A 65 69 63 68 65 6E 20 73 65 74 7A  one Zeichen setz 
 0x00080 65 6E 3A 20 45 6F 66 3D 30 78 30 30 2C 20 45 72  en: Eof=0x00, Er 
 0x00090 72 6F 72 3D 30 78 30 30 2C 20 42 72 65 61 6B 3D  ror=0x00, Break= 
 0x000A0 30 78 30 30 2C 20 45 76 65 6E 74 3D 30 78 30 30  0x00, Event=0x00 
 0x000B0 2C 20 58 6F 6E 3D 30 78 30 30 2C 20 58 6F 66 66  , Xon=0x00, Xoff 
 0x000C0 3D 30 78 30 30 20 48 61 6E 64 66 6C 6F 77 3A 20  =0x00 Handflow:  
 0x000D0 43 6F 6E 74 72 6F 6C 48 61 6E 64 53 68 61 6B 65  ControlHandShake 
 0x000E0 3D 28 29 2C 20 46 6C 6F 77 52 65 70 6C 61 63 65  =(), FlowReplace 
 0x000F0 3D 28 29 2C 20 58 6F 6E 4C 69 6D 69 74 3D 30 2C  =(), XonLimit=0, 
 0x00100 20 58 6F 66 66 4C 69 6D 69 74 3D 30 20 1B 53 41   XoffLimit=0 .SA 
 0x00110 56 52 42 4F 4F 54 56 31 37 76 3F 70 53 61 59 62  VRBOOTV17v?pSaYb 
 0x00120 59 74 FA 54 FA 0D 50 0D 73 42 98 1E 65 0D 41 00  YtúTú.P.sB˜.e.A. 
 0x00130 00 0D 42 00 80 46 0C 94 56 76 0C 94 76 76 0C 94  ..B.€F.”Vv.”vv.” 
 0x00140 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x00150 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x00160 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x00170 6A 77 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  jw.”vv.”vv.”vv.” 
 0x00180 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x00190 76 76 0C 94 76 76 0C 94 60 77 0C 94 76 76 0C 94  vv.”vv.”`w.”vv.” 
 0x001A0 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x001B0 76 76 0C 94 76 76 0D 42 00 80 46 0C 94 76 76 0C  vv.”vv.B.€F.”vv. 
 0x001C0 94 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C  ”vv.”vv.”vv.”vv. 
 0x001D0 94 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C  ”vv.”vv.”vv.”vv. 
 0x001E0 94 76 76 0C 94 76 76 0C 94 1D 77 0C 94 76 76 0C  ”vv.”vv.”.w.”vv. 
 0x001F0 94 76 76 0C 94 9F 76 0C 94 76 76 0C 94 76 76 0C  ”vv.”Ÿv.”vv.”vv. 

And here's AVROSP:
 

 0x00000 43 4F 4D 20 69 73 74 20 6F 66 66 65 6E 20 1B 1B  COM ist offen .. 
 0x00010 1B 1B 1B 1B 1B 1B 1B 1B 53 4C 65 65 72 65 6E 20  ........SLeeren  
 0x00020 64 65 72 20 73 65 72 69 65 6C 6C 65 6E 20 53 63  der seriellen Sc 
 0x00030 68 6E 69 74 74 73 74 65 6C 6C 65 3A 20 54 58 43  hnittstelle: TXC 
 0x00040 4C 45 41 52 20 41 56 52 42 4F 4F 54 73 4C 65 65  LEAR AVRBOOTsLee 
 0x00050 72 65 6E 20 64 65 72 20 73 65 72 69 65 6C 6C 65  ren der serielle 
 0x00060 6E 20 53 63 68 6E 69 74 74 73 74 65 6C 6C 65 3A  n Schnittstelle: 
 0x00070 20 54 58 43 4C 45 41 52 20 42 98 1E 65 4C 65 65   TXCLEAR B˜.eLee 
 0x00080 72 65 6E 20 64 65 72 20 73 65 72 69 65 6C 6C 65  ren der serielle 
 0x00090 6E 20 53 63 68 6E 69 74 74 73 74 65 6C 6C 65 3A  n Schnittstelle: 
 0x000A0 20 54 58 43 4C 45 41 52 20 0D 62 4C 65 65 72 65   TXCLEAR .bLeere 
 0x000B0 6E 20 64 65 72 20 73 65 72 69 65 6C 6C 65 6E 20  n der seriellen  
 0x000C0 53 63 68 6E 69 74 74 73 74 65 6C 6C 65 3A 20 54  Schnittstelle: T 
 0x000D0 58 43 4C 45 41 52 20 59 02 00 41 00 00 4C 65 65  XCLEAR Y..A..Lee 
 0x000E0 72 65 6E 20 64 65 72 20 73 65 72 69 65 6C 6C 65  ren der serielle 
 0x000F0 6E 20 53 63 68 6E 69 74 74 73 74 65 6C 6C 65 3A  n Schnittstelle: 
 0x00100 20 54 58 43 4C 45 41 52 20 0D 42 02 00 46 0C 94   TXCLEAR .B..F.” 
 0x00110 56 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  Vv.”vv.”vv.”vv.” 
 0x00120 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x00130 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x00140 76 76 0C 94 76 76 0C 94 6A 77 0C 94 76 76 0C 94  vv.”vv.”jw.”vv.” 
 0x00150 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x00160 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x00170 60 77 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  `w.”vv.”vv.”vv.” 
 0x00180 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x00190 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x001A0 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x001B0 76 76 0C 94 76 76 0C 94 76 76 0C 94 1D 77 0C 94  vv.”vv.”vv.”.w.” 
 0x001C0 76 76 0C 94 76 76 0C 94 9F 76 0C 94 76 76 0C 94  vv.”vv.”Ÿv.”vv.” 
 0x001D0 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 
 0x001E0 76 76 0C 94 76 76 0C 94 76 76 0C 94 76 76 0C 94  vv.”vv.”vv.”vv.” 

 

Also, AVROSP does this every 256 bytes:
 

 0x002F0 F3 76 0C 94 76 76 EF 85 2A 86 3F 86 3F 86 3F 86  óv.”vvï…*†?†?†?† 
 0x00300 57 86 82 86 67 84 E7 84 68 8F 61 8F 8C 8B 0D 41  A 
 0x00310 01 00 4C 65 65 72 65 6E 20 64 65 72 20 73 65 72  ..Leeren der ser 
 0x00320 69 65 6C 6C 65 6E 20 53 63 68 6E 69 74 74 73 74  iellen Schnittst 
 0x00330 65 6C 6C 65 3A 20 54 58 43 4C 45 41 52 20 0D 42  elle: TXCLEAR .B 
 0x00340 02 00 46 72 8B CC 8E A8 8E DC 8E 49 8C 01 8E E3  ..Fr‹ÌŽ¨ŽÜŽIŒ.Žã 
 0x00350 8C BC 8A 66 8A F0 8B A7 8B 23 8B CF 8A 17 8A B8  Œ¼ŠfŠð‹§‹#‹ÏŠ.Š¸ 
 0x00360 89 47 89 F6 88 AB 8C 4F B7 69 B7 A7 B7 2A B7 2A  ‰G‰öˆ«ŒO·i·§·*·* 

whereas AVRDude doesn't.
Now, I'm not sure what all of this means, but I guess I'll reverse-engineer AVROSP and include the relevant parts in my app.
Might take a while, but 6 days of trial-error without getting anywhere is enough... :P
That way, at least I'll get a reliable progress bar.

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

No, I have not been following this thread.
.
Surely you would use avrdude -c avrosp if you want avrdude to behave like AVROSP.
.
And if you encounter problems, you simply create test programs smaller than 128kB and bigger than 128kB. Compare the performance.
.
I possess one mega2560. Every other mega/xmega is <= 128kB. Since everything is in words 128kB does not break the 16-bit address barrier. 256kB obviously does.
.
David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

There's no -c option "avrosp". "avr911" should do that.

They're all based on butterfly, anyway, as per avrdude.conf.

I can't simply create another test program, since I have to create a solution that flashes the customer's .hex file (which is slightly larger than 128kb) to their proprietary device and I do not have access to their source code (yet).
If I find a workaround, I'll let you know. For now, I guess I'm stuck with having to reproduce the exact behaviour of avrosp.exe...

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Tozzi wrote:

 

AVR Part : ATxmega256A3 

...

avrdude: Device signature = 0x1e9842 (probably x256a3u)

 

 

Not sure it matters in this case, but are you using a xmega256a3u or xmega256a3?

The non 'u' suffix had more bugs than the newer 'u' suffix device.

 

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ront1234 wrote:

Not sure it matters in this case, but are you using a xmega256a3u or xmega256a3?

The non 'u' suffix had more bugs than the newer 'u' suffix device.


I've tried both. Not sure which one is actually built into the unit, but it's probably the newer one.
Signature is the same for both.
 

//EDIT:
Analyzing the differences more closely, I can see that avrdude writes 128 Byte blocks at a time, while avrosp writes 512.
The initial commands differ somewhat, too.

The bootloader responds with 0x0d correctly after each block, but somehow it still fails.

 

I tend to rule out any problems with address range, at least when it comes to parsing the .hex file.

Not sure what exactly goes wrong here, yet, but at this point I've tested every possible and impossible combination of parameters for avrdude, trying different versions, too.
Since AVROSP.exe is not all too complex, I'll analyze its source code and try to incorporate it in my Updater app.
Going to be a lot of work, but no rocket science, and had I done that right away, I'd be done with it by now...

Anyway, thanks for all your help!

When I'm done analyzing all this, I'll post a summary of my findings.

Greetings from Munich, Germany
Stephan

Last Edited: Tue. May 23, 2017 - 05:36 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Tozzi wrote:

Unfortunately, there's no way to tell, since read (verify) always returns 0x00 regardless.
I did sniff out the serial communication by using "Advanced Serial Port Monitor" under Windows, and found out some differences.

 

It would help your debugging if you write a small program to be able to read the actual flash memory being written.

For example, you could just have it spit out the first 256 bytes starting at location 0x0000.

Send those bytes out through the USART just once when it's first powered up.

 

Edit:

You could write the program in assembler and locate it in high memory where the main program won't touch it.

Better yet can you modify the bootloader?

 

 

 

Last Edited: Tue. May 23, 2017 - 05:52 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ront1234 wrote:

You would need to write the program in assembler and locate it in high memory where the main program won't touch it.

Better yet can you modify the bootloader?


Nope, unfortunately, as I said, it's basically a black box to me and I don't have the sources, just the .hex file.
The boot loader is certainly based on AVROSP but probably is somewhat ancient and has had some "hacks" to it to support the larger flash memory(?)...

The main differences I figured out so far:
AVROSP checks for the presence of "AVRBOOT", then issues 0xb2 to find out if block mode is supported (device returns "Y", followed by 0x02 and 0x00).
Then somewhere there's some 41 00 00 0D, followed by blocks initiated by 42 02 00 46 ("B", 512, "F")
Then 512 bytes are written, the device replies with 0x0D. Next block begins with 41 01 00 0D. And so on.

avrdude basically does the same, but splits the 512 bytes in 4 blocks of 128 bytes each before the 41 01 00 0D (Address, I suppose) is being issued.
So there we *might* have the reason.

Guess I'm not giving up on avrdude, yet... 

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You can also read the flash memory (save it to an ihex file) using the Atmel Studio READ command in device programming.

The turn around time is a little slower but without any programming, etc.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Will try. But so far, all my attempts to read from the flash memory have failed (always returns 0x00). :(

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

OK I think I've found another significant difference, which actually does apply to addressing.
 

To sum it up:
AVRdude sends "A" (0x41) followed by the address, and does this every 4 blocks of 128 Byte data.
Data blocks are sent after the sequence "0x42 0x00 0x80 0x46" (B 128 F)
After "0x41 0xff 0x00" ("A" 0xff00) it jumps to "0x41 0x00 0x00" ("A" 0x0000). This is obviously wrong and confirms steve17's assumption.
I just wasn't ready to accept the idea that a tool written in 2004 could get something right that avrdude doesn't.
 

AVROSP.EXE writes 512 Bytes at a time (header is 0x42 0x02 0x00 0x46 or "B 512 F") and each time sends an address header.
It begins using the same one as avrdude (0x41 0x00 0x00 or "A" 0x0000), but when it reaches past 0xffff it switches to 0x48 0x01 0x00 0x00 ("H" 0x010000).

Problem found. At least a significant part of it. I don't want to be overly optimistic now, but this is promising, fingers crossed.
 

Now how to fix this...
Will analyze avrdude and see if this behaviour can be changed without too much effort.

Thanks to all of you who replied for hanging in there with me!

//EDIT: BTW: Here's the relevant part in the AVROSP.EXE source code:
 

void AVRBootloader::setAddress( long address )
{
	/* Set current address */
	if( address < 0x10000 ) {
		comm->sendByte( 'A' );
		comm->sendByte( (address >> 8) & 0xff );
		comm->sendByte( address & 0xff );
		comm->flushTX();
	} else {
		comm->sendByte( 'H' );
		comm->sendByte( (address >> 16) & 0xff );
		comm->sendByte( (address >> 8) & 0xff );
		comm->sendByte( address & 0xff );
		comm->flushTX();
	}

	/* Should return CR */
	if( comm->getByte() != '\r' ) {
		throw new ErrorMsg( "Setting address for programming operations failed! "
				"Programmer did not return CR after 'A'-command." );
	}
}

 

Greetings from Munich, Germany
Stephan

Last Edited: Tue. May 23, 2017 - 07:33 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

FYI, I saw this regarding avrdude, it adds :441 to the config filename when using avr911 with the -C option --> avrdude.conf:441

No idea if this will even work but might be worth a try.

 

 

 

 

This reply has been marked as the solution. 
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

SUCCESS! Heureka!

It works now with avrdude. Version 6.3.

Without digging too deep in there, I just made some q&d modifications which I'll send to Jörg for review.
I've overridden the page size (estimated as 128) with 512, and modified butterfly_set_addr to incorporate butterfly_set_extaddr.
(Based on addr).

It's still roughly 6 times faster than avrosp.exe, but the device boots and reports the correct firmware version now.

This is what I've changed (very quick and very dirty, but both changes were required in order to make it work):

 

--- butterfly.c.orig	2014-07-16 22:14:58.000000000 +0200
+++ butterfly.c	2017-05-23 22:49:20.000000000 +0200
@@ -321,6 +321,10 @@
   PDATA(pgm)->buffersize = (unsigned int)(unsigned char)c<<8;
   butterfly_recv(pgm, &c, 1);
   PDATA(pgm)->buffersize += (unsigned int)(unsigned char)c;
+
+  // overriding it for now
+  PDATA(pgm)->buffersize = 512;
+
   avrdude_message(MSG_INFO, "Programmer supports buffered memory access with buffersize=%i bytes.\n",
                   PDATA(pgm)->buffersize);

@@ -424,6 +428,7 @@

 static void butterfly_set_addr(PROGRAMMER * pgm, unsigned long addr)
 {
+if( addr < 0x10000 ) {
   char cmd[3];

   cmd[0] = 'A';
@@ -432,6 +437,19 @@

   butterfly_send(pgm, cmd, sizeof(cmd));
   butterfly_vfy_cmd_sent(pgm, "set addr");
+
+  } else {
+
+  char cmd[4];
+
+  cmd[0] = 'H';
+  cmd[1] = (addr >> 16) & 0xff;
+  cmd[2] = (addr >> 8) & 0xff;
+  cmd[3] = addr & 0xff;
+
+  butterfly_send(pgm, cmd, sizeof(cmd));
+  butterfly_vfy_cmd_sent(pgm, "set extaddr");
+  }
 }
 

For me, this does it. cool
If I get some spare time, I'll try and figure out why it didn't work in the first place (it should have, as there is actually a butterfly_set_extaddr function in place...).
Also, I don't know yet why the page size is being interpreted (or reported) wrongly as 128 Bytes and I had to override that to make it work.

Greetings from Munich, Germany
Stephan

Last Edited: Tue. May 23, 2017 - 11:25 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Nice work!.

Is that using avr911, I wonder what the butterfly option would do?

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

All the same.
Take a look at avrdude.conf, just minor differences between avr901, 911 and butterfly, all of them referring to butterfly.c ultimately...
I'm going to try and get to the bottom of this, but for now I'm just happy it works... yes

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That's good.  It seems your bootloader has an extended address command.  I guess that is 'H'.  Mine does not.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yea, it's the "H", followed by a 24 bit address, instead of "A" followed by 16.
AVROSP.exe has it; AVRDude has it, too but it's buried in a seperate function (butterfly_set_extaddr) for whatever reason(?) and not being used in my case.
That's a fairly simple patch, however (see above), and I don't think that this one will break anything else.
 

The other weird thing is that avrdude assumes a buffer size of 128 bytes, while 512 seems to be the way to go.
AVRDude does actually even report a page size of 512 bytes in verbose mode but for the buffer size it relies on a response to the "b" command which, after the "Y" reply, for some reason is different from what happens with AVROSP.exe.
Still investigating this, since I want to contribute to the community and not just use my patched version without the developer knowing.
Right now I am simply overriding it, but it's GPL after all...
 

Initially I thought the extremely shorter time it takes avrdude to flash the unit was a hint as to what's going wrong, but it's all good.
avrosp.exe takes more then 6 minutes, but my modified avrdude does it in 50 seconds and it works just fine...
 

Anyway, thanks, Steve17, for pointing out the probability of an addressing problem to me!
This was ultimately the path to finding the solution.
At first I found it highly unlikely that a tool written back in 2004 does this better than avrdude, but here we are...

Greetings from Munich, Germany
Stephan

Last Edited: Wed. May 24, 2017 - 09:22 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The situation on the avrdude doesn't surprise me.  What surprised me is the avr911 bootloader on the device handles the H command.  My avr911 bootloader does not.  

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Avrdude using 128 bytes is surprising.  It is supposed to ask the bootloader on the device what it wants.  I have my bootloader set up for 256 bytes and that is what avrdude delivers.  When I was testing my bootloader I had it set to 32 bytes for a while, and that is what avrdude sent.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

steve17 wrote:

Avrdude using 128 bytes is surprising.  It is supposed to ask the bootloader on the device what it wants.


It actually does, but for some reason the result is wrong or treated incorrectly in my case. AVROSP.exe receives the correct value 0x0200.
I'll dig deeper and try to find the reason, will post here when I do. 

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

OK it seems that somehow, sometimes, 0x0080 is being reported to avrdude, instead of the correct 0x0200.
Must probably it has got something to do with some command(s) avrdude sends before the "b" question.
I suspect it might be the "a" command (auto addr increase). AVROSP.exe does not issue that one.
 

However, I can't be bothered by this any further. Lost enough time to it. Learned a lot, so, personally, I'd not count the time as "lost" though.
GPL is prohibiting me from using this unfortunately (except for a Linux command line tool which I will make available as an add-on-candybar).
Can't put all these disclaimers and other legalese stuff in my Mac OS X Updater, the client would most likely not be very amused by that.
 

Now that I have had to dive into the bits-and-bytes depth of things anyway, I'll implement my own version and live happily ever after. ;)

Greetings from Munich, Germany
Stephan

Last Edited: Thu. May 25, 2017 - 10:50 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks for the update.

 

I'm wondering about the 'H' command.  I may never need it, but maybe someday I will.   Will avrdude send it when needed or must I patch the source code?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It is implemented, but as far as I can tell it's not actually being used.
I'd say patch it, that won't do any harm. Just omit the first part of the patch where I'm overriding the buffer size.

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Tozzi wrote:

It is implemented, but as far as I can tell it's not actually being used.
I'd say patch it, that won't do any harm. Just omit the first part of the patch where I'm overriding the buffer size.

Okay.  I wonder if anyone is working on avrdude now.  Georg Wunsch, forum name dl8dtl, used to do it, but he seems to be inactive lately.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Did you get the source from nongnu.org?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yes, I've used the original source code for 6.3.
I've submitted my patch as a bug report, we'll see if someone picks up on it.
I'm willing to provide assistance, using the log files i've made by sniffing the serial comms.
https://savannah.nongnu.org/bugs...

Greetings from Munich, Germany
Stephan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks.