UPDI Linux prod programming solution? Pyupdi slowness

Go To Last Post
18 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi,

 

I've frustratingly spent more time than I'd like to admit seeking out a right solution to program AVR 0-series devices in production.  We want to use a Linux machine - Pi preferably, but any small Linux box is ok. 

 

My starting point has been trying to use an Atmel-ICE debugger with official tools.  Due to Linux, Atmel Studio's command line programming tool (Windows only) is ruled out.  Microchip's IPE sounded promising as it interfaces with the ICE and claims compatibility with AVRs - but I cannot get it to program memory on the device beyond 0x6FFF for some reason (with either auto or manual memory selection).  No matter what I do, it stops programming at 0x6FFF, writes some number of 0s after, and leaves the rest of the memory at 0xFF.  I have a post in the Microchip forums about this as it seems like being able to use their tool in command line would be cleanest.  I don't see claims of UPDI/AVR support in any of Microchip's PICKit programmers, so I have not pursued trying this yet - although I have been told by company reps that this should be supported.

 

Avrdude does not seem to support UPDI devices.

 

I see pyupdi recommended a lot for this.  I spent a day getting this up and running on a Pi 3 and had to do quite a bit of wrestling with it.  I ended up having to use Python 3 instead of 2.  However, it is slow!  Flashing a ~28kB image takes about 1:15.  There is no way we can have the flashing step take this long in production.

 

Does anyone have advice on tools or approach?  Am I mistaken on any of my information here?  I'm pretty baffled there is not a great working manufacturer solution ~2 years into UPDI's existence, unless I'm sorely missing some information.

 

Thanks!

Last Edited: Thu. Oct 24, 2019 - 03:59 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ajosev5@gmail.com wrote:
I ended up having to use Python 3 instead of 2. 
probably a good plan to look at migrating anyway as Py 2 officially dies 1st Jan 2020

 

EDIT:  https://pythonclock.org/

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yep good call, and I saw that the creator of pyupdi has put in warnings that py 3 will yield best results.  Still very slow, unfortunately!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ajosev5@gmail.com wrote:
 I have a post in the Microchip forums about this as it seems like being able to use their tool in command line would be cleanest.
ipecmd may be functional for AVR though I don't recall which version of MPLAB X IPE and which programmer (Atmel-ICE, MPLAB PICkit 4, MPLAB Snap)

ajosev5@gmail.com wrote:
I don't see claims of UPDI/AVR support in any of Microchip's PICKit programmers, so I have not pursued trying this yet - although I have been told by company reps that this should be supported.
AVR is beta in MPLAB X v5.25 though apparently functional.

ajosev5@gmail.com wrote:
Avrdude does not seem to support UPDI devices
The current AVRDUDE is in Debian buster (stable); UPDI appears complete post 6.3 (16-Feb'16)

ajosev5@gmail.com wrote:
Does anyone have advice on tools or approach?
linuxspi and linuxgpio are effective for AVR ISP; AVR UPDI is 1-wire UART with a protocol so /dev/tty?

Might browse current AVRDUDE source code on UPDI.

AVRDUDE to Atmel-ICE for UPDI should be a go after atfw.exe the Atmel-ICE.

Some inexpensive gang-like production programmers are multiple inexpensive programmers and a motherboard (PC, ARM, etc) into an enclosure.

Atmel-ICE PCBA is well stocked.

 


https://www.avrfreaks.net/search/site/ipecmd

https://www.avrfreaks.net/forum/come-join-us-mplab-now-supports-avrs?page=5#comment-2758631 (Atmel-ICE, MPLAB X)

https://packages.debian.org/search?keywords=AVRDUDE&searchon=names&suite=all&section=all

http://svn.savannah.gnu.org/viewvc/avrdude/trunk/?view=log

AVR Downloader/UploaDEr - Patches: patch #9507, Fix UPDI chip erase [Savannah]

Atmel Studio 7 - Manual Upgrade (or automatic upgrade)

https://www.microchipdirect.com/product/search/all/ATATMEL-ICE-PCBA

 

edit :

manufacturing operators may prefer a GUI :

Production Mode - Developer Help (MPLAB X IPE) (MPLAB IPE)

 

edit2 :

Modify pyupdi to go via tty instead of usb?

 

edit3 :

Atmel-ICE | Features

...

  • Supports UPDI baud rates from up to 750kbit/s

...

A fast USB UART :

XR21B1420 - MaxLinear

very fast over PCIe :

XR17V358 - MaxLinear

 

"Dare to be naïve." - Buckminster Fuller

Last Edited: Thu. Oct 24, 2019 - 04:53 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ajosev5@gmail.com wrote:
I've frustratingly spent more time than I'd like to admit seeking out a right solution to program AVR 0-series devices in production.

...

[pyupdi]

...

Flashing a ~28kB image takes about 1:15.

megaAVR 0-series, mega480x, 128B/page, mega4809 DIP page erase-write is 4ms

28KB * 1page/128B * 4ms/page = 896ms

Is the USB UART's baud rate correctly set?

UPDI is an order of magnitude faster than debugWIRE.

ajosev5@gmail.com wrote:
Does anyone have advice on tools or approach?
Maybe mega4809 is on its way to programming at Microchip (tiny3217 is available, partial reels may be possible)

PROGRAMMING COST LOOKUP (microchipDIRECT)

 


https://www.avrfreaks.net/forum/debugwire-usb-uart?page=1#comment-2291786

 

edit : AVR8 on Microchip Direct | AVR Freaks

 

"Dare to be naïve." - Buckminster Fuller

Last Edited: Thu. Oct 24, 2019 - 08:13 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

gchapman wrote:

UPDI appears complete post 6.3 (16-Feb'16)

 

Has anyone tried UPDI in avrdude, I can't tell for sure what is going on from the source (note facchinm is from Arduino, and this seems to be what got packaged for Raspbian)

 

https://github.com/facchinm/avrd...

 

It seems to mostly be in the jtag3.c file and have a programing type (-c) jtagice3_updi. I wonder if it needs wired like PiUPDI and also given serial port interface parameters. 

my projects: https://github.com/epccs

Debugging is harder than programming - don’t write code you can’t debug! https://www.avrfreaks.net/forum/help-it-doesnt-work

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks for all your info/thoughts!

 

> Is the USB UART's baud rate correctly set?

pyupdi is setting the default 115200, which I am confirming with verbose output.  From looking at the output in real time, it seems to me the script is just taking a lot of time in between page writes.  There are many UPDI operations in between, and it seems to be a slow speed of execution, such that it's only doing 3-4 page writes per second.  I would not guess the baud rate to be the limiting factor based on what I'm seeing.  See snippet below (as I mentioned, the page writes of data only come 3-4 times per second, no wonder it takes so long!)  Maybe a C implementation would be much faster.

 

INFO:phy send : [0x76, 0xe5, 0xe, 0x94, 0x8a, 0x7, 0x5c, 0x1, 0x0, 0x97, 0x9, 0xf4, 0x40, 0xc0, 0xe, 0x94, 0xb, 0x8, 0xc7, 0x1, 0x8, 0x96, 0xe, 0x94, 0x63, 0x7, 0x6b, 0x1, 0xf5, 0x1, 0x1, 0x90, 0x0, 0x20, 0xe9, 0xf7, 0x31, 0x97, 0xea, 0x19, 0xfb, 0x9, 0x32, 0x96, 0xee, 0xd, 0xff, 0x1d, 0xc6, 0x17, 0xd7, 0x7, 0x31, 0xf4, 0xab, 0x1, 0xbf, 0x1, 0xc8, 0x1, 0xe, 0x94, 0x8f, 0x7, 0x28, 0xc0, 0x82, 0xeb, 0x97, 0xe5, 0xe, 0x94, 0xb, 0x8, 0xf, 0x2e, 0xfe, 0xef, 0xcf, 0x2e, 0xdd, 0x24, 0xda, 0x94, 0xf0, 0x2d, 0x1d, 0xc0, 0x66, 0xec, 0x77, 0xe5, 0x8c, 0xe3, 0x91, 0xe3, 0xe, 0x94, 0x98, 0x7, 0x89, 0x2b, 0x41, 0xf0, 0x86, 0xed, 0x97, 0xe5, 0xe, 0x94, 0xb, 0x8, 0xcc, 0x24, 0xca, 0x94, 0xdc, 0x2c, 0xd, 0xc0, 0x8b, 0xee, 0x97, 0xe5, 0xe, 0x94, 0xb, 0x8]
INFO:link STCS to 0x02
INFO:phy send : [0x55, 0xc2, 0x80]
INFO:app Committing page
INFO:app NVMCMD 1 executing
INFO:link ST to 0x1000
INFO:phy send : [0x55, 0x44, 0x0, 0x10]
INFO:phy receive : [0x40]
INFO:phy send : [0x1]
INFO:phy receive : [0x40]
INFO:app Wait flash ready
INFO:link LD from 0x1002
INFO:phy send : [0x55, 0x4, 0x2, 0x10]
INFO:phy receive : [0x0]
INFO:nvm Writing page at 0x4580
INFO:app Wait flash ready
INFO:link LD from 0x1002
INFO:phy send : [0x55, 0x4, 0x2, 0x10]
INFO:phy receive : [0x0]
INFO:app Clear page buffer
INFO:app NVMCMD 4 executing
INFO:link ST to 0x1000
INFO:phy send : [0x55, 0x44, 0x0, 0x10]
INFO:phy receive : [0x40]
INFO:phy send : [0x4]
INFO:phy receive : [0x40]
INFO:app Wait flash ready
INFO:link LD from 0x1002
INFO:phy send : [0x55, 0x4, 0x2, 0x10]
INFO:phy receive : [0x0]
INFO:link ST to ptr
INFO:phy send : [0x55, 0x69, 0x80, 0x45]
INFO:phy receive : [0x40]
INFO:link Repeat 64
INFO:phy send : [0x55, 0xa1, 0x3f, 0x0]
INFO:link ST16 to *ptr++
INFO:link STCS to 0x02
INFO:phy send : [0x55, 0xc2, 0x88]
INFO:phy send : [0x55, 0x65]
INFO:phy send : [0xf, 0x2e, 0xfe, 0xef, 0xcf, 0x2e, 0xdd, 0x24, 0xda, 0x94, 0xf0, 0x2d, 0x2, 0xc0, 0xc1, 0x2c, 0xd1, 0x2c, 0x10, 0x92, 0x17, 0x30, 0x2, 0xc0, 0xc1, 0x2c, 0xd1, 0x2c, 0xc6, 0x1, 0xdf, 0x91, 0xcf, 0x91, 0x1f, 0x91, 0xf, 0x91, 0xff, 0x90, 0xef, 0x90, 0xdf, 0x90, 0xcf, 0x90, 0xbf, 0x90, 0xaf, 0x90, 0x8, 0x95, 0x2f, 0x92, 0x3f, 0x92, 0x4f, 0x92, 0x5f, 0x92, 0x6f, 0x92, 0x7f, 0x92, 0x8f, 0x92, 0x9f, 0x92, 0xaf, 0x92, 0xbf, 0x92, 0xcf, 0x92, 0xdf, 0x92, 0xef, 0x92, 0xff, 0x92, 0xf, 0x93, 0x1f, 0x93, 0xcf, 0x93, 0xdf, 0x93, 0xcd, 0xb7, 0xde, 0xb7, 0xc2, 0x50, 0xd1, 0x40, 0xcd, 0xbf, 0xde, 0xbf, 0x8c, 0x1, 0xcf, 0x5f, 0xde, 0x4f, 0x18, 0x82, 0x19, 0x82, 0xc1, 0x50, 0xd1, 0x40, 0xe, 0x94, 0xf9, 0x0, 0x6f, 0x3f, 0x2f, 0xef, 0x72, 0x7, 0x82, 0x7, 0x92, 0x7]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The version of AVRDUDE distributed with Arduino seems to support several UPDI programmers, including the Xplained Pro (EDBG) and Xplained Mini (mEDBG), as well as jtag2updi using el tangas's programming SW.  I don't know about Atmel ICE.  (Are your sure that your ICE firmware is up-to-date?)  I've used SNAP and PicKit4 with the MPLAB tools, but I haven't programmed anything as large as 0x6FFF, so I don't know if it has the same problem that you are seeing.

 

 

I'm pretty baffled there is not a great working manufacturer solution ~2 years into UPDI's existence

Why baffled?  "Vendor has poor linux support" is very common...

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
There are many UPDI operations in between, and it seems to be a slow speed of execution, such that it's only doing 3-4 page writes per second.  I would not guess the baud rate to be the limiting factor based on what I'm seeing.  See snippet below (as I mentioned, the page writes of data only come 3-4 times per second, no wonder it takes so long!)

I'd love to see a log with timestamps...
Rearranging slightly, and adding comments, I see:

 

INFO:nvm Writing page at 0x4580        Write a page of data

  INFO:app Wait flash ready              (unnecessary)
    INFO:link LD from 0x1002
    INFO:phy send : [0x55, 0x4, 0x2, 0x10]
    INFO:phy receive : [0x0]
  INFO:app Clear page buffer             (unnecessary)
    INFO:app NVMCMD 4 executing
    INFO:link ST to 0x1000
    INFO:phy send : [0x55, 0x44, 0x0, 0x10]
    INFO:phy receive : [0x40]
    INFO:phy send : [0x4]
    INFO:phy receive : [0x40]
  INFO:app Wait flash ready              (unnecessary)
     INFO:link LD from 0x1002
     INFO:phy send : [0x55, 0x4, 0x2, 0x10]
     INFO:phy receive : [0x0]
  INFO:link ST to ptr
  INFO:phy send : [0x55, 0x69, 0x80, 0x45]  (our address ptr being loaded)
  INFO:phy receive : [0x40]
  INFO:link Repeat 64                       (and 64 bytes of data)
  INFO:phy send : [0x55, 0xa1, 0x3f, 0x0]
  INFO:link ST16 to *ptr++
  INFO:link STCS to 0x02
  INFO:phy send : [0x55, 0xc2, 0x88]
  INFO:phy send : [0x55, 0x65]
  INFO:phy send : [0xf, 0x2e, 0xfe, 0xef, 0xcf, 0x2e, 0xdd, 0x24, 0xda, 0x94, 0xf0, 0x2d, 0x2, 0xc0, 0xc1, 0x2c, 0xd1, 0x2c, 0x10, 0x92, 0x17, 0x30, 0x2, 0xc0, 0xc1, 0x2c, 0xd1, 0x2c, 0xc6, 0x1, 0xdf, 0x91, 0xcf, 0x91, 0x1f, 0x91, 0xf, 0x91, 0xff, 0x90, 0xef, 0x90, 0xdf, 0x90, 0xcf, 0x90, 0xbf, 0x90, 0xaf, 0x90, 0x8, 0x95, 0x2f, 0x92, 0x3f, 0x92, 0x4f, 0x92, 0x5f, 0x92, 0x6f, 0x92, 0x7f, 0x92, 0x8f, 0x92, 0x9f, 0x92, 0xaf, 0x92, 0xbf, 0x92, 0xcf, 0x92, 0xdf, 0x92, 0xef, 0x92, 0xff, 0x92, 0xf, 0x93, 0x1f, 0x93, 0xcf, 0x93, 0xdf, 0x93, 0xcd, 0xb7, 0xde, 0xb7, 0xc2, 0x50, 0xd1, 0x40, 0xcd, 0xbf, 0xde, 0xbf, 0x8c, 0x1, 0xcf, 0x5f, 0xde, 0x4f, 0x18, 0x82, 0x19, 0x82, 0xc1, 0x50, 0xd1, 0x40, 0xe, 0x94, 0xf9, 0x0, 0x6f, 0x3f, 0x2f, 0xef, 0x72, 0x7, 0x82, 0x7, 0x92, 0x7]
  INFO:link STCS to 0x02
  INFO:phy send : [0x55, 0xc2, 0x80]
  INFO:app Committing page
  INFO:app NVMCMD 1 executing            (erase and write page)
    INFO:link ST to 0x1000
    INFO:phy send : [0x55, 0x44, 0x0, 0x10]
    INFO:phy receive : [0x40]
    INFO:phy send : [0x1]
    INFO:phy receive : [0x40]
  INFO:app Wait flash ready             (MAYBE this one makes sense...)
  INFO:link LD from 0x1002
  INFO:phy send : [0x55, 0x4, 0x2, 0x10]
  INFO:phy receive : [0x0]

 

That doesn't look awful, although there are several checks for "flash ready" after operations that shouldn't make it NOT ready (or before operations that don't need it to be ready?)   Writing a flash page STOPS the CPU, so a bootloader rarely needs to check for flash busy.  I wonder if the UPDI system is also halted...  (Note that the flash status is never "busy"..)

 

Quote:
Maybe a C implementation would be much faster.

I'm not convinced that it would be.

Have you tried experiments on zippy windows boxes to see what sort of programming times you get there?

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

What is your production size ? if its in thousands I would suggest you to get a decent In-Circuit programmer, that means you just flash the hex files with the setting to the programmer and then you dont need a PC. from my experience I used Softlog programmers ICP2(G3) which supports AVR-0 & AVR-1 series...UPDI.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'll have to look into that AVRDUDE with Arduino.  I do have the latest FW on my ICE...

 

I've used SNAP and PicKit4 with the MPLAB tools

I ordered a PicKit4, since it is supposed to support what I'm doing, and maybe it'll play nicer with IPE.

 

Why baffled?  "Vendor has poor linux support" is very common...

Fair enough.  I'm only a couple years professionally into the embedded world, mostly having been around STM32/ARM devices which have good programming support under Linux...

 

Thanks!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It will be thousands.  I began looking at the Softlog programmers, I like them.  I just wish they had Linux support.  They can be used standalone but I want to be able to switch between different "environments" for bootloader, test FW, production FW flashing... and we would need a Windows PC for that, it looks like.  

 

Thanks!  I like the approach of just going with a true production programmer.  I've looked at a few companies, though, and haven't yet found the magic combination of AVR UPDI support and Linux software.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Have you tried experiments on zippy windows boxes to see what sort of programming times you get there?

I just set it up on my i7 Windows box.  It is faster, completing programming in just under a minute versus 1:15.  So there seems to be some non-trivial host PC execution time overhead.  It may just not be the right tool for what we need - a mass production programmer device is probably in our best interest, though Linux support is not generally a strong suit in these.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

westfw wrote:

INFO:nvm Writing page at 0x4580        Write a page of data

  INFO:app Wait flash ready              (unnecessary)
    INFO:link LD from 0x1002
    INFO:phy send : [0x55, 0x4, 0x2, 0x10]
    INFO:phy receive : [0x0]

 

The UPDI command I marked in red is an important contribution to the slowness of pyupdi. It sets a very conservative timing for the UPDI interface.

I made a few experiments, recompiling my own jtag2updi programmer to use this setting. These are the results (writing a project with 2434 bytes):

 

- Using the more aggressive timing I normally set for jtag2updi:

avrdude.exe: jtagmkII_initialize(): Cannot locate "flash" and "boot" memories in description
avrdude.exe: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.15s

avrdude.exe: Device signature = 0x1e9422 (probably t1614)
avrdude.exe: NOTE: Programmer supports page erase for Xmega devices.
             Each page will be erased before programming it, but no chip erase is performed.
             To disable page erases, specify the -D option; for a chip-erase, use the -e option.
avrdude.exe: reading input file "jtag2updi_lite.hex"
avrdude.exe: writing flash (2434 bytes):

Writing | ################################################## | 100% 0.99s

avrdude.exe: 2434 bytes of flash written
avrdude.exe: verifying flash memory against jtag2updi_lite.hex:
avrdude.exe: load data flash data from input file jtag2updi_lite.hex:
avrdude.exe: input file jtag2updi_lite.hex contains 2434 bytes
avrdude.exe: reading on-chip flash data:

Reading | ################################################## | 100% 0.55s

avrdude.exe: verifying ...
avrdude.exe: 2434 bytes of flash verified

avrdude.exe done.  Thank you.

In this case, total time is 1.69 seconds.

 

- Using the pyupdi settings


avrdude.exe: jtagmkII_initialize(): Cannot locate "flash" and "boot" memories in description
avrdude.exe: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.16s

avrdude.exe: Device signature = 0x1e9422 (probably t1614)
avrdude.exe: NOTE: Programmer supports page erase for Xmega devices.
             Each page will be erased before programming it, but no chip erase is performed.
             To disable page erases, specify the -D option; for a chip-erase, use the -e option.
avrdude.exe: reading input file "jtag2updi_lite.hex"
avrdude.exe: writing flash (2434 bytes):

Writing | ################################################## | 100% 2.68s

avrdude.exe: 2434 bytes of flash written
avrdude.exe: verifying flash memory against jtag2updi_lite.hex:
avrdude.exe: load data flash data from input file jtag2updi_lite.hex:
avrdude.exe: input file jtag2updi_lite.hex contains 2434 bytes
avrdude.exe: reading on-chip flash data:

Reading | ################################################## | 100% 0.61s

avrdude.exe: verifying ...
avrdude.exe: 2434 bytes of flash verified

avrdude.exe done.  Thank you.

In this case, the total time is 3.45 seconds. Since the OP program is about 11.5x larger, it would take ~40 seconds with these settings. With my settings, it would take about 20 seconds (using avrdude/jtag2updi).

So this accounts for about 20s of the slowness, but it still leaves another 35s to be explained by some other factors (?).

 

Now, in theory, using a direct USB link instead of an USB/UART bridge, and the maximum UPDI baud rate (about 900 kbaud) it would be possible to make a programmer ~10x faster than my own jtag2updi, that would upload 28kB in ~2 seconds.

Last Edited: Sat. Oct 26, 2019 - 02:10 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hello ajosev5,  I am curious -  How did you finally solve your production UPDI programming problem.  Did you go with pyupdi ?  Did you go with a production programmer ?  What production programmer did you choose ?

 

I have been using pyupdi on Raspberry Pi zero myself.

Rebeca Chan

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

An R-Pi4 has 6 UART ports. Thanks for the R-Pi Zero report that is what I plan to use, but now I want @mraardvark to set it up for the AVR-Dx line.

my projects: https://github.com/epccs

Debugging is harder than programming - don’t write code you can’t debug! https://www.avrfreaks.net/forum/help-it-doesnt-work

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ron_sutherland wrote:
but now I want @mraardvark to set it up for the AVR-Dx line.

Yes, it looks like there is some work to do on that front soon...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

mraardvark wrote:
soon...

 

smiley

my projects: https://github.com/epccs

Debugging is harder than programming - don’t write code you can’t debug! https://www.avrfreaks.net/forum/help-it-doesnt-work