low footprint USB virtual serial

Go To Last Post
26 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I've been working on a secure bootloader for one of my products with a traditional USART and I have that working great.  Now I'm turning my focus on my autoprogrammer project.  It is a 32U4 and the RX/TX on it are already connected for a PDI situation.  Each has a 220 ohm resistor to the pin 1 on the programming connector which means that they are 440 ohms apart from each other.  My other project had a similar situation in that the RX/TX were pins being used to drive a LED matrix, but I found that in bootloader mode where that matrix isn't being driven, I could connect RX/TX to my USB to TTL adapter and it would work fine.  In that case one, RXD was connected to one of the anodes, TXD to one of the cathodes.  If no RX/TX lines are hooked up, the TX wanted to introduce some noise into the RX which is would sometimes pick up and other times not.  ANYWAY, back to the autoprogrammer 32U4.  I wonder if I should try the RXD and TXD lines to see if it will work even though they have 440 ohms resistance between them.  That is the "quick and minimal development route".  Alternatively, it seems a shame to not use the USB which is already built into the device for communications.  I could just duplicate the protocol I am using which sends/receives 64 byte packets, but I would have to make a PC side program that talks that protocol that would be used for the upgrade.  I'd rather implement a USB virtual serial port and then go the XMODEM CRC protocol route like I did on the USART project.  Is there minimal code for a USB virtual serial port that would run fine in a bootloader?  I have run interrupts in a bootloader before, but I'd rather avoid that if possible.  Maybe the USB isn't worth it and I should try the USART RX/TX route on one...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:

....  Alternatively, it seems a shame to not use the USB which is already built into the device for communications.  I could just duplicate the protocol I am using which sends/receives 64 byte packets, but I would have to make a PC side program that talks that protocol that would be used for the upgrade.  I'd rather implement a USB virtual serial port and then go the XMODEM CRC protocol route like I did on the USART project.  Is there minimal code for a USB virtual serial port that would run fine in a bootloader?  I have run interrupts in a bootloader before, but I'd rather avoid that if possible.  Maybe the USB isn't worth it and I should try the USART RX/TX route on one...

It's always a trade off....  :)   USB loaders always seem to be larger, and a full USB-serial strictly needs to add baud calcs, which is a long divide. They are also harder to get working :)

 

You could look at the code here https://github.com/grigorig/stcgal

 

That does PC-side download, to a Serial and USB version of a MCU loader.

In the STC case the loader is in ROM, but you could easily implement the same protocol in your firmware, and have the PC side already done for you ?

Info on their USB loader is not easy to find (not speaking Chinese) , but grigorig looks to have done some deft reverse engineering here, and made a PC-side link that avoids their GUI, and that you can modify.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:
Is there minimal code for a USB virtual serial port that would run fine in a bootloader?
Likely yes though the following is for an application (mega32U4, interrupts,  WinAVR, USB CDC)

PJRC

PJRC

C code for Teensy: USB Serial

https://www.pjrc.com/teensy/usb_serial.html

PJRC HalfKay bootloader is via USB HID.
If go via USB CDC and by operating systems' drivers then don't need to sign a driver (is that true for macOS?)
Else, a custom descriptor will need a signed driver; examples of such are LUFA AVRISP2 (libusb instead of WinUSB) and Atmel-ICE (composite - program and debug via USB HID, DGI is custom)
USB HID - macOS requires a signature.
 
via http://download.savannah.gnu.org/releases/avrdude/ (AVRDUDE 6.4 has libhidapi)
 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Who-me wrote:
USB loaders always seem to be larger, ...
Paul's PJRC HalfKay bootloader is amazing and its loader source code is relatively simple.

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:
Alternatively, it seems a shame to not use the USB

 

Have you considered using LUFA CDC Bootloader example?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Dean has some really mad bootloaders in fact. As well as the CDC and DFU "obvious" ones there's a nice one that does MSD so that you effectively just drag/drop the new firmware file to the "drive". But I  seem to remember there's even a Printer class one that you can "print" new firmware too. Most of his fit in a 4K section.

 

EDIT See:

 

https://github.com/abcminiuser/l...

 

(I forgot about the HID one!)

Last Edited: Sat. Apr 21, 2018 - 11:11 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks everyone, lots of good things for me to look at!

 

Question - will the AVR be able to switch between bootloader and application without the PC re-enumerating it if it has the same USB configuration?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

No there has to be reenmuneration which is the whole reason HWB/HWBE exists.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hmmmm.  I'm thinking about having it always come up in the bootloader section with BOOTRST.  Then the bootloader will test the application section with a crc16 and execute it if it is good.  All of this before USB enumeration.  Then if there is no good app section crc16, it will stay in the bootloader.  So normally, it will be in application mode unless the application is corrupt or not present.  The application mode has a command that makes it reset and stay in the bootloader mode (I could use the WDRF in the bootloader to detect if the app code reset).  A failsafe would be holding a button while connecting it, in this case the bootloader would also stay in bootloader mode (in case a defective application was installed that didn't allow switching to the bootloader).  With this setup, the pc application could detect which mode it is in and tell the device to switch between modes (bootloader or application) to do what it needs, either update or normal functioning.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:
With this setup, the pc application could detect which mode it is in and tell the device to switch between modes (bootloader or application) to do what it needs, either update or normal functioning.
Linux Vendor Firmware Service keeps a device in its USB DFU bootloader until commanded to jump to the application.

That won't work for a standalone autoprogrammer unless add to the USB DFU bootloader a test of an active USB SOF and connected to the loader.

The factory installed Microchip megaAVR USB DFU bootloader is dependent on the HWB signal; ATmega32U4RC do not have a factory installed USB DFU bootloader.

 

Linux Vendor Firmware Service

https://fwupd.org/

AT90USB1287, ATXMEGA256A3BU :

https://github.com/hughsie/fwupd-test-firmware/tree/master/AVR

Booting from XMEGA application code into the bootloader (and staying there!)

by hughsie

https://www.avrfreaks.net/forum/booting-xmega-application-code-bootloader-and-staying-there

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It uses (USBSTA & _BV(VBUS)) at power up to see how it is powered, via USB or via its ISP header.  I could make the bootloader test this and stay in the bootloader if it is plugged into the PC.  Then though every time you want to use it in application mode, the pc application when enumerating them would need to tell it to boot the application, and if there is no way back, then they would have to unplug/replug one to be able to update the firmware on it.

 

I'm tempted to just make the darn thing use its RX/TX for an XMODEM CRC bootloader like I've already developed.  Usually someone with a programmer is going to be capable of hooking up a TTL to it especially if I add a RX/TX via/hole in the pcb to make it easy.  That seems so much simpler, albeit less convenient for the user.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:
I'm tempted to just make the darn thing use its RX/TX for an XMODEM CRC bootloader like I've already developed.
fyi, USB CDC ACM is easier in Windows 10 than earlier Windows.

Microsoft logo

Microsoft Windows USB Core Team Blog

What is new with Serial in Windows 10

by George Roussos [Microsoft]

July 29, 2015

https://blogs.msdn.microsoft.com/usbcoreblog/2015/07/29/what-is-new-with-serial-in-windows-10/

...

 

1.   Improved Serial over USB driver support in Windows 10

...

Now devices that report these compatible IDs: 

  • USB\Class_02&SubClass_02&Prot_01
  • USB\Class_02&SubClass_02

… including popular prototyping boards like Arduinos – just work with our built-in driver.  

...

USB CDC ACM is simple in Linux (libusb), and IIRC, likewise in Android (tablets, some Chromebooks)

Does macOS require a signature of all USB CDC ACM?

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

gchapman wrote:

Does macOS require a signature of all USB CDC ACM?

 

MacOS doesn't care about VID/PID for USB CDC so all CDC ACM compliant devices will attach just fine. Glad to hear that MS has finally seen the light in this regard.

 

Steve

 

Maverick Embedded Technologies Ltd. Home of wAVR and Maven.

wAVR: WiFi AVR ISP/PDI/uPDI Programmer.

Maven: WiFi ARM Cortex-M Debugger/Programmer

https://www.maverick-embedded.co...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Might consider comparing USB CDC implementations between AVR GCC and IAR EWAVR.

Don't know if Alan is OK with different IDE between bootloader and application.

 

https://www.iar.com/iar-embedded-workbench/#!?architecture=AVR

(expand "Download a free trial!")

...

Restrictions to the Kickstart, size-limited evaluation

  • A 4 Kbyte code size limitation.
  • Source code for runtime libraries is not included.
  • No support for MISRA C.
  • Limited technical support.

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

gchapman - you have lost me, what would be the benefit?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

(EWAVR vs. AVR GCC) reduced sizing of program and data space.

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:
Maybe the USB isn't worth it and I should try the USART RX/TX route on one...
One of your future competitors may be Microchip Technology and its PICkit 4 if (or when?) AVR are added to it.

Possible operator mistakes by misplacing or forgetting to pack :

  • PICkit 4 - SD memory card
  • autoprogrammer - USB UART cable

​You'll be able to compete against Microchip on price, cost, and value (reliability, security, functionality like internal storage and etc)

Price may be reduced when there's a follow-on to mega32U4 (megaAVR 1-series?)

 

Microchip Technology Inc

Microchip Technology

MPLAB PICkit 4 In-Circuit Debugger - pg164140

http://www.microchip.com/developmenttools/productdetails.aspx?partno=pg164140

...

Programmer-to-Go (PTG) support*

  • SD card slot to holds program data
  • Press on the logo to program the target

...

* This functionality is coming soon with firmware update of the product through MPLAB X IDE. 
...

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That is interesting...  I thought about using something like an SD instead of EEPROM originally, but since mine is product powered, I wanted something that could work under different voltage ranges without having to deal with translation/step up/step down/other power source.

 

I love the simplicity of the RX/TX, but the flexibility of the USB.  What I don't like is the complication of extending my existing protocol/command set to support USB, and the pc app having to determine which mode a programmer is in and/or trying to get the programmer in the other mode, etc.  Maybe the hybrid keep it simple approach is, if you hold the button while plugging in a programmer, then it stays in the bootloader for update mode.  I can then add a small button to my pc app that allows you to pick a firmware file and get it done easily.  After an update if the app code is good, it can go ahead and run it.

 

Question - when you invoke a reset using a watchdog reset - does this fully clear SRAM?  I think it does, but maybe I am wrong?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:

Question - when you invoke a reset using a watchdog reset - does this fully clear SRAM?  I think it does, but maybe I am wrong?

In most MCUs  SFR's preset during reset, but RAM is left alone. Usually, a compact loop is used to clear ram as part of SW init.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:
What I don't like is the complication of extending my existing protocol/command set to support USB, ...
PJRC Teensy HalfKay protocol is simple because the HalfKay bootloader is minimal essential and over USB HID; Teensy loader is likewise simple (one C file)

USB CDC ACM should be akin to a UART; it's like a UART from a read of the Windows 10 USB CDC API.

By Python is similar in pySerial.

alank2 wrote:
when you invoke a reset using a watchdog reset - does this fully clear SRAM?
No; only I/O registers are initialized.

 


https://www.pjrc.com/teensy/halfkay_protocol.html

https://github.com/PaulStoffregen/teensy_loader_cli

via https://www.pjrc.com/teensy/loader_cli.html

https://pythonhosted.org/pyserial/index.html

 

"Dare to be naïve." - Buckminster Fuller

Last Edited: Mon. Apr 23, 2018 - 03:57 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Who-me wrote:
Usually, a compact loop is used to clear ram as part of SW init.
From the C11 standard in 5.1.2 Execution environments :

... program startup occurs when a designated C function is called by the execution environment. All objects with static storage duration shall be initialized (set to their initial values) before program startup. The manner and timing of such initialization are otherwise unspecified. 

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:
if you hold the button while plugging in a programmer, then it stays in the bootloader for update mode.
You just designed HWB/HWBE right there! (I'm guessing Atmel were one step ahead on this).
Who-me wrote:
a compact loop is used to clear ram as part of SW init.
Not in the case of avr-gcc. The _do_clear_bss() does as the name suggests and ONLY clears the BSS area for the programs own allocation. CodeVision on the other hand clears all of RAM and THEN sets up .data

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:

You just designed HWB/HWBE right there! (I'm guessing Atmel were one step ahead on this).

 

:) Way back when when I was choosing where to put the button, that would have been a good idea!!!

 

I usually have the bootloader do an application crc test so it always runs, the button just chooses whether it will execute a good application if one is found or whether it will stay in the bootloader waiting for an update.

 

Thanks,

 

Alan

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:
Is there minimal code for a USB virtual serial port that would run fine in a bootloader?
Does it have to be virtual serial?

Reasons :

  • USB HID to Atmel-ICE for programming (an example of usage)
  • simple USB HID source code :

USB initialization problem

by avelino

https://www.avrfreaks.net/forum/usb-initialization-problem

and reference to the PJRC USB megaAVR stack (HID, CDC) within the LUFA documentation.

USB HID can run on a ATmega32U4RC (no factory USB DFU bootloader, no crystal, 8MHz RC, PLL to get 48MHz for low-speed USB)

The operating system API is not UART-like and there may be a gotcha for macOS.

There's Python for Windows USB HID.

 

http://www.fourwalledcubicle.com/files/LUFA/Doc/120219/html/_page__alternative_stacks.html

https://www.pjrc.com/teensy/rawhid.html

http://www.namesdir.com/mirrors/nongnu/avrdude/AtmelICE-kext-for-High-Sierra.readme.txt

https://github.com/rene-aguirre/pywinusb/wiki/Introduction#in-detail

https://pypi.org/project/pywinusb/

 

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think I'm going to drop the virtual serial and just attempt a simple HID type thing with 64 byte packet sending/receiving.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

alank2 wrote:
... the button just chooses whether it will execute a good application if one is found or whether it will stay in the bootloader waiting for an update.
The operator could single tap the reset button for application or double tap for bootloader per the following USB CDC (4KB) bootloader :

Pololu Robotics and Electronics

Pololu

9. The A-Star 32U4 Bootloader

https://www.pololu.com/docs/0J61/9

...

 

Startup logic

...

If the RST line goes low once, the user program will run after a 750 ms delay. If the RST line goes low twice within 750 ms, then the bootloader will run. (This behavior is the same as on boards like SparkFun’s Pro Micro.)

The start-up logic of the A-Star 32U4 Bootloader is shown in the flowchart below:

...

"Dare to be naïve." - Buckminster Fuller