LUFA DFU Bootloader Help

Go To Last Post
5 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Board and Setup
Before I tell you my problem, I'm using the 32u4, and the HWBE fuse is set, while BOOTRST is not. If you want to know other fuses, LOW is 0xFF, HIGH is 0xD0 and EXTENDED is 0xCB.
I should also mention I'm using the Arduino Leonardo, I do have a custom board which is running my application(haven't tried bootloader), but there is a small issue on board requiring a custom adapter for it to work on USB, and because of that it disconnects at random, so it's just easier to make sure everything connects on the Leo first.

The Leonardo also ties HWB pin low with a 10k resistor.
And lastly, in the function void Application_Jump_Check(void), specifically for the Leonardo, it checks to see if IO13 is pulled low, and if not it starts the app, I do have it pulled low.
 

The Problem
I've been trying all day to get the DFU bootloader with no luck, it doesn't even seem to try and connect to the PC.
After I flash the Bootloader, nothing happens, it does not try to connect to my PC, and vise versa. Not even a Descriptor Failed Request Error.

 

End Goal

I eventually want to get the device to boot to the application first, so the user has regular functionality of the device from the get go.
But if the user runs my Firmware Update Software(which I will need to make), it will send a command via USB telling the device to load the bootloader for programming.

The device will then load up the bootloader, and the user can then choose to update to newest, or they choose to update to an older version manually, if they choose to do so.
The device absolutely cannot have a switch that pulls HWB low or high, it has to be done all in software.

 

Current Goal

All I want to do right now, is have the bootloader connect to the PC, and show up in device manager, as long as I can get it to do this, I can get the rest of the way.
I am having success with Atmels DFU Bootloader, although if I want to go to my application, I need to use FLIP to manually start the application, which is completely useless to me.

 

Some possibilities?

1.

I thought maybe the bootloader is just trying to go straight to the application, but since I don't have one flashed, nothing happens.
So for the last half hour I've been trying to merge hex files, I've tried 2 methods, just using a text editor, and then using srec_cat.exe.

 

Maybe someone can confirm if I'm doing this correctly.

 

Text Editor Method: 

I open up both hex files in notepad++, I remove :00000001FF from the end of the bootloader, and then simply paste the application to the end? Someone said this in another post on the forum, although they did say basically, so its possible they left out some instructions.
Then I flashed the MCU in Atmel Studio via ISP, giving me this error during the flashing process.

 

Verifying Flash...Failed! address=0x1000 expected=0x55 actual=0x00

The device is now actually trying to connect, but it gives me a "Device Descriptor Request Failed." I didn't expect it to work, but its worth noting that its trying to do something.
 

Srec_cat.exe Method:

I'm using this command 

 

srec_cat DFU_Bootloader.hex -I CLASS_KEYBOARD_MOUSE_LEONARDO.hex -I -o Merged.hex -I

but I get this error, which is almost exactly the same as the text editor, except I get this while running srec_cat, and it isn't giving me a hex.

 

srec_cat: CLASS_KEYBOARD_MOUSE_LEONARDO.hex: 257: contradictory 00001000 value
    (previous = 55, this one = 80)

 

2.

And then here is another thing that may be making the bootloader try and go straight to the application.

 

Over in this topic,  https://www.avrfreaks.net/forum/atmega32u4-hwb-issue , the user Billy mentions something about BOOT_START.

He specifically says

using 0x7000 as the BOOT_START value

 The only value similar to it is BOOT_START_ADDR, and its defined in assembly, which I have no experience with.
I also read another post somewhere else which I cannot seem to find again, but they mentioned something about needing to set a BootKey or something, to 0x8000, to get the bootloader to work on the Arduino Leonardo.

Other Bootloader Questions
What is the purpose of the HID bootloader, and can you flash your application with it?

Edit1:
Giving the new info from clawson, I merged the bootloader and application(with Notepad++, srec_cat.exe still gave me same error), this time with the application first.

I then flashed the device, and it is connecting to my PC, and I can confirm it is going straight to the application.
I should also mention that I could only get it to upload, if I didn't remove :00000001FF from the end of the application, so could it be ignoring the bootloader completely? I read that you are supposed to remove that.
So I think this means something is happening in the bootloader, and I have a feeling it has something to do with the bootkey I read about.
Does anyone know about the bootkey, I found the place I read about the bootkey, here is the link.  https://github.com/abcminiuser/lufa/issues/42
If you don't want to go to the URL, here is the post, from NicoHood.
 

To add leonardo compatibility you definitely have to set the bootloader key to exactly 0x0800.
Thats how i did it with a 8 bit key and another address.
https://github.com/NicoHood/HoodLoader2

Its not perfect, it would be better to define the specific address as reserved but therefore you need compiler flags and the arduino ide wont take these flags. I also filed a bug there too.

 

Edit2:
I may have spoke too soon,  I did not read the whole issue in the URL I had posted in Edit1. 
He seems to be using the Arduino IDE to upload his sketches, which I am not, so this should not affect me.

Last Edited: Mon. Jun 24, 2019 - 11:17 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

LittleRain wrote:
and then simply paste the application to the end?
Surely that's the wrong way round - the bootloader goes on the end of the app , not the app on the end of the bootloader.

 

Can't help thinking you are jumping ahead anyway - don't worry about including the app at first. The whole point of a bootloader is that it should be complete and standalone and able to do its job with no app code present. So just program only the bootloader then see if you can use Flip to deliver the app to it.

LittleRain wrote:
it will send a command via USB telling the device to load the bootloader for programming.
This is a terrible idea! Consider the situation where you send the app the command to trigger bootloadeing. It starts receiving the new code and erasing / replacing the app code but then the power dies or someone trips over the USB cable or something. So now you don't have a complete/coherent app in the device. Except that you have programmed it so that the app has to be there to trigger bootloading. I guess as long as you have a button on HWB so the user has a "get out of jail free" button you may be OK, but any bootloading system that depends on the app being present is "dangerous".

What is the purpose of the HID bootloader, and can you flash your application with it?

Dean was just showing you what is possible with all the example bootloaders. In reality almost all bootloaders will actually use DFU or CDC-ACM. The former needs a "special program" to deliver app code ("Flip") while CD-ACM just needs any standard program that can send serial data (a "terminal"). His other bootloaders then start to get clever. HID is Human Interface Device but at the end of the day (like all USB classes) it's just "get some binary info from over here to over there". For HID you would need to write some special software on the PC end to feed the data to the link. Some of his other examples are really quite novel. The MSD one makes the USB AVr look like a memory stick so you can just drag and drop a binary file to the "drive" that appears in Windows/Linux and the code will get programmed into the application section. I actually like this a lot - it's about the most natural interface you can offer and end user. You don't even need to install a Terminal or DFU software (or write HID software). You just drag files to it. He also (totally whacky!!) has a "Printer" bootloader. For this one you "print" the application code to it and it gets programmed. Of course trying to arrange to "print" a firmware.bin is not going to be that easy and you may need to write PC software to achieve this. I seem to remember he may even have a MIDI based one - you send it "musical notes" (well really MIDI "Sysex") and it gets programmed. Again this is probably not that easy to achieve without writing some special PC software.

 

So DFU (Flip), CDC-ACM (Terminal) and MSD (drag file) are probably the best/easy ones as the PC side is already supported in various ways. For HID, Printer, MIDI you would likely need to write some PC side USB code to make use of them.

 

PS just checked:  https://github.com/abcminiuser/lufa/tree/master/Bootloaders - I was wrong to remember there being MIDI - I guess I must just have thought myself that MIDI would be another way to do it!

 

PPS just reminded myself that the Mass Storage one is 6K so you need an AVR-USB with an 8KB boot section -that's probably the reason it's not such a widely used choice.

Last Edited: Mon. Jun 24, 2019 - 08:42 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:

Surely that's the wrong way round - the bootloader goes on the end of the app , not the app on the end of the bootloader.

Can't help thinking you are jumping ahead anyway - don't worry about including the app at first.


Ok well that is good to know.

I was just trying to merge them to see if its jumping straight to the app, but I totally agree with you, I need to get the bootloader working without the merge.

Giving the new info, I merged them, and flashed the device, and it is in fact going straight to the application.
I should also mention that I could only get it to upload, if I didn't remove :00000001FF from the end of the application, so could it be ignoring the bootloader completely?
So that means something is happening in software, and I have a feeling it has something to do with the bootkey I read about.

clawson wrote:

LittleRain wrote:
it will send a command via USB telling the device to load the bootloader for programming.
This is a terrible idea! Consider the situation where you send the app the command to trigger bootloadeing. It starts receiving the new code and erasing / replacing the app code but then the power dies or someone trips over the USB cable or something.

I guess as long as you have a button on HWB so the user has a "get out of jail free" button you may be OK, but any bootloading system that depends on the app being present is "dangerous".


I actually did think about that, but I couldn't think of a way where the user would be able to plug in the device, start using it right away, and then if they decide to update at a later time, run the software on the computer without resetting the device. The reset button wont be exposed without taking the device apart, so it makes it a bit trickier.

I wanted to stay away from the user needing to do anything with buttons/switches, but that's a really good idea, at least if there is a backup option it should be alright. But I'm totally open to any other suggestions.

clawson wrote:

 So just program only the bootloader then see if you can use Flip to deliver the app to it.

FLIP will not connect to the LUFA bootloader, like I said its not connecting to my PC at all. The Atmel DFU bootloader does though, and I am able to program with flip.

clawson wrote:

while CD-ACM just needs any standard program that can send serial data (a "terminal").

His other bootloaders then start to get clever. HID is Human Interface Device but at the end of the day (like all USB classes) it's just "get some binary info from over here to over there". For HID you would need to write some special software on the PC end to feed the data to the link.

Some of his other examples are really quite novel. The MSD one makes the USB AVr look like a memory stick so you can just drag and drop a binary file to the "drive" that appears in Windows/Linux and the code will get programmed into the application section. I actually like this a lot - it's about the most natural interface you can offer and end user. You don't even need to install a Terminal or DFU software (or write HID software). You just drag files to it. He also (totally whacky!!) has a "Printer" bootloader. For this one you "print" the application code to it and it gets programmed. Of course trying to arrange to "print" a firmware.bin is not going to be that easy and you may need to write PC software to achieve this. I seem to remember he may even have a MIDI based one - you send it "musical notes" (well really MIDI "Sysex") and it gets programmed. Again this is probably not that easy to achieve without writing some special PC software.

Ah I get it now, I was wondering what the other bootloaders were for, I had thought DFU were the only ones capable of programming.

Serial would be very easy to implement, especially because I plan to write the software in C#, which there are native libraries for.

I've never written software that talks with a USB device, and I am always up for learning more, so I'll most likely go with the HID device. Also, I now see there is a python script that I could convert to C# as well.
My original plan was to call avrdude from my software, using FLIP for the programmer, but being able to do it completely without anything else(apart from libraries) is the best option for me.

As for the other ones, that is amazing! While extremely cool, they are as you said, totally whacky, and I don't think it would be the best option for a consumer product.

clawson wrote:

PPS just reminded myself that the Mass Storage one is 6K so you need an AVR-USB with an 8KB boot section -that's probably the reason it's not such a widely used choice.


My current board has an atmega32u4, but I plan on switching to the atmega8u2, where can I find out how big the boot section is?
My application is about 5k, and the bootloader is 2k, I'm assuming that is ok?

But anyways, thanks for all the info, but unfortunately it still doesn't fix my problem.
When I upload the bootloader on its own, nothing happens, no device in device manager, and FLIP cant connect.
That goes for all, and only the LUFA bootloaders though. 
The Atmel DFU, as well as the Arduino Leonardo bootloader ARE connecting.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0


LittleRain wrote:
, if I didn't remove :00000001FF from the end of the application, so could it be ignoring the bootloader completely?
For definite if you don't remove the '01' record hen that's exactly where the reading of the .hex stops and anything beyond will be ignored. Hex files should only have one '01' on the last line.
LittleRain wrote:
FLIP will not connect to the LUFA bootloader, like I said its not connecting to my PC at all. The Atmel DFU bootloader does though, and I am able to program with flip.
The first step here is how does it enumerate in Windows/Linux (if it even does). Remember that one complication of your design (app triggers boot) is that when power is applied to this device the app will run. It will enumerate as CDC or HID or MSD or whatever it is you have programmed it to do. Now you send the command to say "start bootloading" so now there needs to be a full USB reset. It has to "de-enumerate" as the original class and then re-appear as DFU. In fact this is the whole reason HWB exists. It's a "selector button" that says "do you want the AVR to enumerate as CDC/HID/MSD/etc or do you want it to enumerate as DFU". If it jumps to the bootloader (button held) it's DFU otherwise the app runs and it's the other class.
LittleRain wrote:
I had thought DFU were the only ones capable of programming.
Oh no the very idea of "bootloader" means a program that takes control at boot time (power on) and can decide to reprogram or just run the app"
LittleRain wrote:
so I'll most likely go with the HID device.
Then on the PC side you will want to familiarize yourself with "libusb"
LittleRain wrote:
the atmega8u2, where can I find out how big the boot section is?
All AVR datasheets for devices that have bootloader have a chapter (towards the end) about using bootload facilities and it has a table saying what the BOOTSZ options are. When I look at 8U2 I see:

I'm actually quite surprised. 8K micros don't usually allow for such large bootloader but when it is USB I guess they have to. The top option there is 2K words which is 4K bytes. That's actually half the entire chip !! Hopefully you can use the 1K word (2K byte) option to leave 6K for the app.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:

LittleRain wrote:
, if I didn't remove :00000001FF from the end of the application, so could it be ignoring the bootloader completely?
For definite if you don't remove the '01' record hen that's exactly where the reading of the .hex stops and anything beyond will be ignored. Hex files should only have one '01' on the last line.


Ok, kind of thought so.

I tried removing the ..01FF again, here is the error it gave me.

 

Timestamp:    2019-06-25 12:06:35.998
Severity:        ERROR
ComponentId:    20100
StatusCode:    131106
ModuleName:    TCF command: Modules:add failed.

Unable to parse objectfile E:\Projects\Game Controllers\Adapters\SNES_To_USB\Firmware\SNES_To_USB_Firmware\SNES_To_USB_Firmware\Debug\SNES_To_USB_Firmware_With_DFU_Bootloader.hex: Unsupported format.

clawson wrote:

The first step here is how does it enumerate in Windows/Linux (if it even does).


It's not enumerating at all(This is with the bootloader flashed on its own). The HWB pin is always tied low, which means it should jump straight to the boot loader.

clawson wrote:

Then on the PC side you will want to familiarize yourself with "libusb"


Downloaded the library a month or 2 ago ;)

 

clawson wrote:

All AVR datasheets for devices that have bootloader have a chapter (towards the end)...
 

I'm actually quite surprised. 8K micros don't usually allow for such large bootloader but when it is USB I guess they have to. The top option there is 2K words which is 4K bytes. That's actually half the entire chip !! Hopefully you can use the 1K word (2K byte) option to leave 6K for the app.

I also noticed last night that when you pick your device in atmel studio it shows boot size as well.

That's unfortunate how its 8k - 4k(or2k/other configs..), and not 8k + 4k, I might have to upgrade to the 16u2.
The hid bootloader is exactly 2192 bytes when optimized for size, so unless I can make it even more compact, I don't think the 2k boot will be an option.