Bootloader updating

Go To Last Post
14 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I enjoyed the thread that Balto had going about using an existing SPM in the BL section to update the BL section.

 

I've read that some have suggested updating an available BL area and then using that to update the entire BL, but that would count on the BL section having some available room.

 

If all that is needed as SPM/RET, couldn't a bootloader be written to make sure there is this opcode pair combination on every page in the BL?  Let's take a 4 page small BL for example, you would then have 4 opportunities to call SPM in the BL, right?

 

A fully automated process to update both the BL and APP sections might go something like this:

 

Apparently indenting does not work, wrapping this in a code section instead:

 

#1 Trigger the bootloader in the device and power it (into bootloader mode)
Query the BL to get both the BL and APP versions.
If the BL version is old:
	Use the existing BL to flash an APP that is not the APP, but a program to update the BL (version on it set to 0.00)
	Reset it to allow the APP to run and update the BL.
	Wait the appropriate amount of time or make the APP indicate when it is done, then reset and start at #1.
If the AP version is old:
	Use the BL to update it.
	When done, then reset and start at #1.

The BL updater APP could simply read each BL page and flash it if it does not match what it needs to flash it to.
	It could run multiple times and only flash what is necessary.
	It could be smart enough to scan the BL area and look for the SPM/RTN combination so that if one page was flashed badly, it would still have a way to proceed by using SPM/RTN pairs from another page that still works.


 

Any thoughts on whether this would work or not?  I know a BL should be bug free from day one, but it would be cool to have a way to update it if need be...

Last Edited: Wed. Oct 11, 2017 - 05:39 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

couldn't a bootloader be written to make sure

 I think it's relatively easy (if perhaps not "small") to make your bootloader updateable if you PLAN it that way.  the previous thread was about doing so ... without cooperation.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I agree; it was a fascinating thread to follow.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

While this is "interesting", it is a Seriously Bad Idea ™.  It can only work if you have poorly chosen BLB1 mode bits i.e. allowing SPM to the BLS.  Why on Earth would you ship such a device?

 

This has been addressed in the past:

http://www.avrfreaks.net/comment/1249436#comment-1249436

https://hackaday.com/2014/07/05/overwriting-a-protected-avr-bootloader/

 

It only worked because the author did not program the appropriate lock bits for the BLS.  Were BLB1 mode 2 or 3 in effect (as it should be), this technique would not work.

 

 

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"Read a lot.  Write a lot."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

What would be your recovery strategy if power or comms failed half way through replacing the bootloader?

 

Even for normal bootloader use you should always have this thought in your mind - can this system recover after any kind of interrupt at any point in the process.

 

It's why the bootloader is the only piece of code in the device that must be 100% fault free (and tested to prove this) when the product ships. Anything else in it can be replaced / repaired (robustly!) but I would leave the bootloader well alone - just get the program (as simple as possible) working 100% when you ship out and then there'll never be any reason to consider changing it anyway.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I agree with Cliff. I've written several bootloaders in the past, and always made sure that they get a look in at (external) reset, and are in a protected memory region. I've never wanted any method of invoking the bootloader from the main app. If there were ever any failures, I wasn't informed of them, and the total unit count across 3 or 4 products must exceed 200,000.

 

Having said that, I also found the thread Alan refers to interesting. I think a lot of firmware programmers are fascinated by tricks of that nature, especially those of us of a certain age, who had to cope with slow processors and tiny memories. Does anyone remember the "compare immediate" trick with the 6800? Some method of saving a single byte, by either branching to the operand byte of a CMP # and executing it as an instruction, or executing the compare, which only affected flags? The actual purpose escapes me right now...

 

Quebracho seems to be the hardest wood.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

John_A_Brown wrote:
Does anyone remember the "compare immediate" trick with the 6800? Some method of saving a single byte, by either branching to the operand byte of a CMP # and executing it as an instruction, or executing the compare, which only affected flags?
Don't remember that one but a "clever trick" I do remember (as I broke compatibility of a lot of Sinclair Spectrum games in later models where I changed the firmware) was a Z80 "trick" that relied on the fact that Z80 had three interrupt modes. "IM 2" was one in which the target address of the vector was formed by the contents of the I register and any value that happened to be on the databus at the time of the interrupt to index into a table of vectors. If you made a table that had all the same values (0xFFFF) then you could guarantee the target of the interrupt jump whatever the data bus held at the time. As explained more clearly here:

 

http://www.breakintoprogram.co.u...

 

Games programmers used the fact that there was an "unused" block of 256 0xFF bytes at the "end" of the Spectrum ROM to save them having to create a table in their own (limited) code space. (they set I = 0x39)

 

Guess which dummy came along to the spectrum firmware (which was now going to use four bank switched ROMs rather than just one) and saw some "empty space" at the end of the original ROM where he thought it would be possible to add the ROM bank switching routines that "no one would notice".

 

How I wish I'd known about the "IM2 trick" before I did this. Turns out that about 30..40% of existing "Spectrum 48K" games were using it! (they being the ones I broke the operation of).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It depends on whether the bootloader is the only way to update, or a way to update.  If it is the only way, then yes, it needs to be perfect and those boot lock bits need to protect it.  If it is a way to update, but you have alternatives if the worst happens, then with an updatable bootloader you can not only fix something, but add more features if you want to.  It gives more options though it is riskier.  Sure you probably should have foreseen any features you might want, but that doesn't always happen.  To answer your question cliff about what if it fails - this is why I was suggesting the SPM/RET combination be in multiple pages of the bootloader firmware.  If one page fails, there will still be what is necessary around in another page to fix it.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

alank2 wrote:
but you have alternatives if the worst happens, then with an updatable bootloader you can not only fix something, but add more features if you want to.
Go on. If you have "alternative" ways (which I take to mean ISP or PDI or JTAG or TPI or something) then if the bootloader needs replacing why don't you just use that? What is the point of going through dangerous hoops to make a secondary bootloader attempt to update the primary bootloader.

 

Also if you have "alternatives" how do you avoid them blitzing the existing bootloader when they are used anyway? Most JTAG/ISP/etc is going to start with "Chip Erase" and there goes the existing bootloader. So presumably, when these "alternatives" are you used you are actually going to be programming a combined binary of app code and a snapshot of the bootloader to "put it back" anyway ?

alank2 wrote:
this is why I was suggesting the SPM/RET combination be in multiple pages of the bootloader firmware. If one page fails, there will still be what is necessary around in another page to fix it.
I fear you miss the point. A bootloader is a lot more than just "SPM;RET". There is some comms software (usually) and there is a "control loop" that steps through the pages (after reception) and keeps calling the SPM;RET. To get an entire app or even bootloader replaced you need ALL of that stuff intact. You might well have multiple copies of SPM;RET to mitigate interruption but how does the control loop know to call copy 3 not copy 2 because copies 1 and 2 have been blown away.

 

Also bear in mind Joey's point. Any "robust" bootloader is going to have BLB bits that prevent any accidents that might over-write itself (such as communication noise that corrupted the target address for a page write etc).

 

From time to time we see threads here about people exceeding 4K of bootloader code or worrying about IVSEL because they wanted to use interrupts in a bootloader or how to integrate the picture/font for their GLCD into the bootloader to put a picture on the screen to keep the user "entertained" while the bootloading is happening. All these folks have completely missed the point. A bootloader should be a smal and extremely simple comms and page control loop with an SPM and nothing more. With the possible exceptions of USB or CAN as the comms channel I have no idea why interrupts would ever be required. It should be small and simple so that unit testing can perform 100% coverage and you can test it for every conceivable eventuality.

 

What I've also done in the past, 10..20 years ago, is as soon as we make prototypes of some new design we give everyone in the company one to simply use and we only ever update their code via bootloading (no JTAGing or anything like that) so that the bootloader gets as much exposure to "real life conditions" as possible because, like I said, it's the only bit that must be 100% right on day one and will never have any opportunity to change.

 

(oh and if the user needs "entertaining" for the 30 seconds every 6 months when the device is powered on and starts to bootload then just flash an LED for them or something to show "signs of life" - don't go putting HD44780 or GLCD into a bootloader just to say "be patient and wait 30 seconds while I do this")

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
I fear you miss the point. A bootloader is a lot more than just "SPM;RET". There is some comms software (usually) and there is a "control loop" that steps through the pages (after reception) and keeps calling the SPM;RET. To get an entire app or even bootloader replaced you need ALL of that stuff intact. You might well have multiple copies of SPM;RET to mitigate interruption but how does the control loop know to call copy 3 not copy 2 because copies 1 and 2 have been blown away.

 

The concept is an application "payload" that is designed to completely update the bootloader.  It would know which pages need to be flashed and have the content to flash them with.  All it needs is a way to run SPM.  Once it has done its job, the bootloader would replace it with the proper application code.  The idea is that if SPM/RET was purposefully put on multiple pages of the bootloader, then even if a page flash goes wrong, it still has others it can use.

 

It would have to be smart enough to LPM the bootloader and find where valid SPM/RET fragments are present.  When it finds a valid one, it knows which one to call.  A stack frame can then be created to use a RET command to call the SPM/RET and return back, right?  The original bootloader could be designed to always include the SPM/RET combination on every page of the bootloader flash so one would always be available.

 

I don't know if it can also set the SPMEN before using RET to call SPM without it taking more than the 4 cycles, or if it really needs to find where SPMEN is set, then SPM, then RET and call where SPMEN is set instead of SPM.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Sounds a bit like trying to determine a solution for a problem that doesn't really exist. But, yeah, an interesting intellectual exercise - probably worth a bit of experimentation. But to test it for robustness arrange some kind of "test harness" to invoke the process a few thousand times and then randomly have it disconnect power and some randomly chose number of milliseconds after each invocation.

 

Is the device still alive at the end of that? If yes then you can probably consider it "viable".

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The concept is an application "payload" that is designed to completely update the bootloader.

But... now you have a bootloader that depends upon an application.  That's back asswards.  What if the bootloader update fails mid-way, say due to a power failure?  The now-broken bootloader will be the first piece of code to run after power is restored.  With a broken bootloader, you can't guarantee that it will vector to the app space to re-run your bootloader-update payload app, even if that app could detect the condition and correct for it.

 

I have visions of cartoon vacuum cleaners!:

 

 

How appropriate that he's trying to catch a 'bug', yet he succeeds only in sucking everything else, including himself and his 'bug catcher', into a white oblivion.  Note that the RGB hex code for white is 0xFFFFFF ;-)

 

Fritz Freleng must have been a firmware programmer on the side ;-)

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"Read a lot.  Write a lot."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

Last Edited: Wed. Oct 18, 2017 - 03:34 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

joeymorin wrote:
What if the bootloader update fails mid-way, say due to a power failure?  The now-broken bootloader will be the first piece of code to run after power is restored.  With a broken bootloader, you can't guarantee that it will vector to the app space to re-run your bootloader-update payload app, even if that app could detect the condition and correct for it.

 

That is a very good point.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Does anyone remember the "compare immediate" trick with the 6800?

I believe I do.  I've never used it, but I recall scratching my head over a piece of disassembled code found (IIRC) in the ROM for the CoCo's Microsoft BASIC (or Extended BASIC, or DOS BASIC,... I can't fully recall).  Quite the 'aha' moment when I realised what was going on.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"Read a lot.  Write a lot."

"We see a lot of arses on handlebars around here." - [J Ekdahl]