Bootloader on RS232 - page write time and baud rate

Go To Last Post
16 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi all,

Sorry about the novel. My questions are summarised at the end.

I want to use a bootloader for may MEGA128 for flash programming. I already have an RS232 connection for data collection and sending simple commands to the MCU and would like to use this cable for updating the program as well. I want to send the program in intel hex format because I can use the default .hex file, it has nice little checksums, and the extra characters shouldn't really be an issue. I also like the fact that no special program is needed on the PC.

Despite the large number of bootloader topics and bootloader examples (including a intel hex specific example from P. Manwaring in the project section), I still have some questions relating to speed and timing. I hope that I haven't missed anything obvious.

Basically, if I use a baud rate of 9600bps and send 128kBytes in intel hex, I will have 43 characters per 16 bytes of information, where each byte has a frame of about 10 bits and will therefore need (128k/16 x 43 x 10b/Frame)/9600 b/s = (344 kFrames x 10b/Frame)/9600 b/s = 358 seconds.

I could possibly deal with 6 minutes, as I do not need to program often and instead of running outside with my laptop and MKII, I could go and get a coffee; however, if I could increase the baud rate to 230kbs, this would be better and if I was forced to reduce the baud to 2.4kbs, it would suck completely. So how do I allow for a high baud rate?

Given that my PC interface just sends the whole .hex file of ascii characters to the MCU, the minimum time between each character is determined by the slowest part of my bootloader routine.

Using GCC, it appears I only have the option of programming a whole flash page at a time. If so, I have no idea how long this page write operation takes. In the MEGA128 manual, table 111 indicates that up to 4.5ms are required for a page erase and write. I have been unable to follow the maths, but if even 1ms is required between sending characters, I can forget the whole thing.

So...

does it really take in the ms range to write a page of flash memory?

Is there a way of writing to flash one word at a time - thereby reducing the min time between characters by a factor of up to 128?

Can I use interrupts and a second page buffer to read incoming characters while I am carrying out a page write?

Is it recommended to use interrupts at all in a bootloader?

What happens if I am writing to a page in the NRWW section and relying on interrupts? Do I have to choose between being able to write to the NRWW section OR using interrupts in my routine?

As always, any help or ideas much appreciated!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
does it really take in the ms range to write a page of flash memory?
~3.5-4.5ms any time you write a page, set by the internal rc oscillator (fixed number of cycles).
Quote:
Is there a way of writing to flash one word at a time - thereby reducing the min time between characters by a factor of up to 128?
You can write 1 bit at a time if you wanted to. But it takes the same time (per write) no matter what (see above). Writing one page of data is fastest of course, since you have less writes.
Quote:
Can I use interrupts and a second page buffer to read incoming characters while I am carrying out a page write?
Unless you can buffer the whole hex file (or a major portion of it anyway), it won't do any good, as you still have to wait at some point. Since you need to wait, you have to deal with that (PC side), so why buffer (more than a record or two).
Quote:
Is it recommended to use interrupts at all in a bootloader?
I wouldn't. Others may.

You simply need to use a line delay of ~20ms for what you want (since no flow control). Depending on how you do it, you have to allow for the fact that you can end up crossing pages when writing a record, and would then need 2 writes to flash for the record. I end up using 17ms line delay just from trial and error (lowest possible without ever an error).

The big time gain that can be made in the ascii/hex method, is getting the hex records to match the size of SPM_PAGESIZE (reduce number of writes/line delays)-
https://www.avrfreaks.net/index.p...

My ascii/hex bootloader-
http://www.mtcnet.net/~henryvm/A...
I have been using for about a year or so. I like it and use it all the time, and I use terratermpro for my terminal. I'm modding it to handle >64k avr's and am working on an app to spit out a bootloader hex file (no compiling, just select options wanted), but its low priority.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Many thanks for your reply, curtvm!

So if I write one bit, it takes the same time as writing a page? I suppose I will be writing page by page then.

Using the line delay to wait for the page write, so that the baud rate does not have to be ridiculously slow is a great idea. Can I decrease the line delay by erasing all the pages before reading the hex file in, so that I only have to wait for the page write commands?

I don't suppose there are any GCC commands that can change the hex output to something more practical?

I have started doing some file manipulation using Octave, and figure I can make the .hex file into anything I want, but whether it is worth the trouble depends on how fast I can get the standard intel hex file sent.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
So if I write one bit, it takes the same time as writing a page? I suppose I will be writing page by page then.
You are always writing a page no matter what. Whatever the temp page holds, will be written. If all the bits in the temp page are 1 except for 1 bit which is 0, then even though the whole page is written, only 1 bit of flash will be changed (assuming its not already 0). That's the way flash works, and can be used to your advantage.

Quote:
Can I decrease the line delay by erasing all the pages before reading the hex file in, so that I only have to wait for the page write commands?
I require an erase before programming, so in my case I don't have to erase/write for each record.

Quote:
I don't suppose there are any GCC commands that can change the hex output to something more practical?
Unless you want to recompile avr-objdump (you probably don't), you either figure out srec (it may be able to), or simply write a script in your language of choice. I did the latter, only to test my theory and to implement encryption. Its not that hard, but is not used daily because I only deal with 16k or less. If I was using a large avr, it would probably be worth the effort if using the bootloader frequently.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

My bootloader uses the per-line format of the hex file to my advantage. The bootloader prompts the host for each line with either a "@" meaning "give me the next line" or "!" meaning "the last line's CRC didn't work or is otherwise bad, give it to me again". A simple TeraTerm script can be used to send the hex, or it can be written into the download program supporting the device.

In fact, if you go looking, there's an XMODEM bootloader somewhere. This would do a similar sort of prompt-response protocol with built-in checking. In hind sight I wish I had used this bootloader as many terminal emulators have an XMODEM file transfer mode built in.

BTW, my serial I/O currently runs at 56 KB but I have run this same protocol at 115 KB with no problems.

Stu

Engineering seems to boil down to: Cheap. Fast. Good. Choose two. Sometimes choose only one.

Newbie? Be sure to read the thread Newbie? Start here!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You can use the output of the bootloader as flow control, as Stu does, but that means the PC side has to be a little smarter than 'dumb'. I have sort of the same setup, where I get a command prompt after each record. The command prompt character determines what I'm able to do, along with an error character at the end of a bad record or usart error. I have tried using these with a 'smarter' terminal ('flow control'), and it does speed things up a little since the usual line delay value is for the worst case possible. In my case, I'm dealing with relatively small avr's, and is not worth the effort (to me). One thing you can do with a 'smarter' terminal, is check that the characters echoed back from the bootloader match the ones sent, which would be in addition to the record checksum. Although possible to get a good checksum with bad data, it would be harder to also have echoed characters screw up exactly as it did going into the bootloader.

I also do not like automatically retransmitting failed records (like xmodem for example), since I want to know about ANY failures. I never get transmission failures, so if I happen to get one, I want to know about it (somethings wrong, or is failing). Transmitting over a phone line is a different story of course, and if using the phone line to do updates, I would be in favor of something like xmodem.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'd go with Xmodem too if given a free choice. It's nice that the payload in the 132 bytes packets is 128 bytes which is either the page size or a nice binary multiple of it for the AVR. Obviously use Xmodem/CRC rather than Xmodem/checksum for better data integrity

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
In fact, if you go looking, there's an XMODEM bootloader somewhere. This would do a similar sort of prompt-response protocol with built-in checking. In hind sight I wish I had used this bootloader as many terminal emulators have an XMODEM file transfer mode built in.

That's a great idea.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Xmodem sounds good.

Does this mean that I send a straight binary file to the MCU?

If so, is there an easy way to generate this using GCC on the command line?

Does the xmodem terminal program automatically add the header and checksum?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You can change a single line of a standard MFile-generated makefile to switch the output format from Intel Hex to raw binary. Note that the file extension doesn't change by default, but the file's contents will change format.

(If you have planned for any "holes" in your flash file, though, you'd be well advised to double check first to see how avr-objdump deals with those holes - does it fill them with 1's, or fill them with 0's, or silently omit them entirely? The first possibility would be ideal, the second possibility might potentially wear out your Flash more quickly depending on how intelligent your bootloader is at detecting unchanged pages, and the third possibility would most likely lead to incorrect functionality. If you needed to guarantee the way holes were filled, then you'd be better off leaving avr-objdump's output in Intel Hex format, and converting to raw binary using SRecord, which gives you the option of specifying exactly what default value to assign to holes.)

When coupled with the Xmodem protocol for error detection, you should be perfectly safe to use the raw binary format to optimize transfer speeds.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

At the moment I am using kontrollerlab with linux and do not have a makefile as such, but putting stuff into a text field called 'compiler instructions' or something has worked in the past.

Any idea what that line of code was exactly that creates the bin file? I have previously searched the GCC manual without any luck and have looked in the lib-c manual as well.

Can I put the binary file into the x-modem terminal and expect it to send the frame header and crc bits for me?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
Can I put the binary file into the x-modem terminal and expect it to send the frame header and crc bits for me?

Yes, any reputable XMODEM file transfer program will automatically insert the headers and checksums/CRCs (depending on receiver capability) for you.

Quote:
do not have a makefile as such, but putting stuff into a text field called 'compiler instructions' or something has worked in the past.

It would not be an instruction to the compiler, but rather an instruction to the avr-objcopy utility. This stage is run after the compiling and linking stages have both already finished. I've never actually used kontrollerlab, so I don't know if it would be possible to customize it for this purpose.

It might be easier to use SRecord to post-process the hex file into a raw binary file. SRecord is available as a package in several Linux distributions. I won't pollute this thread by getting into details unless you ask.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks heaps. I just downloaded SRecord and it appears to be just what I'm after.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
you'd be well advised to double check first to see how avr-objdump deals with those holes
when avr-objdump outputs a binary image (with -O binary), there is no address info except for the byte location within the file, so any 'holes' have to be filled with something (its 0 in this case) and the binary ends at the last byte produced.

(I use binary output when creating my own 'big' hex records, or encrypted records, since it means no decoding of a hex record is needed, only encoding)

Quote:
the second possibility might potentially wear out your Flash more quickly depending on how intelligent your bootloader is at detecting unchanged pages
I could be wrong, but I wouldn't worry about wearing out flash, unless your name is Bob, and you live in Florida, use a Mega32, and flash at least 10 times a day on the same Mega32 for about 3 years.

Actually, even a 'smart' bootloader will need to erase the first page on a mega128 probably 99 times out of 100, since the copy data routine will be using a different data load address any time the size of the app code changes. Since a bit needs changing, an erase is needed. A 'super-smart' bootloader that can also determine when only bits need to go from 1->0, would probably help only a little. Since the first page(s) will probably wear out first, it will not matter that the other pages have some life left in them. That's my theory. (Eeprom is a different story, and I think IS worth the little effort needed to reduce the number of erases).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Curt wrote:
Quote:
the second possibility might potentially wear out your Flash more quickly depending on how intelligent your bootloader is at detecting unchanged pages
I could be wrong, but I wouldn't worry about wearing out flash, unless your name is Bob, and you live in Florida, use a Mega32, and flash at least 10 times a day on the same Mega32 for about 3 years.
Not that you have anyone in mind, though. Lucky for you, that mythical Bob uses ImageCraft and isn't likely following this forum. :wink:

Stu

Engineering seems to boil down to: Cheap. Fast. Good. Choose two. Sometimes choose only one.

Newbie? Be sure to read the thread Newbie? Start here!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

curtvm wrote:
Since the first page(s) will probably wear out first, it will not matter that the other pages have some life left in them. That's my theory.

Point well taken.