How do I load large amounts of data when programming (exceeding flash)?

Log in or register to post comments
Go To Last Post
15 posts / 0 new
Author
Message
slow_rider
slow_rider's picture
Level: Posting Freak
Joined: Fri. Feb 16, 2007
Posts: 1280 View posts
Location: Israel
#1
Posted by slow_rider: Sun. Oct 22, 2017 - 12:29 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm using an Xmega with 128K of Flash memory. I also need to store about 256K of data that could be pulled by the code while the program runs. Right now I am trying to do this using a micro SD card but even at this point this seems like an issue because the way the card is being read and the data stored in FAT file system.

 

I would like to connect a serial flash (SPI) that will hold all this non-volatile data, question is - what is the best way to load it onto the serial flash. It seems a little far fetched I could someone program everything using the JTAG debugger at a single go. Loading the code onto the external flash is possible as a separate step but if there is a better way I would like to hear about it.

Tags:

Tools, Compilers and General Programming
Top
js
js's picture
Level: Moderator
Joined: Wed. Mar 28, 2001
Posts: 27822 View posts
Location: Sydney, Australia (Gum trees, Koalas and Kangaroos, No Edelweiss)
Posted by js: Sun. Oct 22, 2017 - 01:49 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Have a function in your code that reads the data from the serial port and store it into the serial flash.

 

That's what I have (in a smaller scale) with my electronic signs where you can download up to 1K of data from a text file, the function is only accessible at start up when entering a special character in the first few seconds followed by a password to get into, what I call, a monitor mode where that function is available as well as others to change parameters.

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

Top
slow_rider
slow_rider's picture
Level: Posting Freak
Joined: Fri. Feb 16, 2007
Posts: 1280 View posts
Location: Israel
Posted by slow_rider: Sun. Oct 22, 2017 - 06:47 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That's the option I can also implement, although it seems like something that will take some time to debug. Not sure if this is simpler than writing the external flash directly.

 

Top
clawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 90768 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Sun. Oct 22, 2017 - 12:51 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I've done the same with AT45. I simply implemented Xmodem/CRC so I could send the data from a "standard" terminal program and on through the AVR, into the DataFlash. If I had my time again I would have used Ymodem in fact (far easier to tell the length and where the data actually ends) 

Top
SolarFreak
SolarFreak's picture
Level: Rookie
Joined: Tue. Sep 27, 2011
Posts: 23 View posts
Posted by SolarFreak: Sun. Oct 22, 2017 - 02:00 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

slow_rider wrote:
It seems a little far fetched I could someone program everything using the JTAG debugger at a single go. Loading the code onto the external flash is possible as a separate step but if there is a better way I would like to hear about it.

 

For the purpose of my reply, I'm going to assume that "single go" means connecting the JTAG debugger / programmer and executing a programming routine on the target.  No need to connect a second interface, etc.

 

This might be overly convoluted, error prone, etc, but working with this requirement, here's how I would go about it:

 

Write a series of programs that contain code to transfer blocks of your 256K to the SPI flash, the remainder of the "program" would be the data to load.  Data will be split up over these programs, so you need 2-3 of them.

 

Put together a batch file / script to program your target that will load each program in turn, pause long enough for it to program the external flash, load the next, wait, etc.  Once this is complete, load the final program and complete.

 

Depending on your application, like if you're programming 1000's of devices, this approach might be worth pursuing.  If it's just a few, probably not.

Top
theusch
theusch's picture
Level: 10k+ Postman
Joined: Mon. Feb 19, 2001
Posts: 37838 View posts
Location: Wisconsin USA
Posted by theusch: Sun. Oct 22, 2017 - 02:16 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

slow_rider wrote:
Right now I am trying to do this using a micro SD card but even at this point this seems like an issue because the way the card is being read and the data stored in FAT file system.

OK, I'll bite -- probably billions of devices in the world use SD cards to store bulk data.  (photos, tunes, ...)

 

But in >>your<< particular unique application, it won't work.

 

Tough row to hoe, then.

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

Top
jgmdesign
jgmdesign's picture
Level: Moderator
Joined: Mon. Mar 19, 2007
Posts: 14192 View posts
Location: Long Island New York
Posted by jgmdesign: Sun. Oct 22, 2017 - 02:22 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

With the SD card you can load it externally with a $3.00 USB writer and Windows explorer.

Jim

If you want a career with a known path - become an undertaker. Dead people don't sue! - Kartman

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB user

Top
Brian Fairchild
Brian Fairchild's picture
Level: Raving Lunatic
Joined: Thu. Oct 18, 2001
Posts: 3123 View posts
Location: Eastern England.
Posted by Brian Fairchild: Sun. Oct 22, 2017 - 02:35 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

SolarFreak wrote:

Write a series of programs that contain code to transfer blocks of your 256K to the SPI flash, the remainder of the "program" would be the data to load.  Data will be split up over these programs, so you need 2-3 of them.

 

 

Hmmm, so a bootloader-sort-of-thing that when loaded moves the lower xxxx kbytes into the SPI flash and then writes a value to EEPROM so that the next 'bootloader' knows what it is doing.

"This forum helps those that help themselves."

"How have you proved that your chip is running at xxMHz?" - Me

"If you think you need floating point to solve the problem then you don't understand the problem. If you really do need floating point then you have a problem you do not understand." - Heater's ex-boss

Top
slow_rider
slow_rider's picture
Level: Posting Freak
Joined: Fri. Feb 16, 2007
Posts: 1280 View posts
Location: Israel
Posted by slow_rider: Sun. Oct 22, 2017 - 02:38 PM (Reply to #6)
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

theusch wrote:

OK, I'll bite -- probably billions of devices in the world use SD cards to store bulk data.  (photos, tunes, ...)

 

But in >>your<< particular unique application, it won't work.

 

Tough row to hoe, then.

 

When you put it that way it makes it sound like my question does not make any sense, and I don't think that fair.

I've stated that I'm currently going with an SD card just to make a quick prototype. The final application will have a PCB that will be coated and having any kind of detachable memory card on it would make coating more difficult, will further limit board design and case design. Having a connector on top for programming is easier. These are my requirements, and they are not as special as your comment might suggest. 

Top
theusch
theusch's picture
Level: 10k+ Postman
Joined: Mon. Feb 19, 2001
Posts: 37838 View posts
Location: Wisconsin USA
Posted by theusch: Sun. Oct 22, 2017 - 11:16 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

slow_rider wrote:
and they are not as special as your comment might suggest.

Apparently not.  Thus, you should have many proven examples to choose from.

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

Top
jgmdesign
jgmdesign's picture
Level: Moderator
Joined: Mon. Mar 19, 2007
Posts: 14192 View posts
Location: Long Island New York
Posted by jgmdesign: Sun. Oct 22, 2017 - 11:54 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

theusch wrote:

slow_rider wrote:
and they are not as special as your comment might suggest.

Apparently not.  Thus, you should have many proven examples to choose from.

 

THere was a thread on this recently where the OP was going to pot(encapsulate) the circuit board and did not want to use a SD card.  Cannot remember the thread title though.

 

JIm

If you want a career with a known path - become an undertaker. Dead people don't sue! - Kartman

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB user

Top
meslomp
meslomp's picture
Level: Raving Lunatic
Joined: Wed. May 2, 2007
Posts: 5220 View posts
Location: Nieuwegein, Netherlands
Posted by meslomp: Mon. Oct 23, 2017 - 07:09 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

jgmdesign wrote:

theusch wrote:

slow_rider wrote:
and they are not as special as your comment might suggest.

Apparently not.  Thus, you should have many proven examples to choose from.

 

THere was a thread on this recently where the OP was going to pot(encapsulate) the circuit board and did not want to use a SD card.  Cannot remember the thread title though.

 

JIm

But that was information that was not put in this question, and as such might have been for a different product, or there might have been a change in concept.

 

As the programming of the "external memory" is only once done.

why not make a special programmer for just the memory?

get data through USb/Uart to the programming board and through that program the onboard serial flash.

All you need to do is make sure that the Host processor does not do anything during that time.

So might be that first you set the board controller in "flash programing mode" then connect the programmer and program the onboard flash. and then set the board controller back to "normal mode"

you can then pot the board. or do it up front but leave the 6 needed programming lines available.

 

 

Top
clawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 90768 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Mon. Oct 23, 2017 - 07:27 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I assume most people here develop "factory test software" to check the integrity of the electronics during fabrication? Put the "flash loader" stuff into that so it runs once the board is proven. It will then replaced by the final application software before the device ships out.

Top
N.Winterbottom
N.Winterbottom's picture
Level: Resident
Joined: Tue. Oct 1, 2002
Posts: 595 View posts
Location: Redbourn, Herts, UK
Posted by N.Winterbottom: Mon. Oct 23, 2017 - 09:54 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
I assume most people here develop "factory test software"

Not unless I really have to, I don't. All my Test functions & Test / Configuration commands go into the customer firmware.

 

The most extreme example matching the OP problem that I can give is a Datalogger we designed with multi-lingual UI. The text for menus, UI & help was stored in external flash and uploaded via serial comms. This was beneficial in that whenever a new language became available all we had to do was upload a new languages file. In fact (given the up-loader program) the customer could have done this if necessary.

 

Top
mojo-chan
mojo-chan's picture
Level: Raving Lunatic
Joined: Thu. Jan 24, 2008
Posts: 2068 View posts
Posted by mojo-chan: Fri. Nov 3, 2017 - 12:53 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

I once had to do this and simply added test pads to the flash memory SPI bus. Then when the board was being tested on a bed of nails the test jig could simply write the data directly to the flash memory while holding the microcontroller in reset.

Top