SD Card Idea... Crazy or Sane?

Go To Last Post
15 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I am doing a project that could make use of an SD card to read images and sound files. These files are already in a format needed for my routines, and were created by custom software I wrote on the PC.

So I was reading over plenty of SD and FFS tutorials, and it seems that there are a lot of gotchas when using FFS and MMC. A lot of complaints about certain cards not working, and issues between FAT16 or FAT32, long or short file names, and then just the bulk of the code itself.

So I had an idea, which I will probably try no matter how crazy it sounds, but I thought I would get the opinions of fellow Freaks. My idea is this...

Since I am only wanting to read data, and I have total control over the data format (from my PC BMP/WAVE file converter), why use FAT at all? I could just put an ID string at the beginning of my file like "ATOMICZ" and then write a DWORD right after containing the length of this file in bytes and then then name as a fixed length string.

ATOMICZ,000013028039,IntoScreen.....(data)

I could drop those files on the SD card through windoze and then my AVR could start at sector 0, looking for the ID of the first file. Once it finds the first file (whatever it may be), it can then return the filename. If it's not the file I want, then the length is read and then that many bytes are skipped. Then the next file is found and checked until I have the name of the file I want.

This may seem cumbersome, but in reality, once a "wrong" file is found, the AVR can skip to the next one right away based on the known size of that file. Even with 1000 files, it wouldn't be all that bad.

The benefits to this (as I see them)...

- No bulky and complex FFS library needed
- Use FAT12,16,32 or whatever from your PC
- Use any size SD card from any manufacturer
- Zero pain - always works no matter what

So my plan is basically to "embed" the file structure in each file and then sequentially read / skip until I have the file I want. On the PC side, everything seems normal to Mister Gates and his file system.

Besides the SD init code, there would not be much else to the routines. This method should be super lightweight and ultra fast. Assuming FLCK/2 (say 10MHZ) and a serial stream of 8 bits, this method should be able to reach 1MB/Sec?

Ok, now chime in and let me know why this won't work before I begin a late night of assembly coding fun!

Cheers,
Brad

I Like to Build Stuff : http://www.AtomicZombie.com

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

This assumes that Windows (or other OS) will write files to the SD card contiguously (all together with no gaps)!
This is not allways the case (hence all that busines with the FAT to jump from cluster to cluster in the right order) and the directory stuff to find where in the FAT the first cluster is located.

For your scheme to work, your Windows app must have sufficient control as to write your images and audio files to specific blocks on the SD card and a way to find the free space. You may also have to handle deletion and update (to a bigger size!).

There are scale issues; chasing down 1 or 2 MegaBytes is reasonable; but some cards are 4 or 8 GIGABYTES or more. You might take 10-15 MINUTES to find your file!

Still, thats is doable on Windows (lookup opening a drive in the MSDN documention).

Mike.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

dd (unix) is enough for the task.
Prepare a disk image on the host computer and then just block write it to the device.
Similar utilities exist for windows

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

For those old enough to remember, the Flex operating system treated the disk as a linked list. You could use a similar scheme with a forward link in the header of each file. This would allow you to traverse your files quickly in one direction. I gather you basic use case is write once, read many.

Note that this does not get away from the issue of different cards requiring different initialisation.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Not a bad strategy, really. To skip to the next file, you don't have to read all the bytes, just add the length of the file to your pointer, divide by the block size to find which block contains the next file name.

Perhaps easier would be to start with a list of the files, each with a starting position and size. Even more, if you sorted this list by name, you could find a specific file by binary search rather than searching through all of them.

The largest known prime number: 282589933-1

It's easy to stop breaking the 10th commandment! Break the 8th instead. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'd just use FAT. Stick to 8.3 filenames and switch off LFN support. The issues with 1/2/4GB cards and whether they will initialise or not is going to happen long before you start doing FAT so you'll see those issues anyway whether FAT is used or not. Once the card is reading/writing sectors then FAT is kind of bound to work.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Interesting, never considered the fragmentation issue. I guess the easy solution would be to format the SD and then dump the files on from the PC right after. I thought about doing some RAW disk writing, but didn't find much info on it for .NET examples.

I would like to find a way to use the PC normally (drag and drop), but then have the AVR get access without needing FFS.

I could always use a "container", which the PC appends all my files together in a huge file and then writes it to the SD. That should make all files contiguous and would be an easy PC app to code. It would just read every file in a directory, and then spew out one giant binary to be copied to the SD.

Still thinking about trying this.

Brad

I Like to Build Stuff : http://www.AtomicZombie.com

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

And you write the whole works to an img file. This represents the exact physical arrangement of data on the card. Use a program to write this img file to the card and read it with whatever code you like in your avr.

The largest known prime number: 282589933-1

It's easy to stop breaking the 10th commandment! Break the 8th instead. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

In Linux it's real easy to do this kind of thing as you just open the whole drive (/dev/sdX) or the whole partition (/dev/sdX1) as if it were a file then write to it. In fact it might be worth installing VirtualBox and a copy of Ubuntu just so you could do this (even easier if you can simply use the dd command from a shell).

DOS can do it too but there's some tortuous syntax involving /.//PhysicalDriveN or similar which is not as easy as Linux device naming.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If I ever find the time, I will try a few things out and post my results.

Brad

I Like to Build Stuff : http://www.AtomicZombie.com

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I love your http://www.LucidScience.com site.

The largest known prime number: 282589933-1

It's easy to stop breaking the 10th commandment! Break the 8th instead. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

you could also use an avr to write to the sd card. somake an USB link to the PC let the PC spit its data to the AVR and let the AVR write the data to the SD card. then you do not need fat file system.
from what I recall (but this is long time ago) I have read that you should be able to access the sd card just like you do an onboard dataflash. Problem is that you then never can use it at a PC without low level formatting

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Just use FAT. and short file names.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
So I was reading over plenty of SD and FFS tutorials, and it seems that there are a lot of gotchas when using FFS and MMC. A lot of complaints about certain cards not working, and issues between FAT16 or FAT32, long or short file names, and then just the bulk of the code itself.

If you want to support any sd-card from any size this could be important. But on the other hand, in many applications you have the control in your hands. By using always the same type of SD-card a lot of those problems can be rejected. And mind that a lot of testing was done on the FATFS libraries. Making your own system, ok but you also need to test it. Code size can be reduced by using read-only with (Tiny-)FATFS. So if you like to re-invent wheels and have the budget (time) and energy to make the routines youself, do it. If you want a stable system which is used -and tested- by many people, use FATFS. You tell you allready have software written to produce your files on the PC, why not extend this PC software so it writes the files to the SD card ?

Patrick

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

This is one of the methods to make things easier for the MCU.
However if you remove the FAT filesystem layer and write your own filesystem layer, you are still left with card communication layer, so you must still be able to initialize card properly to be able to read anything. If initialization succeeds, then most likely there will be little problems with different FAT filesystems, of course the overhead on a 8-bit micro.

In general, there are at least these methods:
1) Just use FAT filesystem, easy and fast for everyone except poor 8-bit micro.

2) Use FAT filesystem, but in a custom way, like, write one big file to card and make sure it is continuous, and let the MCU to locate that file in some way and use the space it allocates as MCU wants (your method is fine if you know any previous files that are now deleted but not overwritten yet did not have that same header). Another method is that the file must reside on root dir and have certain name to locate its starting sector, then you also know its size, and can even verify from the FAT table if it is contiguous. Just a note that don't make the requirement that it is the first entry in the directory table, as that is usually used for disk label, and Mac OS can create some special dir/file starting with a dot for resource forks, and maybe there might even be a recycle.bin directory from some OS.

3) Use SD card as raw blocks of data. A bit harder to write disk images to card, maybe, but there are programs that do this (dd on unix, windows has imaging utilities too). Easiest for 8-bit micro, hardest for you and user.