AVR Freaks

Yes, it'll do that. This is "bad block remapping". Say you write 512 bytes to sector 123 but the card either already knows that 123 is "damaged" or that, while doing a test read of the written data it finds it was not stored correctly it will add 123 to the remap table and take the next sector from the spare "remap pool". In future when there are read/writes to 123 it may well be using sector 54273551 in the remap pool but you aren't aware of this. Of course things really hot up when 54273551 in the remap pool then shows a write failure itself! It then has to remap a remap....

 

Even magnetic HDDs from Seagate and Western Digital have been doing thus same remapping thing for decades.

Aye, there's the rub! (actually I think they use ECC and redundancy) The sectors in an SD/MMC are just bytes in a NAND array so all the usual rules about updating NAND apply. You can write several times to the same set of bytes as long as all the bit transitions are in the 1->0 direction but as soon as you need a 0->1 transition you have to read out the page contents, update the new date, erase the page, then write the updated stuff back.

 

Your scenario was a file that ended early the first time. So maybe only the first 217 bytes of the 512 were written. The nature of erased flash means the remaining 295 bytes will be 0xFF. If you now reopen the file and add "hello world" from byte 217 onwards then it will just make the 1->0 transitions necessary to make the next 11 bytes of 0xFF turn into "hello world". In that sense it is writing into the SAME sector so I would call this "overwrites" though perhaps that word was badly chosen and I really should have used "appends". Of course what you may actually do is open the file, seek to the end, then seek back 11 bytes and THEN write "hello world". So now you are changing data - truly over-writing. If you did this either all the bit transitions are 1->0 and it just does a plain write, or somewhere in there a bit in the existing data has to go 0->1 to turn it into "hello world" so now the page read, update, erase, write thing would have to come into play.

 

BTW I am 99% certain the page size is NOT just 512 bytes - so that complicates things a bit further.

 

FAT all works by multiples - Say you write 1,500 bytes to a file. It writes 512 to the 1st allocated sector, 512 to the 2nd allocated sector and 476 bytes into the 3rd sector. It then writes 1500 into the size field in the directory entry. If you now reopen that file later and seek to the end and write 300 bytes then it knows that it has to start writing at the 477th byte in the 3rd sector (adding to what is already there) then that spills over into the 4th sector where 164 bytes are read. Of course, at some stage one of these writes that spans the 512 byte sector size boundaries is also going to hit the allocation unit boundary too. Say the allocation unit (cluster) size is 16KB so there are 32 sectors per AU then as it writes over the boundary of the 32nd sector it cannot just continue on to write into the adjacent, 33rd sector, for all it knows that might already be part of an existing, written file, so this is where "FAT" actually comes into play. It now has to go back to the FAT tables, find the next unallocated AU entry, write the number of that into the AU entry for the previous link in the FAT chain then multiply that number by the AU size (32 sectors), offset from the base of the data area and continue writing the next 32 sectors here. (as I say it's all about "multiples", multiples of the sector size when writing bytes and multiples of the AU size as it hits the cluster boundaries)

 

What it does not do in the 512+512+476 case is simply "leave behind" the 476 bytes and some how copy them to the new sector where the next 36 of the 300 being added are appended. It really does go back and update (better word than "overwrite") the sector where the existing 476 are held.

 

Like I say if you reopen the file and don't seek to the end but seek to somewhere less than the previous end and then write that truly is "overwrite"ing.

 

BTW if it had been that things like the 476th get "orphaned" then the FAT system would have to keep a map of each and every sector to say whether it were useable or not". There just isn't room for that. The whole reason you have AU size granularity (and hence often "wasted space" when writing short files) is that the FAT tables couldn't be any larger to support a finer granularity. In grown up filing systems like NFS and EXT3/EXT4 the block size (effectively AU/cluster size) is fixed at 4KB so you never waste more than 4095 bytes and on average it will be 2048 bytes per file. But in FAT, where AU size is typically 16KB or 32KB you can waste up to 32767 bytes and on average 16384 bytes per file when the AU size is 16KB. On the other hand FAT is "dead simple" compared to NTFS/EXTn !

 

BTW because of the Microsoft patents on FAT (well LFN) at one stage I looked at using an alternate format on SD/MMC. I got most of the "read" stuff working to implement UDF but I stopped short of completing the "write" functions which (like FAT) are quite a lot more complex than reading. "Universal Disk Format" is nice because it is "open". Sadly it never seem to have got much beyond usage in optical media such as CDs/DVDs

 

 

Ah but the card is probably the major power consumer in the system. So it may continue to process when "writes" have finished.

WOW Cliff, you must have ditched the tablet and use a real keyboard or maybe added a real keyboard to the tablet? Lots of long posts...

True enough - these days I do a lot of Freaks reading on my phone when out walking my cats (once they've heard a mouse they'll sit and stare at a patch of undergrowth for 20..30 minutes!) but sometimes I use my PC (often when engaged in boring conference calls!). It's much easier doing stuff on a PC!!

If you add a printf() to the disk_read() / disk_write() functions how many of each are done at each step I wonder?

No, what I was asking is for an open(), seek(), write(), close() how many sector reads and how many sector writes are involved in each operation.

 

I'll assume that during the mount() the FS already read and cached the BPB so it should be able to calculate where the FAT and root directory are located but then to open() (assuming CD = \) then it may need to read N sectors sequentially until it finds the one that has the 32 bit directory entry with the name that matches (far more reading if LFN enabled!). From this it can get the file size and also the starting AU. So it may need nothing more than N reads until it find the directory entry. That gets it as far as knowing where the initial AU is which is all it needs at this stage. The seek() could be very quick (it was in your case) or it could be quite long. If the offset is within the AU it does not need to do any sector reading, it just updates the file pointer. But if it's to a position beyond the first AU it may need to start walking the FAt chain at this stage - or it might defer this until the actual read/write operation. The write() *may* be as simple as one sector write, or it could involve a whole host of activity if it tips over an AU boundary.

 

As I say, I'm just interested to know how many read/write each high level file operation involves. Do not assume that a write() of 40 bytes (say) is as simple as one 512 byte sector write. In fact until flush (sync) it could be that the data write is simply cached in the buffer within the FIL structure.

What I do during dev, especially when "pin challenged" or no convenient interface, is to log a set of interesting items to internal EEPROM and then read back with ISP and interpret.