microSD cards and Timing

Go To Last Post
15 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

Greetings -

 

Just made a discovery, somewhat by accident, and thought that I'd pass along what I found, in case someone else gets bitten by this.

 

I use an Kingston 8GB SpeedGrade 4 uSDHC card. For close to the last year, I have been pounding away at that card. Have written literally thousands of files, varying from a few bytes to 25Meg. A couple of times, my software "ran away" and created 9999 named zero-length files. The library I use is FatFs.

 

For the last 6 months or so, I have been having real problems with file writes that run far too long (at one point, they occasionally took over 300ms for a 256 byte block). This has caused no end of grief. This was in a device that sleeps, then is awakened every 250ms to do its "thing". 300ms was a total killer. I was sure it had something to do with the software because my previous product that uses the same spec uSD card had no such problems.

 

Just a few days ago, I was chatting with an EE friend about a variety of problems we have shared. I happened to mention this particular problem. He asked whether or not I had tried a different card?

 

No......

 

So, just as a trial, when I got home, I tried a reformat. I had reformatted the card several times, but it was just a generic reformat. I decided to use the software tool provided by the SD card association. It offers the generic format and an option that "clears" every byte on the card. The latter took around 15 minutes and that is what I used. 

 

When I put the card in the instrument, the "normal" write time for a 256 byte block dropped from around 26ms to around 3ms. And, at least during the test time with the logic analyzer, there were NO long writes!

 

Wahoo!

 

Jim

Jim Wagner Oregon Research Electronics, Consulting Div. Tangent, OR, USA http://www.orelectronics.net

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Since sdcards are nand flash based, the sectors begin to die after around 1000 writes. The sd card controller has to detect this a relocate the dud sectors. This takes some time. Reformatting basically forces the hand of the controller and the dud sectors get retired. Rinse and repeat until all sectors are dead.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That is what I suspected. Helps to clarify what is going on.

 

Interestingly, the DiskUtility formatter on my Mac did what appears to be a standard "format" and I have used it several times recently, without the dramatic change. It was only after using the SD Association utility AND its write over all of the bytes of the card that the change occurred. I can imagine that the write over the entire card forces it to evaluate every byte of every sector and retire those that no longer function acceptably. It is very easy to imagine such an improvement when every sector it tries to use is quickly judged as OK during run-time.

 

Jim

 

Jim Wagner Oregon Research Electronics, Consulting Div. Tangent, OR, USA http://www.orelectronics.net

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Mr Karman's answer is only partially correct. Because after significant (but not excessive) use, the Flash controller is performing remapping operations anyway as part of wear leveling.

 

So! The effect that Jim has experienced is called Write Amplification. A grossly simple reason it comes about is when there are no free pre-erased blocks in which to write a sector and data must be moved around first to create space. The card does not know which sectors contain data that is important or whether those sectors comprise part of an deleted file, Therefore all sectors that have previously been written to are treated as containing important data which must be preserved during the remapping operation.

 

The Wikipedia article https://en.wikipedia.org/wiki/Write_amplification is long and not an easy read but the contributing factors are explained there. There is mention of the TRIM command to regain unused space at flash controller level but I doubt FatFs supports that.

 

The data recovery folks, Ontrack have a blog here: https://www.ontrack.com/blog/2019/01/31/what-is-write-amplification-wa-and-how-does-it-effect-ssds/ but opposite to Wikipedia it is a bit too simplistic.

 

In short then; when you did the long format using the SD Card Association tool you effectively pre-erased the entire card, reducing write amplification to zero.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ka7ehk wrote:
I decided to use the software tool provided by the SD card association.
https://www.avrfreaks.net/forum/fatfs-and-disk-removal#comment-2645466

ka7ehk wrote:
Wahoo!
smiley

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

N.Winterbottom wrote:
There is mention of the TRIM command to regain unused space at flash controller level but I doubt FatFs supports that.
FatFs - Configuration Options - FF_USE_TRIM

Is FatFs's trim same as SD's trim?

 

"Dare to be naïve." - Buckminster Fuller

Last Edited: Mon. Mar 11, 2019 - 12:02 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

gchapman wrote:
Is FatFs's trim same as SD's trim?
Good question. If you follow into the source to see what the TRIM thing does you see it ultimately doing things like:

#if FF_USE_TRIM
			rt[0] = clst2sect(fs, scl);					/* Start of data area freed */
			rt[1] = clst2sect(fs, ecl) + fs->csize - 1;	/* End of data area freed */
			disk_ioctl(fs->pdrv, CTRL_TRIM, rt);		/* Inform device the data in the block is no longer needed */
#endif

So this in turn seems to be dependent on an IOCTL that does" CTRL_TRIM". Looking at the most recent issue of ffsample then in the AVR directory the mmc_avr_spi.c file has this support for that IOCTL:

	case CTRL_TRIM:		/* Erase a block of sectors (used when _USE_TRIM in ffconf.h is 1) */
		if (!(CardType & CT_SDC)) break;				/* Check if the card is SDC */
		if (mmc_disk_ioctl(MMC_GET_CSD, csd)) break;	/* Get CSD */
		if (!(csd[0] >> 6) && !(csd[10] & 0x40)) break;	/* Check if sector erase can be applied to the card */
		dp = buff; st = dp[0]; ed = dp[1];				/* Load sector block */
		if (!(CardType & CT_BLOCK)) {
			st *= 512; ed *= 512;
		}
		if (send_cmd(CMD32, st) == 0 && send_cmd(CMD33, ed) == 0 && send_cmd(CMD38, 0) == 0 && wait_ready(30000)) {	/* Erase sector block */
			res = RES_OK;	/* FatFs does not check result of this command */
		}
		break;

So that seems to be doing a block of sectors erase.

 

Google says that while not all cards support block erase for those that do it is:

 

CMD32 - set start

CMD33 - set end

CMD38 - erase selected block of sectors

 

EDIT: It appears that FatFs uses this as a "fast delete" if available

Last Edited: Mon. Mar 11, 2019 - 12:49 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I couldn't see an application level function which is why I didn't think it was supported but digging around at the low level ioctl layer http://elm-chan.org/fsw/ff/doc/dioctl.html I found this:

 

Standard ioctl command used by FatFs

 

Command Description
CTRL_TRIM Informs the device the data on the block of sectors is no longer needed and it can be erased. The sector block is specified by a DWORD array {<start sector>, <end sector>} pointed by buff. This is an identical command to Trim of ATA device. Nothing to do for this command if this funcion is not supported or not a flash memory device. FatFs does not check the result code and the file function is not affected even if the sector block was not erased well. This command is called on remove a cluster chain and in the f_mkfs function. Required at FF_USE_TRIM == 1.

 

Although for it to be of any use, it seems one must also implement:

 

Command Description
MMC_GET_CSD Read CSD register into a 16-byte buffer pointed by buff. (MMC/SDC specific command)

 

I'm quite impressed actually.

 

Last Edited: Mon. Mar 11, 2019 - 01:38 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That starts to make sense now.

 

Thanks for the input, everyone.

 

Jim

Jim Wagner Oregon Research Electronics, Consulting Div. Tangent, OR, USA http://www.orelectronics.net

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
Since sdcards are nand flash based, the sectors begin to die after around 1000 writes.

Small correction.

There are different manufacturers of uSD cards and different levels of quality.

I do not have much experience with uSD cards, but just enough to never buy the cheapsest uSD cards I can find on ali / ebay ever again.

 

I'm quite dissapointed in Swissbit.

I was looking for a number on write endurance, but all I could find in their datasheets was marketing bullshit on how great and wonderfull their cards are, spiced up with some easy numbers such as 20000 insertions.

 

Doing magic with a USD 7 Logic Analyser: https://www.avrfreaks.net/comment/2421756#comment-2421756

Bunch of old projects with AVR's: http://www.hoevendesign.com

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Paulvdh wrote:
I was looking for a number on write endurance, but all I could find in their datasheets was ...
The datasheets are behind an access wall.

MicroSD Memory Cards - Swissbit (mid-page, Downloads)

 

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You can have what is considered poor endurance in a first quality nand device - the technology used determines this. Like most things, you get what you pay for.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

On the whole I would only ever buy Sandisk. Given that SD are so cheap the odd extra $ is pretty insignificant to get some guarantee of quality. 

 

I've not had much problems with SD but I have had USB memory sticks from "cheap" sources that have had some catastrophic failure that meant their capacity could not be recovered.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:

On the whole I would only ever buy Sandisk. Given that SD are so cheap the odd extra $ is pretty insignificant to get some guarantee of quality. 

 

I've not had much problems with SD but I have had USB memory sticks from "cheap" sources that have had some catastrophic failure that meant their capacity could not be recovered.

 

I bought a pair of PNY USB memories. One of them has to be formatted every time I try to use it. The other just always works. Strange. I wonder if these are counterfeit PNY parts. They came from Wal*Mart where I got my counterfeit GERBER pocket knife.

The largest known prime number: 282589933-1

It's easy to stop breaking the 10th commandment! Break the 8th instead.