NGW100 SD card write speed

Go To Last Post
16 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

hi everybody again.

I think I saw a post about this already, but I couldn't find it with google on avrfreaks.net :(

However...here is the problem:
I'm trying to record 96khz stereo audio (16 or 24 bits) to a SD card. Furthermore, the idea is to record 2 stereo channels (4 mono streams).

I once did a test with the SD card and I could get about 1.1MiB/seg write speed. I don't remember if i used fat or plain 'dd'.
Now i'm using vfat and I cant get more than 0.35MiB/sec which is too slow for my app. This happens on two SDs, one Kingston and the other one SanDisk. I couldn't try with other ones.

However, read speed is better than 2.5 MiB/sec.
I checked the MCI clock and it seems alright (I get 15MHz).

Maybe the vfat layer is slowering things down, but I think there must be something to do to get higher write speeds since windows can do 2.74MiB/sec when writing on the same SD. Of course I'm not expecting to compare a PC with AVR32 embedded linux, but what I mean is that it should be something doable.

any hints?

btw, i'm using kernel 2.6.23

Thanks

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hmm, I did a bunch of tests with the bonnie++ benchmark a while back. That was an ext2 fs and an older kernel but speeds were about what you were seeing anyway.

https://www.avrfreaks.net/wiki/in...

Donno what we'd be looking at to improve this kind of performace; being an SD card there aren't any seek patters to optimize or anything!

You might try recompiling your kernel using the NOP block elevator rather than CFQ (which should be the default IIRC). It probably won't make much difference but it's all I got ;-)

-S.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

cbecker wrote:

I'm trying to record 96khz stereo audio (16 or 24 bits) to a SD card.

I'm working on the same type of project (24/96 stereo PCM to WAV). :)

I've come to the conclusion that you need to first read in the whole SD card FAT (or enough for your recording), pre-allocate the sectors you will write to, save that info into RAM (or in my case a separate microSD card), then use that info during the recording to know what sectors to write to. The write has to be done directly to the SD card, and then when finished, update the FAT. I don't have a clue how to do this in Linux, and as such I'm trying to do it without an OS. Anyone know how to mod the Linux file system to support such a writing scheme? I'd like to also be able to playback MP3 and FLAC, and that would be a lot easier if I could use Linux.

I've also learned that you need to implement a circular FIFO buffer large enough to handle periodic stalls in writing to flash. In my previous attempts with other micros, I measured up to 75ms with Compact Flash, but haven't tested SD yet.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Here's an interesting thread, 1.1 MBytes/sec write:
https://www.avrfreaks.net/index.php?name=PNphpBB2&file=viewtopic&t=51489

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Indeed FAT isn't a fast filesystem but it is simple enough that if you can hack on the fs code, you can tune it for your performance. I very much doubt you can do FAT preallocation in any sane way under Linux without actually attacking the FAT fs code in the kernel.

Using async mount modes is a good speed hint but I don't know how relevant it is for FAT. Worth a shot though.

-S.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

well first thanks to all of you for the quick reply!
avrfreaks always surprise me :)

If i am right you can read the SD as a block device by using the /dev/mmcblkxxxx devices. Interpreting fat or fat32 is something to be done, but there are many libraries out there, open source too.

Brandon, are you using an AP7000 for your project?
I'm using the NGW100 with AD1871 ADCs (two of them, cascaded).
The board does fine recording stereo through network or to ram (/dev/shm).

If I can manage to get a solution I will post it.

thanks everybody!

Carlos

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

cbecker wrote:
If i am right you can read the SD as a block device by using the /dev/mmcblkxxxx devices. Interpreting fat or fat32 is something to be done, but there are many libraries out there, open source too.
True actually, I guess there's no reason you need to mount the SD card formally. Or better yet, mount it through the FUSE (filesystem in user space) libfat. The userspace library would be much better to hack on than the in-kernel one but you remain with a simple read/write interface once your mods are complete.

I know performance is an issue and fuse is slower than in-kernel but only slightly so - any FAT preallocation mods should allow you to get the speed you require anyway.

-S.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

thank squidgit.
another way is to use my own format to record on the SD, but it would be a waste since the avr32 can surely handle fat the right way if it is done right.

thanks.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Right, you can do that too. You know, I think I'm getting tired, I could have sworn you wanted to access the SD card as a mass storage device over USB. Since re-reading your first post you don't have that restriction then yeah go forth and use your own format.

You know, I have no idea how to access block devices through their device files; I've only ever done it inside the kernel. In there it's moderately complex and I know FUSE needs some kind of sysfs connection. Might be easier just to use a FUSE and be done with it!

-S.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I did some tests yesterday and I found interesting results.
I tried writing blocks by using:

dd bs=1024 count=8192 if=/dev/zero of=/dev/mmcblk0

I got less than 1MiB/sec... quite bad.

However, later, I formatted the card with windows with FAT filesystem.. mounted with either -o async or -o sync and I got more than 1.4MiB/sec.. quite strange!
Looks as if even when writing blocks Linux looks for some information regarding partitions, I don't know.
Before re-formatting I got a mmcblk0 and a mmcblk0p1 device. After reformatting there is only mmcblk0 and that's the one I use with mount. Any ideas here?
I could explain the slow write speeds on the first posts since there was data on the FAT so then finding appropiate empty blocks could be time consuming, but this, I don't know.

In the end, now it's working with FAT and I was able to save 96kHz 24bits sterep (actually 32 bits used when saving) in realtime. That's about 768kbytes/sec.

I still have to try saving a long stream, like minutes or so, but it seems to be working. I think the FAT will have to be 'clean' or something similar before being able to do this kind of sound recording.

thanks everybody.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

cbecker wrote:
Brandon, are you using an AP7000 for your project?

The first version of my project is a minimal parts count version with 16/48 audio that only records to wav file, so no. I have a tiny 64 Mbit 8 pin serial flash chip that I'm going to try the pre-allocating thing with. If my calculations are right, I only need to read about 190 bytes a second of cluster data while writing the audio.

One thing about write speed I realized, a Linux version has sufficient RAM to store the whole FAT in RAM, thus eliminating the need to pre-allocate. The RAM starved FAT versions seem to work by reading the FAT continuously as it's writing data. Which greatly slows things down.

The second version of my project will have a AP7000, Linux, a display, FLAC, APE, OGG, MP3 decoding support, and THAT analog compressor/limiters for the recording frontend. Larger and a lot more parts and bigger battery, but also a lot more useful.

My main concern with Linux is, how do I get it to talk to the codec? Probably I have to figure out how to write a Linux audio driver right? After I learn how Linux works. And learn C... hehe.. I'm quite a ways from getting anything done. I did have success today getting SD working on the UC3A though (see my post in the Academy forum).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Brandon604 wrote:
My main concern with Linux is, how do I get it to talk to the codec? Probably I have to figure out how to write a Linux audio driver right?
If the driver isn't already written then yep :-). Which analog codec are you going to be using? Just the abdac inside the AP7k? 'coz that driver's already written.

-S.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think brandon talks about a I2S (SSC) driver for the external ADC.
I had to do it myself since there wasn't any example on the current linux kernel. It wasn't difficult but there are many things to consider.
DMA is something you can't avoid, specially at 96kHz.

Brandon, if you need any help with that just ask.

well I will let you know if I get some more details or new results on the SD filesystem stuff.

thanks!

Carlos

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

squidgit- I'm going to try it with one of the Wolfson codecs- I have a WM8776 and if I can get my toaster reflow oven project done, may try the tiny WM8750.

cbecker- thanks and will do! :)

I just posted a thread about possibly getting stereo A/D with one SSC:
https://www.avrfreaks.net/index.php?name=PNphpBB2&file=viewtopic&t=63326

And I got 340 KBytes/sec in my latest SD write speed tests on the UC3 (at 12 MHz SPI too), so I'm not going to bother with the pre-allocating idea.

cbecker- your speed tests are great news for when I roll this with the AP7000 and 24/96. :)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Re write speeds to SD: isn't there a way to pre-erase blocks on SD and/or MMC cards, so that writes can go faster? I don't think that any Linux filesystems (other than MTD based ones like JFFS2) understand that notion. But it might be a good way to ensure better write throughput.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

mojojojo
You are indeed correct.
It is very important to be carefull of the pre-erase parameter/command though as my experiments show it to be persistant. That is, if you use it, use it always or you may erase more than you wanted.

It is to be sent before a multi-block write as a cmd55/cmd23 pair. Doing so changes the cards aggressiveness on erasing blocks that might be overwritten. The problem is, is you are doing a 128kbyte write and inform the card of the coming 256 block write with a 256 block set_pre_erase command it appears you are in fact instructing the card that any successive multi-block writes are also planned to be 256 blocks. This might prove fatal when doing a 2-block multi-write to update some part of the FAT right after writing the file or files.

On the posetive side I can inform that I did indeed see almost a doubling in write performance vs having no "clean" blocks before write.

Might be worth mentioning I am doing 600kbyte/s at the moment using an ATmega64 @16MHz and regular spi @8MHz.
1.1 Mbyte/s is not very impressive for something capable of full bus speed in SDIO mode (4bit).
My card is a 4 GB Sandisk SDHC card and it benchmarks to 11 Mbyte/s raw performance using a SD card reader.