Architectural/design question: fast data logger

Log in or register to post comments
Go To Last Post
21 posts / 0 new
Author
Message
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
#1
Posted by steverino: Fri. Aug 20, 2010 - 02:35 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi,

I find myself back into low level embedded work after a hiatus of a few years. I've been asked to improve an existing data gathering system. I could use some suggestions from the experts here.

The system has two xmega32's: the first gathers and processes sensor data, and passes it along to the second. The second shoves the data into an SD card via Chan's FatFs. The data path between the two xmegas is uart.

I've been asked to up the data acquisition rate to 320Kbaud.

Interprocessor com: I'm stuck with the uart as the interprocessor communication channel. Is it realistic to run uarts at 460 or 920 kbaud and expect the < 1% error rate that atmel suggests is possible?

SD card: based upon the current system running at 16MHz, my gut tells me a data rate of 320Kbaud will overwhelm the FatFs and the SD card.

SD card alternatives: SPI sdram? Something usable as one large data spacewith no allocation overhead seems the way to go. Ideas other than sdram? Non vol is not a requirement.

Thanks,
Steve

Tags:

AVR Microcontrollers, megaAVR and tinyAVR, ATmega32
Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Fri. Aug 20, 2010 - 02:43 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Sorry...capacity...
4+ hours of data --> 4.6Gbits of storage.

Top
Gold Markclawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 101194 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Fri. Aug 20, 2010 - 02:57 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Are the two AVRs run from the same clock or crystals? If so then you can use 1M or 2M baud rate if you like with no worries.

As for SD write speeds. Using the 4 wire SD connections they can do 1M..2M/s but using MMC one wire (as almost all AVR designs do) the badnwidth is going to be much lower. I'd imagine it's the SD write speed that will actually be your bandwidth limiter, not the AVR<->AVR link. As a test I guess you could just open a file, and keep fseek'ing and rewriting the same test data to get an idea of the bandwidth that can be achieved.

Cliff

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Fri. Aug 20, 2010 - 03:08 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Each xmega is running off its internal clock.

I will try your file write suggestion with various buffer sizes.

Thanks.

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Fri. Aug 20, 2010 - 03:12 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
you could just open a file, and keep fseek'ing and rewriting the same test data to get an idea of the bandwidth that can be achieved.

Cliff


My current SD access mode is to open a file and leave it open. I haven't been seeking at all. Should I be?

Top
Gold Markclawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 101194 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Fri. Aug 20, 2010 - 04:32 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You don't have to seek but if you don't then you are going to create a very big file - maybe that doesn't matter?

As for the clocks, I guess with the guaranteed 1% accuracy of the Xmega maybe it'll be OK but for high speed UART I think I'd be using a more accurate clock source.

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Fri. Aug 20, 2010 - 04:56 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
You don't have to seek but if you don't then you are going to create a very big file - maybe that doesn't matter?

Yes, that's not a problem.

clawson wrote:
As for the clocks, I guess with the guaranteed 1% accuracy of the Xmega maybe it'll be OK but for high speed UART I think I'd be using a more accurate clock source.

I've been promoting a hardware spin to do just that. (Thinking ahead to a likely suggestion from someone, a switch from uart to spi is not possible, even with the hardware rev.)

Thanks!

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Fri. Aug 20, 2010 - 05:18 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
You don't have to seek but if you don't then you are going to create a very big file - maybe that doesn't matter?

I'm curious. Why is that? At first I wondered about each small write using a larger block of SD space. Not sure how a seek would change things, though.

Top
Gold Markclawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 101194 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Fri. Aug 20, 2010 - 05:40 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I simply mean normal file access. If in POSIX land I do:

FILE * fout;

int main(void) {
  int i;
  char * ptr;

  fout = fopen("file.dat", "w");
  for (i=0 ; i<10 ; i++) {
    ptr = (char *)rand();
    fwrite(ptr, 1024, 1, fout);
  }
  fclose(fout);
}

I end up with a file holding 10K of random data. While if I do:

FILE * fout;

int main(void) {
  int i;
  char * ptr;

  fout = fopen("file.dat", "w");
  for (i=0 ; i<10 ; i++) {
    ptr = (char *)rand();
    fwrite(ptr, 1024, 1, fout);
    fseek(fout, 0, SEEK_SET);
  }
  fclose(fout);
}

I still end up writing 10K of data (and taking almost exactly the same length of time) but I end up with a 1K file.

I was just concerned you may have file size problems if you went with method A but I guess it takes a while to write 4..8GB from an AVR!

But forget the fseek() thing.

Cliff

Top
ala42
ala42's picture
Level: New Member
Joined: Sun. Jun 6, 2010
Posts: 5 View posts
Posted by ala42: Fri. Aug 20, 2010 - 11:27 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
I simply mean normal file access. If in POSIX land I do:

FILE * fout;

int main(void) {
  int i;
  char * ptr;

  fout = fopen("file.dat", "w");
  for (i=0 ; i<10 ; i++) {
    ptr = (char *)rand();
    fwrite(ptr, 1024, 1, fout);
  }
  fclose(fout);
}

I end up with a file holding 10K of random data.


You end up with 10 1k blocks read from a random start address. On a real OS you get an access violation.

I have read somewhere that SD cards delay writing from time to time, so it will be useful to measure the time that writing each block takes, so you know the maximum delay time. Just measuring an average data rate is not sufficient, as you will run out of buffers if the maximum delay is too high.
You could also directly check that you get the sustained datarate you need.
You have to benchmark and completly fill each card you want to use as the write speed is not constant on all parts of the card.

MediaCodeSpeedEdit

Top
Bronze Markbobgardner
bobgardner's picture
Level: 10k+ Postman
Joined: Wed. Sep 4, 2002
Posts: 26609 View posts
Location: Orlando Florida
Posted by bobgardner: Sat. Aug 21, 2010 - 01:28 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

320Kbps isnt a standard rate... you could sample at that rate, then send it over at max baud divisor... 1 mbps. The 320Kbps is about 40K Bps... 80 sector writes a sec, 12 ms per sector. Those things write about 5ms a sector, right?

Imagecraft compiler user

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Sat. Aug 21, 2010 - 05:27 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The 320K bps I quoted is based on the data requirements: 20 bytes of worth of sensor data sampled at a rate of 2K Hz.

FatFs blocks interrupts regularly during writes. On the SD logging xmega I do run the uarts on interrupts, but since FatFs is blocking I see the need to avoid both tasks happening simultaneously. I've been sneaking small SD writes inbetween uart interrupts. This is sort of okay for my current 100 Hz rate, but I don't like the scheme and it sure won't scale up to the target 2K Hz sampling.

I feel like I need a fairly fundamental change.

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Sat. Aug 21, 2010 - 05:31 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

bobgardner wrote:
...send it over at max baud divisor... 1 mbps.

Have you tried running avr uarts at this rate and had good success? I'll try it out, for sure, but I find myself thinking it seems farfetched. The distance is only a few inches, but neither xmega has a crtstal (something i will change with next hw revision). Neither does either one currently self calibrate its clocks. (Something else I'll change.)

Steve

Top
damien_d
damien_d's picture
Level: Resident
Joined: Mon. Dec 15, 2008
Posts: 921 View posts
Location: Brisbane, Australia
Posted by damien_d: Sat. Aug 21, 2010 - 06:40 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

steverino wrote:

Interprocessor com: I'm stuck with the uart as the interprocessor communication channel. Is it realistic to run uarts at 460 or 920 kbaud and expect the < 1% error rate that atmel suggests is possible?

SD card: based upon the current system running at 16MHz, my gut tells me a data rate of 320Kbaud will overwhelm the FatFs and the SD card.

Some points to consider:

1. We're operating a UART (off a crystal) at 921600 baud at 60% utilisation without issues, but we use the DMA feature to do so.

2. You can spin the xmegas up to 32MHz with the internal oscillator. Accuracy is claimed to be 2% (iirc, maybe 1%), but you might be able to clock one from the other meaning clock drift between the two won't be an issue.

Do you know whether the UART or the SD card is the bottleneck? If you;re using the SPI on the USART, you can also use DMA mode on that which may relieve the CPU load talking to the SD card (if you're using it in that mode).

Finally, as you in a position to mod the hardware (just to rule this option in or out).

-- Damien

Top
Kartman
Kartman's picture
Level: 10k+ Postman
Joined: Thu. Dec 30, 2004
Posts: 24797 View posts
Location: Melbourne,Australia
Posted by Kartman: Sat. Aug 21, 2010 - 10:13 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Why is FatFs blocking interrupts? It shouldn'd need to unless it is a fix for some re-entrancy situation.

Top
Gold Markclawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 101194 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Sat. Aug 21, 2010 - 12:32 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

You end up with 10 1k blocks read from a random start address. On a real OS you get an access violation.

Yeah I know that - so what? The whole point was to simply write some random data - my use of rand() and casting it to a byte pointer was hardly an accident was it? :?

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Thu. Sep 2, 2010 - 09:12 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I apologize for sucking you all in with such a question and then disappearing. I had to go provide some help to my out of town parents. I'll provide some feedback to your suggestions and questions above as I can. In the meantime, I thank you for your assistance to date.

Steve

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Thu. Sep 2, 2010 - 09:17 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
Why is FatFs blocking interrupts? It shouldn'd need to unless it is a fix for some re-entrancy situation.

Great question. It is no longer blocking interrupts. I had been experimenting while debugging my FatFs port. :oops:

Top
dak664
dak664's picture
Level: Raving Lunatic
Joined: Sun. Jun 15, 2008
Posts: 2118 View posts
Location: North Carolina USA
Posted by dak664: Thu. Sep 2, 2010 - 11:33 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Can you reduce the bandwidth requirements with data compression? High bits per channel and/or sample rates should allow for a lot of compression.

Top
sparrow2
sparrow2's picture
Level: Raving Lunatic
Joined: Mon. Oct 7, 2002
Posts: 7133 View posts
Location: Denmark
Posted by sparrow2: Fri. Sep 3, 2010 - 12:06 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
20 bytes of worth of sensor data sampled at a rate of 2K Hz
That sounds like a lot of data are you sure that all 20 byte change! (I would expect some of it to be a ID or ?)

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Fri. Sep 3, 2010 - 11:30 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Great. Just great.

Due primarily to the current and projected supply shortages of the xmega families in the required packages, atmel just got designed OUT of the product.

No, it wasn't my decision. I got designed out along with the xmega.

Thanks atmel.

Top