Architectural/design question: fast data logger

Go To Last Post
21 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi,

I find myself back into low level embedded work after a hiatus of a few years. I've been asked to improve an existing data gathering system. I could use some suggestions from the experts here.

The system has two xmega32's: the first gathers and processes sensor data, and passes it along to the second. The second shoves the data into an SD card via Chan's FatFs. The data path between the two xmegas is uart.

I've been asked to up the data acquisition rate to 320Kbaud.

Interprocessor com: I'm stuck with the uart as the interprocessor communication channel. Is it realistic to run uarts at 460 or 920 kbaud and expect the < 1% error rate that atmel suggests is possible?

SD card: based upon the current system running at 16MHz, my gut tells me a data rate of 320Kbaud will overwhelm the FatFs and the SD card.

SD card alternatives: SPI sdram? Something usable as one large data spacewith no allocation overhead seems the way to go. Ideas other than sdram? Non vol is not a requirement.

Thanks,
Steve

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Sorry...capacity...
4+ hours of data --> 4.6Gbits of storage.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Are the two AVRs run from the same clock or crystals? If so then you can use 1M or 2M baud rate if you like with no worries.

As for SD write speeds. Using the 4 wire SD connections they can do 1M..2M/s but using MMC one wire (as almost all AVR designs do) the badnwidth is going to be much lower. I'd imagine it's the SD write speed that will actually be your bandwidth limiter, not the AVR<->AVR link. As a test I guess you could just open a file, and keep fseek'ing and rewriting the same test data to get an idea of the bandwidth that can be achieved.

Cliff

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Each xmega is running off its internal clock.

I will try your file write suggestion with various buffer sizes.

Thanks.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
you could just open a file, and keep fseek'ing and rewriting the same test data to get an idea of the bandwidth that can be achieved.

Cliff


My current SD access mode is to open a file and leave it open. I haven't been seeking at all. Should I be?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You don't have to seek but if you don't then you are going to create a very big file - maybe that doesn't matter?

As for the clocks, I guess with the guaranteed 1% accuracy of the Xmega maybe it'll be OK but for high speed UART I think I'd be using a more accurate clock source.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
You don't have to seek but if you don't then you are going to create a very big file - maybe that doesn't matter?

Yes, that's not a problem.

clawson wrote:
As for the clocks, I guess with the guaranteed 1% accuracy of the Xmega maybe it'll be OK but for high speed UART I think I'd be using a more accurate clock source.

I've been promoting a hardware spin to do just that. (Thinking ahead to a likely suggestion from someone, a switch from uart to spi is not possible, even with the hardware rev.)

Thanks!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
You don't have to seek but if you don't then you are going to create a very big file - maybe that doesn't matter?

I'm curious. Why is that? At first I wondered about each small write using a larger block of SD space. Not sure how a seek would change things, though.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I simply mean normal file access. If in POSIX land I do:

FILE * fout;

int main(void) {
  int i;
  char * ptr;

  fout = fopen("file.dat", "w");
  for (i=0 ; i<10 ; i++) {
    ptr = (char *)rand();
    fwrite(ptr, 1024, 1, fout);
  }
  fclose(fout);
}

I end up with a file holding 10K of random data. While if I do:

FILE * fout;

int main(void) {
  int i;
  char * ptr;

  fout = fopen("file.dat", "w");
  for (i=0 ; i<10 ; i++) {
    ptr = (char *)rand();
    fwrite(ptr, 1024, 1, fout);
    fseek(fout, 0, SEEK_SET);
  }
  fclose(fout);
}

I still end up writing 10K of data (and taking almost exactly the same length of time) but I end up with a 1K file.

I was just concerned you may have file size problems if you went with method A but I guess it takes a while to write 4..8GB from an AVR!

But forget the fseek() thing.

Cliff

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
I simply mean normal file access. If in POSIX land I do:

FILE * fout;

int main(void) {
  int i;
  char * ptr;

  fout = fopen("file.dat", "w");
  for (i=0 ; i<10 ; i++) {
    ptr = (char *)rand();
    fwrite(ptr, 1024, 1, fout);
  }
  fclose(fout);
}

I end up with a file holding 10K of random data.


You end up with 10 1k blocks read from a random start address. On a real OS you get an access violation.

I have read somewhere that SD cards delay writing from time to time, so it will be useful to measure the time that writing each block takes, so you know the maximum delay time. Just measuring an average data rate is not sufficient, as you will run out of buffers if the maximum delay is too high.
You could also directly check that you get the sustained datarate you need.
You have to benchmark and completly fill each card you want to use as the write speed is not constant on all parts of the card.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

320Kbps isnt a standard rate... you could sample at that rate, then send it over at max baud divisor... 1 mbps. The 320Kbps is about 40K Bps... 80 sector writes a sec, 12 ms per sector. Those things write about 5ms a sector, right?

Imagecraft compiler user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The 320K bps I quoted is based on the data requirements: 20 bytes of worth of sensor data sampled at a rate of 2K Hz.

FatFs blocks interrupts regularly during writes. On the SD logging xmega I do run the uarts on interrupts, but since FatFs is blocking I see the need to avoid both tasks happening simultaneously. I've been sneaking small SD writes inbetween uart interrupts. This is sort of okay for my current 100 Hz rate, but I don't like the scheme and it sure won't scale up to the target 2K Hz sampling.

I feel like I need a fairly fundamental change.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

bobgardner wrote:
...send it over at max baud divisor... 1 mbps.

Have you tried running avr uarts at this rate and had good success? I'll try it out, for sure, but I find myself thinking it seems farfetched. The distance is only a few inches, but neither xmega has a crtstal (something i will change with next hw revision). Neither does either one currently self calibrate its clocks. (Something else I'll change.)

Steve

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

steverino wrote:

Interprocessor com: I'm stuck with the uart as the interprocessor communication channel. Is it realistic to run uarts at 460 or 920 kbaud and expect the < 1% error rate that atmel suggests is possible?

SD card: based upon the current system running at 16MHz, my gut tells me a data rate of 320Kbaud will overwhelm the FatFs and the SD card.

Some points to consider:

1. We're operating a UART (off a crystal) at 921600 baud at 60% utilisation without issues, but we use the DMA feature to do so.

2. You can spin the xmegas up to 32MHz with the internal oscillator. Accuracy is claimed to be 2% (iirc, maybe 1%), but you might be able to clock one from the other meaning clock drift between the two won't be an issue.

Do you know whether the UART or the SD card is the bottleneck? If you;re using the SPI on the USART, you can also use DMA mode on that which may relieve the CPU load talking to the SD card (if you're using it in that mode).

Finally, as you in a position to mod the hardware (just to rule this option in or out).

-- Damien

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Why is FatFs blocking interrupts? It shouldn'd need to unless it is a fix for some re-entrancy situation.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

You end up with 10 1k blocks read from a random start address. On a real OS you get an access violation.

Yeah I know that - so what? The whole point was to simply write some random data - my use of rand() and casting it to a byte pointer was hardly an accident was it? :?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I apologize for sucking you all in with such a question and then disappearing. I had to go provide some help to my out of town parents. I'll provide some feedback to your suggestions and questions above as I can. In the meantime, I thank you for your assistance to date.

Steve

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
Why is FatFs blocking interrupts? It shouldn'd need to unless it is a fix for some re-entrancy situation.

Great question. It is no longer blocking interrupts. I had been experimenting while debugging my FatFs port. :oops:

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Can you reduce the bandwidth requirements with data compression? High bits per channel and/or sample rates should allow for a lot of compression.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
20 bytes of worth of sensor data sampled at a rate of 2K Hz
That sounds like a lot of data are you sure that all 20 byte change! (I would expect some of it to be a ID or ?)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Great. Just great.

Due primarily to the current and projected supply shortages of the xmega families in the required packages, atmel just got designed OUT of the product.

No, it wasn't my decision. I got designed out along with the xmega.

Thanks atmel.