SD binary data logger...sanity check

Log in or register to post comments
Go To Last Post
10 posts / 0 new
Author
Message
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
#1
Posted by steverino: Mon. Jul 19, 2010 - 07:34 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I've been reading all I can find on data logging to an SD card, and subsequent uploading of data to a PC. Is there an issue with logging in binary form rather than ascii?

I plan to save 16 bit values in buffers of 256 entries. When a buffer is full, it shall be passed to f_write. I have no need of any data delimiters (TAB, comma, etc.).

I don't need to worry about sector allocation or about maintaining the FAT. The FatFs does all that for me. Once I open a file I can keep calling f_write until the entire card is full.

Uploading of data to a Windows PC will be primarily via uart transfer logged to a PC data file. (Not my decision.) I do need to worry about ordering of data (eg lsb first, endian) over the serial line.

Parsing the data from the PC file should not be a problem as long as the parser uses 16 bit access.

Have I forgotten something? I do apologize for asking you to verify my design. I have worked primarily with ascii data files recently and am feeling a bit rusty.

I appreciate any constructive comments you may have.

Steve

Tags:

AVR Microcontrollers, megaAVR and tinyAVR
Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Mon. Jul 19, 2010 - 08:22 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Smack.

It's not necessarily an either or situation, is it? I can log the data in binary, and convert to ascii before putting it into the uart stream. The expense of ascii is limited to a non time and space critical activity. (Data logging and data upload are mutually exclusive in this application.)

Top
Gold Markclawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 101193 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Mon. Jul 19, 2010 - 10:16 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Why send it over slow old UART - just have the user unplug the SD card and plug it into the PC and then the processing software on the PC can just fopen() the file and read it direct - that's the whole point/advantage of using FAT because both the AVR and the PC can read/write it.

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Mon. Jul 19, 2010 - 02:24 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

That was my assumption when first hearing of the project. Then it became my recommendation when told of the need for a cable based transfer. The SD card is a little bit tedious to dig out, thus the serial xfer. Maybe when they get tired of waiting they'll dig out a screwdriver.

Steve

Top
Gold Markclawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 101193 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Mon. Jul 19, 2010 - 02:36 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well if this is still in the design phase consider making the cable a USB rather than RS232 cable. If you base the design on an AT90USB you could easily make the SD look like a USB-MTD device so when you plug in the USB cable it would look like a "USB memory stick" and you could just drag/drop files to it for easy transfer. If you go with UART you then need a data transfer protocol like Xmodem/Ymodem or similar.

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Mon. Jul 19, 2010 - 07:44 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

After an admittedly cursory read, I see the AT90USB is a uart-usb gateway. If as you suggested I build it into a cable, then my current system still uses its uart to connect to the world. The "cable" simply makes my current system appear to the host as a USB device. I'd still have to live with uart speeds, but I would get drag and drop file xfer. Hmmm.

Okay, stupid questions...
Is the AT90USB available preprogrammed for this purpose. It appears it would be nearly the same code as used in the xplain's at90usb.

There must be some api which my current system would have to meet to provide file lists and such over the usart. Do you by chance have a pointer? Like many things atmel, I didn't find it.

Steve

Top
Bronze Marktheusch
theusch's picture
Level: 10k+ Postman
Joined: Mon. Feb 19, 2001
Posts: 40014 View posts
Location: Wisconsin USA
Posted by theusch: Mon. Jul 19, 2010 - 07:49 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

After an admittedly cursory read, I see the AT90USB is a uart-usb gateway.

The suggestion was more to use an AT90USB___ family member as your AVR model of choice. It would have your application, as well as built-in USB support for host communications.

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Tue. Jul 20, 2010 - 12:04 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Got it.

Thanks.

Top
Gold Markclawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 101193 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Tue. Jul 20, 2010 - 01:16 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

My suggestion to use an AT90USB was not to use it as a USB-UART as that (VCOM under CDC) is just a very small fraction of what it's capable of. Because it's a fully programmable USB Device it can also act as an MTD class device. MTD (Memory Technology Device) is exactly what "USB memory sticks" are. You must be familiar with the fact that if you plug a USB memory stick into a PC then it's immediately recognised as the "next drive" in the system. So if you have drives C: and D: already then it might appear as a drive E: and you can do operations to it just as you would to any hard drive. So you can view it with exporer.exe, copy files to/from it, rename them, delete them and so on.

If you setup an AVR AT90USB based design for this logger then it could operate so that as soon as you plug it (either directly or at the other end of USB cable) into a PC you could do:

D:\>dir *.txt
 Volume in drive D is DATA
 Volume Serial Number is 3636-53FD

 Directory of D:\

01/09/2009  11:31             1,158 disks.txt
26/10/2009  11:20             8,294 junk3.txt
               2 File(s)          9,452 bytes

and the data you had logged would immediately be visible as .dat or .txt or whatever files on the device. You don't need special software like Hyperterminal to connect to and copy files from the logger you literally just:

D:\>copy disks.txt c:\data_store\

(or however it is you do that in the new money). This would make your device far easier for the user to interact with. If, for example, the data you'd written on the AVR was in ASCII format you could even:

D:\>type junk3.txt
echo 'CYCLE_IN_DAY_DEF,33,44,"08:00:00","09:00:00"
CYCLE,44,"241~241~241~241~241~241~241~241~241~241~241~241~241~241~241~241~241~24
1~241~241"
CYCLE,44,"241~241~241~241~241~241~241~241~241~241~241~241~241~241~241~241~241~24
1~241~241"
CYCLE,48,"271~241"
CYCLE,48,"271~241"
CYCLE,49,"243~271~243~271"
CYCLE,49,"243~271~243~271"
CYCLE,50,"243"
CYCLE,50,"243"
CYCLE,51,"271~271~271~271"
CYCLE,51,"271~271~271~271"
CYCLE,52,"241"
CYCLE,52,"241"
CYCLE,53,"272"
CYCLE_CAMPAIGN_TYPE,44,241,300000
CYCLE_CAMPAIGN_TYPE,44,241,300000
CYCLE_CAMPAIGN_TYPE,48,241,15000
CYCLE_CAMPAIGN_TYPE,48,271,15000
CYCLE_CAMPAIGN_TYPE,48,241,15000
CYCLE_CAMPAIGN_TYPE,48,271,15000
CYCLE_CAMPAIGN_TYPE,49,243,20000
CYCLE_CAMPAIGN_TYPE,49,271,10000
CYCLE_CAMPAIGN_TYPE,49,243,20000
CYCLE_CAMPAIGN_TYPE,49,271,10000
CYCLE_CAMPAIGN_TYPE,50,243,60000
CYCLE_CAMPAIGN_TYPE,50,243,60000
CYCLE_CAMPAIGN_TYPE,51,271,10000
CYCLE_CAMPAIGN_TYPE,51,271,10000
CYCLE_CAMPAIGN_TYPE,52,241,20000
CYCLE_CAMPAIGN_TYPE,52,241,20000
CYCLE_CAMPAIGN_TYPE,53,272,20000
CYCLE_IN_DAY_DEF,33,44,"14:00:00","15:00:00"

Top
steverino
steverino's picture
Level: Rookie
Joined: Tue. Jun 29, 2010
Posts: 37 View posts
Posted by steverino: Tue. Jul 20, 2010 - 05:24 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I've been bouncing around on this concept since I first read your suggestion. At first I correctly assumed the at90usb was just what it is...a ucontroller with well integrated usb capability. I even wrote a reply on that. Then I looked at the xplain, didn't really understand why they'd put two fairly complete ucontrollers on there, noticed an atmel quote refering to the at90usb as a usb-uart gateway, so deleted my reply and wrote what you saw above.

:-) Sometimes it's just plain fun to retrace one's thought processes.

Anyway, back to my earlier deleted reply...the project is far enough along to prevent changing controllers right now. We're at a working well prototype stage and adding the SD card is just to assist in data gathering for verfication. They can just open the box and dig out the SD card. They might even be happy about doing so when I tell them how long it'd take to xfer the data via serial.

I apologize for the mental flailing and dragging you two guys along with it. I'd like to think that normally I'd take a little more time to understand better before posting, but the workload is just plain huge right now.

Thanks, though, for the input. I appreciate it.

Top