AVR Freaks :: View topic - LUFA USB Mass Storage Misuse

AVR Freaks

AVR GCC forum - LUFA USB Mass Storage Misuse

curtvm - Mar 19, 2009 - 02:35 PM
Post subject: LUFA USB Mass Storage Misuse

This is just ideas I have, and would like comments from those with usb knowledge on whether I'm barking up wrong trees. (I'm late to the USB scene).

I have a USBKey, Dean's LUFA drivers, and a brain that doesn't easily wrap itself around things pertaining to USB. Anyway, I like Mass Storage devices, since no 'special' drivers are needed, and any os (half-way modern at least) can 'talk' to it. I would also like to communicate with the avr from the pc, but do not want to deal with drivers (dll's or whatever the unixes use). For example, the USBKey for its built-in bootloader uses drivers (libusb+inf I think), the rs232-usb on the Win side you need an inf, and using HID for 'general' comm use you need to talk to a driver, etc. (I"m not an expert, so this is just what I understand to be the case).

Now I suppose one could learn to use these things, but I'm thinking I want to communicate by way of Mass Storage instead. Any os can deal with files, any programming language can deal with files, so I figure why not communicate with the avr as a file based device (or 'block' device, I guess).

I took the USBKey, the LUFA drivers, and made myself a 512 byte 'ram' drive (1 mbr, 1 fat table, 1 dir sector, 1 data sector). After playing around a lot, I got it to the minimal amount of data needed in the mbr,fat,dir to make it work without bringing Windows to its knees (Windows is one 'baby' os- it can choke on about anything you feed it that it doesn't like in the mbr/fat/dir). (Also for some reason, Windows needed at least 8 physical sectors to be happy, even though my ram drive was only 4 sectors). When booting to Fedora via a live cd, it was easier to please (and never started choking when things were screwed up).

The 'ram' drive as a ram drive it not real useful by itself, but it does provide a 'window' in which to communicate.

Next I made 3 dir entries- FLASH.AVR, EEPROM.AVR, and SRAM.AVR, and generated a fat table to match (along with appropriate file sizes, and an increase in the 'drive' sector count). Depending on which sector was going to be read by the pc, it would return either the ram based sectors (0-2, mbr/fat/dir), the flash (next 256 sectors), eeprom (8 sectors), or sram (16 sectors, io addresses are now returned 0 as I didn't want to read them as not all can be read without side effects). So basically I can plug in the USBKey, read the 3 files which read flash, eeprom and sram (pretty fast, too).

Right now, I can only read these files, but there is no reason writing can't be done to flash/eeprom/sram (and io with some extra care). A bootloader comes to mind- where the 'upload' is simply an upload of a binary file.

There are many variations one can dream up, and I'm sure this is 'old news' to those that deal with anything bigger than a little avr. (I think I'm into unix territory, where 'everything' is a file. Or something like that.)

So, is Mass Storage Misuse a bad idea, good idea, unnecessary, or none of the above? Assuming I want to write my own app to communicate to an avr via usb, is there an easier way than making it 'file' based as I am attempting? (and can be cross platform compatible). I suppose the cdc device would be the 'standard' way to go, but then I have to deal with serial port programming on the pc side, which is not as easy as 'file' programming.

I can probably post my examples for the usbkey if anyone is interested (its not very pretty at the moment, though).

clawson - Mar 19, 2009 - 02:51 PM
Post subject: RE: LUFA USB Mass Storage Misuse

Quote:

(I think I'm into unix territory, where 'everything' is a file. Or something like that.)

That's exactly what I was thinking - this is a very *nix'y way of looking at the world.

Yesterday I just wrote a small application to allow text to be displayed in an "on screen display" that is "on top" of other X11 graphics in a Linux system - the way I did the interface was to have it "watch" a file, then to control it I just write text or control commands into that file.

So your idea here seems very familiar. Obviously the other way you could do it is to make the USB look like a CDC class device (a comm port in fact) then on the PC side (because all USB aware OS support CDC) you just need a standard terminal program - but that's probably a bit too obvious.

Cliff

curtvm - Mar 19, 2009 - 03:58 PM
Post subject: RE: LUFA USB Mass Storage Misuse

Quote:

the other way you could do it is to make the USB look like a CDC class device (a comm port in fact) then on the PC side (because all USB aware OS support CDC) you just need a standard terminal program

That seems to be the 'normal' route to go, and if I was to use 'off-the-shelf' apps such as terminals, then that would be the way to go I suppose. But if I want to write my own app, it seems a file based approach seems the easiest and most universal. Although technically it may not the case, it also seems more 'direct' to me. For example, the bootloader idea I floated, it seems more direct/simple to write to a file called FLASH.AVR than to communicate via a com port in some specific protocol/method/whatever to accomplish the same thing (would be about as simple as it gets- copy binary file to 'fake' file, read back file to verify).

Or if I wanted a file called PORTS.AVR and PINS.AVR where I could simply read/write 'directly' to these io registers as needed. Its like a direct link to the avr brain instead of having to communicate via some protocol. Make up whatever 'files' you want, and simply communicate via these 'files'.

I'm sure I haven't thought it out thoroughly enough yet, but I am coming up with quite a few ideas. I guess a 'Mass Storage Bootloader' would be a nice starter project (I think I'm already 80% there with what I have already, since there really is not much more to do).

One downside is you have to deal with 512 byte blocks (unless I'm missing some programming trick), but the speed is pretty fast so I don't think its a major drawback. I can read the 128k FLASH.AVR file in about a second.

wek - Mar 19, 2009 - 04:39 PM
Post subject: Re: RE: LUFA USB Mass Storage Misuse

curtvm wrote:

Quote:

That seems to be the 'normal' route to go, and if I was to use 'off-the-shelf' apps such as terminals, then that would be the way to go I suppose.

The only point of using a class is to use existing software.

OK, big part is to use existing drivers; but you might also want to use existing applications. Or, said in another way, nothing will prevent a user to use an existing application using files and disks, from using your implementation in the same way. Is your implementation robust enough to survive such an abuse?

The OS is also free to play whatever game it wants with MSDC - it can shuffle sectors at its will and write them in whichever order it finds convenient, and it can cache sectors on its own discretion. There might be other goodies I am not aware of.

curtvm wrote:

But if I want to write my own app, it seems a file based approach seems the easiest and most universal.

At the user level, there is little difference between using serial and files. You open and close it as a file, and in fact you can also read and write (just that it makes little sense to read without checking whether there is any character to read).

Both (all) classes are made for a purpose, and IMHO they should be used so.

JW

curtvm - Mar 19, 2009 - 06:19 PM
Post subject: RE: Re: RE: LUFA USB Mass Storage Misuse

Quote:

The OS is also free to play whatever game it wants with MSDC - it can shuffle sectors at its will and write them in whichever order it finds convenient, and it can cache sectors on its own discretion

I control the sectors, so I can 'generate' the mbr/fat/dir sectors on-the-fly and let the os 'write' the date/times on the dir entries to keep it happy. No changing file sizes, file names, or deleting files though (it won't happen, so the os/app certainly won't like that if attempted). Cached data is a problem, and could be a showstopper unless the os+app can 'force' a new read. I'm testing on Vista now, which seems more 'aggressive' on caching the sectors than XP was. More to learn.

wek - Mar 19, 2009 - 06:57 PM
Post subject: RE: Re: RE: LUFA USB Mass Storage Misuse

A read only FS is fully in your hands, but as long as you don't buffer all the data sectors, a malicious OS can write any funny sequence - write data sectors in an unexpected order, and AFTER that write FAT describing a weird intermixed structure of sectors.

Now you will ask: why would anybody do that? But that's not the question. The question is, what are you going to do to cope with this, whoever in Redmond, WA or elsewhere decides to write it so.

"I tested it and it worked" is good for a toy (and toys are a serious thing), but not a product.

JW

clawson - Mar 19, 2009 - 08:14 PM
Post subject: RE: Re: RE: LUFA USB Mass Storage Misuse

Quote:

a malicious OS can write any funny sequence

Yeah but the FAT, root, etc. only get written if a file is created/written. As long as you prevent that you are safe though I know that Windows has a nasty habit of "indexing" and writing hidden files on a supposedly dormant drive so you do need to stop that behaviour.

I've been bitten by this myself. The PVRs I program use FAT32 on their HDDs and to keep things simple they stick to old style 8.3 (single entry in directory) filenames. Then XP comes along in its hobnails and tramps all over everything with LFNs. Sad

In a decent operating system like Linux it's easy to persuade the OS to mount a drive as read-only so you know you can do "forensic analysis" on it unimpeded WinDos has nothing like this.

Cliff

curtvm - Mar 19, 2009 - 08:47 PM
Post subject: RE: Re: RE: LUFA USB Mass Storage Misuse

I'm just a 'hobby' guy, so nothing serious at stake, but I can see the os is going to be the problem. It does appear caching at the file level is a problem. Getting deeper than the file level like whatever a disk editor appears to be doing is not what I had in mind (supposed to be simple), so I may as well get in line behind everyone else and follow the rules.

At least I learned a little about the FAT file system and made myself a 512 byte ram drive (and other weird variations).

(Cliff- I was seeing Windows 'having a good time' in the directory sector even on my fat12 drives, and was a bit confused at first.)

simavr - Jun 25, 2010 - 08:41 AM
Post subject:

Hi curtvm,

I am also trying to make a tiny ram disk with FAT12 on it. I had no luck so far, windows detect the disk but only ask to format it. Would you be able to share your mbr, fat table, dir sector and data sector?

Thanks
Simon

curtvm - Jun 26, 2010 - 01:37 AM
Post subject:

Here is a minimal mbr sector-

Code:

EB 3C 90 4D 53 57 49 4E 34 2E 31 00 02 01 01 00 

01 10 00 04 00 F8 01 00 01 00 01 00 00 00 00 00

00 00 00 00 00 00 29 CC 63 F0 C8 4E 4F 20 4E 41

4D 45 20 20 20 20 46 41 54 31 32 20 20 20 00 00

...

00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA

4 sectors of 512bytes- 1mbr, 1fat, 1dir, 1data.

This would get you 1 data sector of 512 bytes.

simavr - Jun 26, 2010 - 09:31 AM
Post subject:

Thanks!!

I was setting the three first byte to 0x00. After changing that, XP is detecting the disk.