clawson's expertise required (reading FAT32 directories)

Go To Last Post
18 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi clawson, quick question!  When I create more than 63 folders in FAT32 only 63 are displayed when I insert the media into the computer.  Now 63 is a clue I'm guessing!

Can you think of any of the folder fields would cause this error if not set properly?

 

Thanks,

 

Wm.

 

EDITED: The FAT Directory is Okay!

EDITED: When I change the partition to another one it works!, getting there.

This topic has a solution.
Last Edited: Thu. Sep 12, 2019 - 04:12 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Fianawarrior wrote:
I insert the media into the computer.
What "computer" and what operating system/filing system?

 

Anyway you are right that the binary split is indicative.

 

I guess it's really handling 64 not just 63. Now a directory entry is 32 bytes and sectors are 512 bytes so there are 16 directory/file entries per sector. So the issue seems to be a 4 sector thing? I kind of hope you are not using 4 sector clusters but if you were I would say you are failing to transition a cluster boundary.

 

Anyway dd (or otherwise copy) the first few MB of the device data and make it available in the internet somewhere. If you .zip it I think you can attach up to 10MB to a post here. The key thing is to see what a tool like WinHex makes of the image.

 

Oh and I've edited the thread title.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Think I've traced it down to the function that allocates the FAT Entry.  See Below:

 

 

int allocate_fat_entry(struct ff_file * stream){

	unsigned int free_fat = 0, * fat32;

	// allocate temporary storage to wipe clean the allocated fat entry
	char * data = safe_malloc(total_addr_space(stream));

	memset(data, 0, total_addr_space(stream));


	// allocate a sector to process
	char * fat = safe_malloc(endian16(stream->Master_Parameter_Block->BPB_BytsPerSec));

	unsigned int fat_table_addr =  endian16(stream->Master_Parameter_Block->BPB_ResvdSecCnt) + stream->partition_index;//

	int fat_cluster = 0;

	for(fat_cluster = 0; fat_cluster < endian32(stream->Master_Parameter_Block->BPB_FATSz32); fat_cluster++){



		// read the first entries in the FAT table
		dma_hardware_read(1, fat_table_addr + fat_cluster, 1, fat, 0);

		// system is using FAT32, 32 bits per FAT location
		fat32 = (unsigned int *)fat;


		// search for free fat entry
		for(free_fat = 2; ((* fat32 != 0x00000000) && (free_fat != 130)); free_fat++, fat32++);



		if(* fat32 == 0x00000000){

			* fat32 = 0x0fffffff;
			dma_hardware_write(1, fat_table_addr + fat_cluster, 1, fat, 0);
			break;
		}
	}

	free_fat += (fat_cluster * 128);


	// clear located new fat entry
	unsigned int addr = cluster_address(stream, free_fat);

	dma_hardware_write(1, addr, stream->Master_Parameter_Block->BPB_SecPerClus, data, 0);



	safe_free(fat);
	safe_free(data);

	return free_fat;
}

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I don't get it:

	// allocate a sector to process
	char * fat = safe_malloc(endian16(stream->Master_Parameter_Block->BPB_BytsPerSec));

that presumably allocates 512 bytes? OK then you have:

		// read the first entries in the FAT table
		dma_hardware_read(1, fat_table_addr + fat_cluster, 1, fat, 0);

so that reads one sector's worth to the buffer but then you have:

		// search for free fat entry
		for(free_fat = 2; ((* fat32 != 0x00000000) && (free_fat != 130)); free_fat++, fat32++);

what are the significances of 2 and 130 in this? Surely there's 64 lots of 32 bit FAT entries in 512 bytes?

 

Also don't do this:

fat32 = (unsigned int *)fat;

if you meant

fat32 = (uint32_t *)fat;

One of those is portable, one isn't.

 

In fact you should not use unsigned/signed char, short, int, long, long long in any code you write, especially not if this  is a filing system intended for multiple architectures. The one base type it's OK to use is "char" but only when you are dealing with characters.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

"what are the significances of 2 and 130 in this? Surely there's 64 lots of 32 bit FAT entries in 512 bytes?"

 

512 bytes/ 4 bytes = 128;

 

The offset is for the first cluster being 2

 

 

 

EDITED: I'm wrong in my assumption that I must offset the FAT Table by two. 

Last Edited: Wed. Sep 11, 2019 - 05:59 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The last folder it displays now is 128.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The 2 offset is only at the start of the first sector in the FAT

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:

The 2 offset is only at the start of the first sector in the FAT

 

Yip.  I have another strange exFAT setting.  The offset is 4 and not 2...  Doesn't give but I'll keep it in my head in case another media is different.

As to my problem with it only displaying 128 folders in Windows 10 I'll sleep on it.  Even a clue would do wonders for my head.  I can't think what it could be. 

 

 

EDITED: Clue, the root folder will only display 128 folders and inside another folder it will display another 128 folders.  The FAT allocation is working!

 

 

 

So my question is, would one of the folder fields, say the source folder size field need to be set to display more than 128 folders?

Last Edited: Wed. Sep 11, 2019 - 08:33 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I check the memory buffer!  Okay!  Was not big enough.  Then when I was formatting the drive I released that the default allocation option was 4096.  

 

God I was STUPID!!!!

 

Sorry clawson for wasting your time. 

 

EXPLAINATION: I was working on the assumption that the allocation unit was larger than that as I had been working on exFAT before switching back to FAT32.  I'm working on the part that spans multiple allocations.

Last Edited: Wed. Sep 11, 2019 - 10:25 PM
This reply has been marked as the solution. 
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

But why does it stop listing after one AU? The fact is that directories are just like multi-AU files. When you are reading the directory and you reach the AU boundary (4096 in your case) then you need to go back to the FAT and find the entry for the current AU then see if it contains the FFFFFFF7 end of chain marker or does it have the number of another AU - if so then continue reading the directory there.

 

In older FAT systems (FAT12 and FAT16 (I think!)) then the root dir was limited to a single AU but in FAT32 it's a normal file chain.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I have not yet implemented the code to span clusters, that's what I'm working on

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

OK so with 4K AUs then 128 is bound to be the limit for now.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Clawson, just how big should a directory chain of clusters be?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

As big as it needs to be. You don't put any limit on file chains (well maybe there is something like a 4GB limit?) so why would you put a limit on directories? A directory is really just a file made up of a load of the 32 byte entries.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0


Hi Clawason, how should I manage the root cluster in exFAT?  Its already marked with the value 3 shown below.  Can I just go ahead and update the entry for the next cluster chain in root?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Dunno exFAT so cannot help. I did post a link to the document M$ have released about it about a week ago, does that not help?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

had a look at a number of exFAT formats.  There appears to be two different methods for calculating the offset in the exFAT File Allocation Table.  I'll start coding tomorrow to rite a function that will scan the File Allocation Table to calculate the offset.

 

 

Thanks Clawson

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Had to scan the start of the FAT table to find the offset for the root.  See Below:]

 

unsigned int scan_fat(struct ff_file * stream){

	// point byte map to temporary data used for processing
	unsigned int * ByteMap = (unsigned int *)stream->cache_base;

	unsigned int fat_offset = 0;


	dma_hardware_read(1, data_to_cpu32(stream->Bios_Parameter_Block->FATFisrtSector) + stream->partition_index, 1, stream->cache_base, 0);


	if((*ByteMap == 0xfffffff8) && (*(ByteMap + 1) == 0xffffffff) && (*(ByteMap + 2) == 0x00000003)){


		ByteMap += 2;
		fat_offset = 2;

		while((*ByteMap + 1) == (*(ByteMap  + 1))){

			fat_offset++;
			ByteMap++;
		}

		if((*ByteMap + 2) == (*(ByteMap  + 2)))
			return (fat_offset + 2);
	}
	return 0;
}

Note, this is only one type of header.  There are others that need to be parsed.