[C#] Reading the Master File Table http://www.allegro.cc/forums/view/604162 Allegro.cc Forum Thread matthew@allegro.cc (Matthew Leverton) Wed, 02 Jun 2010 01:12:34 +0000

Is there any way to gain access and parse the $MFT using only C# methods? I've seen a couple of methods, including using PInvoke, or importing lots of DLLs, but they seem really confusing. I'm hoping there's a better way to do that?

I need the full file/directory tree of all drives and I'm just not going to parse the entire hard drive to get that :P

]]> no-reply@allegro.cc (CursedTyrant) Wed, 26 May 2010 03:25:06 +0000

You'll probably have to use PInvoke. It's not that bad though, and for standard Win32 API, pinvoke.net has prewritten bindings for pretty much everything.

]]> no-reply@allegro.cc (Kibiz0r) Wed, 26 May 2010 04:28:39 +0000

I highly doubt that sort of thing would be exposed via a .NET api. Using Invoke (platform invoke) with the relevant API is probably your best bet.

]]> no-reply@allegro.cc (BAF) Wed, 26 May 2010 07:30:20 +0000

Does (as in I can't be bothered to search in more depth) the WMI (http://en.wikipedia.org/wiki/Windows_Management_Instrumentation) let you do this? I use it to interrogate processes and stuff and I bet it has file table capabilities.

]]> no-reply@allegro.cc (Neil Walker) Wed, 26 May 2010 18:24:42 +0000

I've managed to write something like this, tough it's probably horribly, horribly fucked up. Can anyone take a look and tell me if and what I'm doing wrong? Also, I want to know what should I do next :P That piece of code below is based on something I found on some forum:

Quote:

Call the api function <DeviceIoControl> with the FSCTL_GET_RETRIEVAL_POINTERS ControlCode to find out, where the MFT is located on the volume ( the MFT can be fragmented ).

Create a volume-handle with the api function <CreateFile>.

Set a pointer to the mft-clusters on the volume. Call the api function <SetFilePointerEx> to do this. You can use the created volume-handle on this function call.

Read the clusters with the api function <ReadFile>.

#SelectExpand
1public const int FSCTL_GET_RETRIEVAL_POINTERS = 0x00090073; 2 public const int FILE_CURRENT = 1; 3 4 [DllImport("kernel32.dll", ExactSpelling = true, SetLastError = true, CharSet = CharSet.Auto)] 5 static extern bool DeviceIoControl(IntPtr hDevice, uint dwIoControlCode, 6 IntPtr lpInBuffer, uint nInBufferSize, 7 IntPtr lpOutBuffer, uint nOutBufferSize, 8 out uint lpBytesReturned, IntPtr lpOverlapped); 9 10 [DllImport("Kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)] 11 public static extern IntPtr CreateFile( 12 string fileName, 13 [MarshalAs(UnmanagedType.U4)] FileAccess fileAccess, 14 [MarshalAs(UnmanagedType.U4)] FileShare fileShare, 15 IntPtr securityAttributes, 16 [MarshalAs(UnmanagedType.U4)] FileMode creationDisposition, 17 int flags, 18 IntPtr template); 19 20 [DllImport("kernel32.dll")] 21 static extern bool SetFilePointerEx(IntPtr hFile, long liDistanceToMove, 22 IntPtr lpNewFilePointer, uint dwMoveMethod); 23 24 [DllImport("kernel32.dll", SetLastError = true)] 25 private unsafe static extern bool ReadFile( 26 IntPtr hFile, // handle to file 27 byte[] lpBuffer, // data buffer 28 int nNumberOfBytesToRead, // number of bytes to read 29 ref int lpNumberOfBytesRead, // number of bytes read 30 int* ptr 31 // 32 // ref OVERLAPPED lpOverlapped // overlapped buffer 33 ); 34 35 public static unsafe void GetMFT() 36 { 37 IntPtr inputBuffer = IntPtr.Zero; 38 IntPtr outputBuffer = IntPtr.Zero; 39 40 uint lpBytesReturned = 0; 41 42 UInt64 n = 0; 43 IntPtr ptr = new IntPtr(&n); 44 45 IntPtr tmpHandle = CreateFile(@"\\\\.\\C:", FileAccess.Read, FileShare.ReadWrite, IntPtr.Zero, FileMode.Open, 0, IntPtr.Zero); 46 DeviceIoControl(tmpHandle, FSCTL_GET_RETRIEVAL_POINTERS, inputBuffer, (uint)Marshal.SizeOf(inputBuffer), outputBuffer, (uint)Marshal.SizeOf(outputBuffer), out lpBytesReturned, IntPtr.Zero); 47 SetFilePointerEx(tmpHandle, 0, ptr, FILE_CURRENT); 48 byte[] data = doReadFile(ptr, sizeof(IntPtr)); 49 } 50 51 public static unsafe byte[] doReadFile(IntPtr argHandle, int InputReportByteLength) 52 { 53 int BytesRead = 0; 54 byte[] BufBytes = new byte[InputReportByteLength]; 55 if (ReadFile(argHandle, BufBytes, InputReportByteLength, ref BytesRead, null)) 56 { 57 byte[] OutBytes = new byte[BytesRead]; 58 Array.Copy(BufBytes, OutBytes, BytesRead); 59 return OutBytes; 60 } 61 else 62 { 63 return null; 64 } 65 }

]]> no-reply@allegro.cc (CursedTyrant) Fri, 28 May 2010 01:58:37 +0000

Does it work? If not, what doesn't work? :P Personally, I've never directly interfaced a C# application with anything non-.NET before.

]]> no-reply@allegro.cc (bamccaig) Fri, 28 May 2010 03:15:58 +0000

It's not really a problem with interfacing .NET with non-.NET stuff. It's mostly just the fact that I have no idea what the hell should I do. :P There doesn't seem to be a clear documentation/paper/tutorial/book/whatever on how to read a directory tree from $MFT and I just don't know how to proceed.

]]> no-reply@allegro.cc (CursedTyrant) Fri, 28 May 2010 03:18:28 +0000

What are you trying to accomplish? There probably isn't clear documentation because this is quite an unusual task...

]]> no-reply@allegro.cc (BAF) Fri, 28 May 2010 03:31:15 +0000

I need the full directory & file tree of each logical drive, and I don't want to parse directories/files just to get that (I've tried, it's just too damn slow). Since $MFT contains records for every directory and file on a logical drive, I want to extract that data from there (it would be a lot faster).

EDIT: So, does anyone know how to: a) read the MFT and b) iterate through and read all the MFT records?

]]> no-reply@allegro.cc (CursedTyrant) Wed, 02 Jun 2010 00:14:03 +0000

I imagine there must already be existing code for this. GNU find can do it, albeit it's still quite slow for an entire file system (I think "logical drive" is irrelevant in UNIX's user land). The alternative is using an index, like GNU locate/updatedb, which is actually quite fast.

Oh, ... Windows? Accept defeat. You might try Cygwin, but it obviously depends on just what you're doing where. .NET probably means that GNU findutils won't work very well for you.

I still find it hilarious how easily my entire[1] Linux file system is indexed (and has been for years) compared to Windows. IIRC, Windows 7 only indexes user folders and the like (and even that is slow), making the Start Menu search feature nearly useless for a power user, and barely useful even for a regular user.

While I'm ranting, I also find it infuriating that Windows still lacks a decent "Home folder" concept. It sort of exists (%USERPROFILE%[2], anyone?), but Windows Explorer doesn't list it as a quick jump, and indeed still uses fake paths when going to "special" folders, like "Pictures" AKA "My Pictures" (or should I say "Libraries > Pictures", WTF?!), making you have to click two or three times to get to root of the home folder.

References

  1. Well I think there are probably some excluded areas by default, but for actual good reasons.
  2. 13 keystrokes, if anyone is counting.
]]> no-reply@allegro.cc (bamccaig) Wed, 02 Jun 2010 00:30:51 +0000

I'm sure there's some code that does just that, but I searched and searched, and found nothing. I don't care if it's C++ (it's pretty much the only way to do this, and I can use pinvoke.net as reference for methods I'd need), I just don't know how to do what I want. Like I said before, $MFT contains a record for every file and directory on the logical drive and thus, parsing it would be a hell of a lot faster (even if it's more than just a few MBs) than parsing every directory and file on the hard drive.

Any ideas?

]]> no-reply@allegro.cc (CursedTyrant) Wed, 02 Jun 2010 00:42:24 +0000

I'd recommend studying the API as if you were in C++, ignore pinvoke for the moment and figure out how you would do it using the plain API. Then move it to pinvoke.

And if all else fails, $MFT is an actual file on the filesystem, so you can open it and parse it yourself.

]]> no-reply@allegro.cc (Kibiz0r) Wed, 02 Jun 2010 00:47:39 +0000
Kibiz0r said:

And if all else fails, $MFT is an actual file on the filesystem, so you can open it and parse it yourself.

How is that even possible? :P I'm thinking of chickens and eggs. :-X

]]> no-reply@allegro.cc (bamccaig) Wed, 02 Jun 2010 01:07:47 +0000
bamccaig said:

How is that even possible? :P I'm thinking of chickens and eggs. :-X

Its most likely a virtual file, just a pointer to a section of disk, rather than a real file entry.

]]> no-reply@allegro.cc (Thomas Fjellstrom) Wed, 02 Jun 2010 01:10:30 +0000

@CursedTyrant: Have you stumbled across this yet? He seems to claim to have done (and be doing) exactly what you want.

]]> no-reply@allegro.cc (bamccaig) Wed, 02 Jun 2010 01:12:34 +0000