Articles » Languages » C# » COM Interop

System.IO.Directory Alternative using WinAPI

Marijan Nikic, 8 Apr 2019

Faster and better alternative to System.IO.Directory IEnumerable methods EnumerateDirectories, EnumerateFiles and EnumerateFileSystemEntries

Download DLL - 3.2 KB

Download source - 22.6 KB

Introduction

Recently, I was working on a project that needed to read the contents of a Windows directory, so I used the .NET provided System.IO.Directory class' EnumerateDirectories, EnumerateFiles and EnumerateFileSystemEntries methods. Unfortunately, there is a big downside to using these functions, and that is that if they run into a file system entry that has denied access to the current user, they immediately break - instead of handling such an error and continuing, they will just return whatever they have gathered up to the moment of breaking - and won't complete the job.

It is impossible to handle this from the outside of the methods because if you handle it, you will only get in the returning IEnumerable only partial results.

I have searched everywhere for a solution to this problem, but I was not able to find a workaround that doesn't use the aforementioned methods. So I decided to play around with Windows API and create alternative methods. The result was not only better (in a way that the methods do not break on "Access denied") but it appears to be even faster than the .NET's original methods.

Using the Code

The project itself is a Class Library type, it is not executable but building it will compile the methods into a DLL file, which you can reference into another project, and use it from there like this:

using System.IO;

DirectoryAlternative.EnumerateDirectories
(path, "*", SearchOption.AllDirectories).ToList<string>();

I used the same namespace as the original procedures (System.IO), and named the class DirectoryAlternative - so the usage would be as similar as possible to the original class.

The methods themselves are named the same way, they use the same parameters, and from the outside look absolutely the same as the original ones.

Here is an example of the usage of methods:

System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
string path = "V:\\MUSIC";
List<string> en = new List<string>();
sw.Start();
try { en = Directory.EnumerateDirectories
(path, "*", SearchOption.AllDirectories).ToList<string>(); } catch { }
sw.Stop();
Console.WriteLine("Directory.EnumerateDirectories : {0} ms / {1} entries",
sw.ElapsedMilliseconds.ToString("N0"), en.Count.ToString("N0"));
sw.Reset();
en = new List<string>();
sw.Start();
en = DirectoryAlternative.EnumerateDirectories(path, "*",
SearchOption.AllDirectories).ToList<string>();
sw.Stop();
Console.WriteLine("DirectoryAlternative.EnumerateDirectories :
{0} ms / {1} entries", sw.ElapsedMilliseconds.ToString("N0"), en.Count.ToString("N0"));
sw.Reset();
en = new List<string>();
sw.Start();
try { en = Directory.EnumerateFiles(path, "*",
SearchOption.AllDirectories).ToList<string>(); } catch { }
sw.Stop();
Console.WriteLine("Directory.EnumerateFiles : {0} ms / {1} entries",
sw.ElapsedMilliseconds.ToString("N0"), en.Count.ToString("N0"));
sw.Reset();
en = new List<string>();
sw.Start();
en = DirectoryAlternative.EnumerateFiles
sw.Stop();
Console.WriteLine("DirectoryAlternative.EnumerateFiles : {0} ms / {1} entries",
sw.ElapsedMilliseconds.ToString("N0"), en.Count.ToString("N0"));
sw.Reset();
en = new List<string>();
sw.Start();
try { en = Directory.EnumerateFileSystemEntries
sw.Stop();
Console.WriteLine("Directory.EnumerateFileSystemEntries : {0} ms / {1} entries",
sw.ElapsedMilliseconds.ToString("N0"), en.Count.ToString("N0"));
sw.Reset();
en = new List<string>();
sw.Start();
en = DirectoryAlternative.EnumerateFileSystemEntries
(path, "*", SearchOption.AllDirectories).ToList<string>();
sw.Stop();
Console.WriteLine("DirectoryAlternative.EnumerateFileSystemEntries : {0} ms / {1} entries",
sw.ElapsedMilliseconds.ToString("N0"), en.Count.ToString("N0"));

Console.ReadKey();

The above code snippet compares directly the original methods' performance and the DirectoryAlternative methods - I used a very large directory with 70.000+ file system entries:

As you can see, the DirectoryAlternative methods run almost 2x faster.

How the Code Works

The code uses several Win API functions to move around the file system (I believe these same functions are used in the original .NET methods):

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
struct WIN32_FIND_DATA
{
public uint dwFileAttributes;
public System.Runtime.InteropServices.ComTypes.FILETIME ftCreationTime;
public System.Runtime.InteropServices.ComTypes.FILETIME ftLastAccessTime;
public System.Runtime.InteropServices.ComTypes.FILETIME ftLastWriteTime;
public uint nFileSizeHigh;
public uint nFileSizeLow;
public uint dwReserved0;
public uint dwReserved1;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 260)]
public string cFileName;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 14)]
public string cAlternateFileName;
}

[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
private static extern bool FindClose(IntPtr hFindFile);

[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
private static extern IntPtr FindFirstFile
(string lpFileName, out WIN32_FIND_DATA lpFindFileData);

[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
private static extern bool FindNextFile
(IntPtr hFindFile, out WIN32_FIND_DATA lpFindFileData);

In short:

FindFirstFile searches for a first file system entry that it can find using the provided pattern (lpFileName) and returns a HANDLE (IntPtr) to this file

FindNextFile searches for the next file system entry that matches the specified pattern - we use this method to go through all the files / directories

FindClose is used to close the HANDLE

All the file information is gathered inside the WIN32_FIND_DATA struct and returned as an out type parameter.

For more information on these methods, you can look them up here.

The main method is the method Enumerate. All other methods are wrapped around this one.

private static void Enumerate(string path, string searchPattern,
SearchOption searchOption, ref List<string> retValue, EntryType entryType)
{
WIN32_FIND_DATA findData;
if (path.Last<char>() != '\\') path += "\\";
IntPtr hFile = FindFirstFile(path + searchPattern, out findData);
List<string> subDirs = new List<string>();

if (hFile.ToInt32() != -1)
{
do
{
if (findData.cFileName == "." || findData.cFileName == "..") continue;
if ((findData.dwFileAttributes &
(uint)FileAttributes.Directory) == (uint)FileAttributes.Directory)
{
subDirs.Add(path + findData.cFileName);
if (entryType == EntryType.Directories ||
entryType == EntryType.All) retValue.Add(path + findData.cFileName);
}
else
{
if (entryType == EntryType.Files ||
entryType == EntryType.All) retValue.Add(path + findData.cFileName);
}
} while (FindNextFile(hFile, out findData));
if (searchOption == SearchOption.AllDirectories)
foreach (string subdir in subDirs)
Enumerate(subdir, searchPattern, searchOption, ref retValue, entryType);
}
FindClose(hFile);
}

The method takes all the parameters from the original Enumerate methods (path, searchPattern, searchOption) plus a by-reference argument retValue, and entryType, which is an enum:

private enum EntryType { All = 0, Directories = 1, Files = 2 };

This enum serves as a selector whether only directories, only files or both should be returned.

The Enumerate method calls FindFirstFile and subsequently iterates through all other file system entries by calling FindNextFile. If entryType = Files, it will add all the files in the retValue list. For Directories, it will add only directories, and for All it will add both.

The method is called recursively if searchOption = AllDirectories. The results from all the (recursive) calls are gathered in one variable retValue. In the first version of the method, I used return type List and concatenated it after each recursive call into the retValue variable from the calling function, but the solution with by-ref argument proved to be much, much faster.

In the end, each call's HANDLE for file search will be closed by calling method FindClose.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Marijan Nikic

User Interface Analyst Raiffeisenbank Austria

Follow
this Member
Unfollow

Croatia

I acquired Masters degree in computing science on Faculty of Electrical Engineering and Computing in Zagreb, Croatia in 2009. Following my studies, I got a job in a Croatian branch of Austrian-based Central and Eastern European bank Raiffeisen Bank as an MIS (Management information systems) analyst.
I have been working there since 2010, as an IT expert within the Controlling department, maintaining the Oracle's OFSA system, underlying interfaces and databases.
Throughout that time, I have worked with several different technologies, which include SQL & PL/SQL (mostly), Cognos BI, Apparo, Datastage, ODI, Jenkins, ...
I have recently taken a lot of interest in scripting with VBScript and Windows batch scripting. Privately, I am mostly doing Windows Forms and Console apps in Visual Studio, in C#.

Comments and Discussions

4 messages have been posted for this article Visit https://www.codeproject.com/Articles/1383832/System-IO-Directory-Alternative-using-WinAPI to post and view comments on this article, or click here to get a print view with messages.

Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web06 | 2.8.190407.1 | Last Updated 7 Apr 2019

Article Copyright 2019 by Marijan Nikic
Everything else Copyright © CodeProject, 1999-2019