Articles » Languages » C# » Utilities

File System Helpers

Gene Stetz, 19 Feb 2019

This article describes a C# utility which provides for directory and file comparison.

Download source files - 7.7 MB

Introduction

This article describes a C# utility for helping with file system operations. Specifically, this article begins with the introduction of functionality providing the capability to perform comparisons of directories and their files contained within. This is a comparison of directory structure based upon attributes. Although this utility begins with this functionality, it is meant to be expanded to include other useful functions besides just directory comparisons. This article, though, focuses only on performing directory comparisons.

Background

Many of us such as myself rely upon a 3rd party cloud backup storage solution for our local data, documents, and code we create. The cloud backup software I use is reliable and cost effective, but the one aspect which is not provided is the notification of a change or deletion of a file or directory within an area marked for cloud backup.

For instance, say that I store all of my documents in my drive letter “D:\” on a particular computer, and I mark this drive for cloud backup. The cloud backup software backs up all files and directories on this drive. The cloud backup software notices if a file has changed locally and updates such change in the cloud, too. Likewise, if a file or directory is deleted locally, it deletes the same in the cloud, freeing up space in the cloud since I pay for the amount of data stored there and don’t want to be charged for something I no longer need.

Alright, so we recognize the benefit of using a cloud backup solution for our backup needs, but most cloud backup solutions don’t include a notification or alert when something changes in a way not anticipated. The idea of receiving a notification for unexpected changes became the motivation for writing my own utility for performing this.

Although I’ve mentioned that I use the FileSystemHelpers directory comparison functionality to alert myself of unintended changes, it does not perform the alert itself. Rather, I’ve componetized things so that this utility merely provides directory comparison functionality, and another program I’ve created as a scheduled task, actually uses this utility and handles the notification part. Hence, I’m following “separation of concerns” here. Also, realize that although I’ve presented FileSystemHelpers as an answer to a limitation of cloud backup software, this utility can certainly be applied toward other situations unrelated to cloud backup. I’ve only mentioned cloud backup as how I began with the idea for creating this utility.

Using the Code

Composition of the Visual Studio Solution

The Visual Studio solution of FileSystemHelpers is comprised of two projects written using the .NET Framework 4.6.1. The program is contained within a project named CoreDistributable. I also created an integration test project aptly named as such using NUnit which I referenced via NuGet. The tests within provide an example of the usage of the directory comparison functionality of the utility class, as well as, insure that the utility works as intended.

Why I Created Integration Tests Rather Than Unit Tests

Although I might have been able to create unit tests, I found that it would have been a lot more work to attempt to create elaborate mocks or fakes of directories and files with little or no added value over performing the same using actual files and directories using integration tests. For my integration tests, I merely took some directories on my computer and made a copy of them to another location. Depending on the type of test, I would either delete a file(s) or subdirectory from one directory, or make a simple modification of a file in a directory, such as appending a single letter to a “readme.txt” file. I found that doing this was much easier and faster than all of the set-up needed for creating mocks or fakes.

The Need to Populate the App.config File in IntegrationTestProject

Because the test project is an integration project and your files and directories are different than mine (for what you’ll test), you’ll need to populate the parameters in the test project’s App.config file. The coded tests reference this config. For instance, you’ll need to populate the below displayed parameters. The value with “You Fill In” will need to be populated with whatever you want to compare against, in this case, the absolute path of the destination directory which you want to compare your source directory against.

add key="MissingFiles_NoneMissing_DestinationDirectory" value="You Fill In"
add key="MissingFiles_OneMissing_DestinationDirectory" value="You Fill In"

Discover Whether a File Has Changed

I like to know when certain things change locally which I didn’t expect to change, and hence want to be notified of such a change before having it finalized in the cloud. For example, I backup to the cloud my finished music and video files. I don’t expect these files to be changed on my local computer. But, if an accident happens, I’d like to know of such an unexpected change before it gets permanently updated in the cloud. The below test shows an example of how this can be accomplished by a program that invokes the manager class in the utility to capture a result data transformation object containing any detected file size differences.

The particular test shown below is taken from the DirectoryComparisonTests class and demonstrates how the utility can be used to detect whether a file has changed.

// Prep the two directories so that all equivalent files between them have the same size
// except one file which has different sizes between the source and destination directory.
[Test]
public void TestDiffFileSizeComparison_OneDiff()
{
var destinationDirectory =
ConfigurationManager.AppSettings["FileSizes_OneDiff_DestinationDirectory"];

var comparisonResultDto =
FsoAttributesManager.FindFilesWithDifferentSizes(_sourceDirectory, destinationDirectory);

var expectedNodeDepth = int.Parse(ConfigurationManager.AppSettings["FileSizes_OneDiff_NodeDepth"]);
var expectedFileName = ConfigurationManager.AppSettings["FileSizes_OneDiff_FileName"];
var fnd = comparisonResultDto.ComparisonDestinationResult
.Where(x => (x.NodeDepth == expectedNodeDepth)
&& (x.ShortName.Equals(expectedFileName, StringComparison.CurrentCultureIgnoreCase)));

Assert.IsTrue(fnd.Count() == 1);
}

Note that the method FindFilesWithDifferentSizes does not actually open up equivalent files in two directories but rather uses the file size attribute of each file to make a basic determination of whether a file has changed between its source and destination directories.

Discover Whether a File Has Been Deleted

I like to know when documents in a particular directory get deleted. There are times in which I wish to no longer keep a document, but I’d like to be notified before the deletion is made permanent in the cloud.

The particular test shown below is taken from the DirectoryComparisonTests class and demonstrates how the utility can be used to detect whether a file has been deleted.

// Prep the two directories so that the destination directory is a copy of the source directory
// except that one file is missing from the destination.
[Test]
public void TestMissingFiles_OneMissing()
{
var destinationDirectory =
ConfigurationManager.AppSettings["MissingFiles_OneMissing_DestinationDirectory"];

var comparisonResultDto =
FsoAttributesManager.FindMissingFilesInDestination(_sourceDirectory, destinationDirectory);

var expectedNodeDepth =
int.Parse(ConfigurationManager.AppSettings["MissingFiles_OneMissing_NodeDepth"]);
var expectedFileName = ConfigurationManager.AppSettings["MissingFiles_OneMissing_FileName"];

Assert.IsTrue(fnd.Count() == 1);
}

Points of Interest

The directory and file comparison methods contained within the primary class FsoAttributesManager make frequent use of Linq to Object queries. The below method FindFilesWithDifferentSizes is an example of this.

public static DirectoryComparisonDto FindFilesWithDifferentSizes
(string directorySourceFullPath, string directoryDestinationFullPath)
{
var dirComparisonDto = new DirectoryComparisonDto
{
ComparisonType = ComparisonType.DifferentFileSizes,
ComparisonTestDateTime = DateTime.Now
};

var dirSrcContentsDto = CreateDirectoryContentsDto(directorySourceFullPath);
var dirDestContentsDto = CreateDirectoryContentsDto(directoryDestinationFullPath);

dirComparisonDto.DirectoryContentsSource = dirSrcContentsDto;
dirComparisonDto.DirectoryContentsDestination = dirDestContentsDto;

var allMatchingFiles = (
from filesInSrc in dirSrcContentsDto.ItemList.Where
(x => x.GetType() == typeof(FileAttributesDto))
join filesInDest in dirDestContentsDto.ItemList.Where
(x => x.GetType() == typeof(FileAttributesDto))
on new { filesInSrc.RelativePath, filesInSrc.ShortName }
equals new { filesInDest.RelativePath, filesInDest.ShortName }
select (FileAttributesDto)filesInDest).Distinct().ToList();

var filesWithSameSize = (
from filesInSrc in dirSrcContentsDto.ItemList.Where
(x => x.GetType() == typeof(FileAttributesDto))
join filesInDest in dirDestContentsDto.ItemList.Where
(x => x.GetType() == typeof(FileAttributesDto))
on new { filesInSrc.RelativePath, filesInSrc.ShortName }
equals new { filesInDest.RelativePath, filesInDest.ShortName }
where
((FileAttributesDto)filesInSrc).Size == ((FileAttributesDto)filesInDest).Size
select (FileAttributesDto)filesInDest).ToList();

// To find all matching files between the source and destination,
// but which have different sizes,
// subtract away the filesWithSameSize from allMatchingFiles.
// Conceptually this is the same as a T-SQL outer join with null
// on the side for which you want to find the missing members...
var filesWithDiffSizes = (
from allM in allMatchingFiles
join sameM in filesWithSameSize
on new { allM.RelativePath, allM.ShortName }
equals new { sameM.RelativePath, sameM.ShortName } into outerGrp
from sameM in outerGrp.DefaultIfEmpty()
where sameM == null
select (FileAttributesDto)allM).ToList();

filesWithDiffSizes.ForEach(x =>
{
dirComparisonDto.ComparisonDestinationResult.Add(x);
});

return dirComparisonDto;
}

Notice from the above presented method that I’ve used an anonymous type when using Linq to join collections in memory on more than one attribute (column if this were SQL). The below code taken from the above is an example of this:

on new { filesInSrc.RelativePath, filesInSrc.ShortName }
equals new { filesInDest.RelativePath, filesInDest.ShortName }

Also, notice the Linq syntax that is the equivalent of a T-SQL outer join excluding the missing members. The below code is an example of this:

from allM in allMatchingFiles
join sameM in filesWithSameSize
on new { allM.RelativePath, allM.ShortName }
equals new { sameM.RelativePath, sameM.ShortName } into outerGrp
from sameM in outerGrp.DefaultIfEmpty()
where sameM == null

For both updates and deletions, I use the directory comparison capability in FileSystemHelpers towards a notification of such changes, as a way of basically saying, “Are you sure you want to keep this change, be it a modification or a deletion?”

Please be aware that although I’ve previously presented the two use cases of identifying accidental updates and deletions, you may have a situation in which you want to be notified of accidental additions. The directory comparison functionality of FileSystemHelpers can be used for that, too.

History

18th February, 2019: Initial version

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Gene Stetz

Software Developer

Follow
this Member
Unfollow

United States

I am a web, software, and database developer having primarily a .Net and SQL Server concentration. I am also interested in leveraging other technologies where they are best utilized.

Comments and Discussions

0 messages have been posted for this article Visit https://www.codeproject.com/Articles/1277749/File-System-Helpers to post and view comments on this article, or click here to get a print view with messages.

Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web04 | 2.8.190214.1 | Last Updated 19 Feb 2019

Article Copyright 2019 by Gene Stetz
Everything else Copyright © CodeProject, 1999-2019