C# – file enumerate issue

cfile-ionetvisual-studio-2008

I am using VSTS 2008 + C# + .Net 3.5 to develop a console application. I need to enumerate the most recent 50 files in current folder (to read file content, to get file meta data like file name, creation time, etc.). Since the current folder has about 5,000 files, and if I use Directory.GetFiles API, all 5,000 files' meta data information will be read into memory. I think it is a waste since I only need to access the most recent 50 files.

Any solutions to access only the 50 most recent files in current directory?

Best Answer

This solution still loads metadata about all files, but I would say it's fast enough for most uses. The following code reports that it takes around 50ms to enumerate the 50 most recently updated files in my Windows\System32 directory (~2500 files). Unless the code is run very frequently I would probably not spend time optimizing it a lot more:

FileInfo[] files = (new DirectoryInfo(@"C:\WINDOWS\System32")).GetFiles();
Stopwatch sw = new Stopwatch();
sw.Start();
IEnumerable<FileInfo> recentFiles = files.OrderByDescending(
                                              fi => fi.LastWriteTime).Take(50);
List<FileInfo> list = recentFiles.ToList();
sw.Stop();
Console.WriteLine(sw.ElapsedMilliseconds);
list.ForEach(fi => Console.WriteLine(fi.Name));

Update

Based on the discussion in the comments regarding using date/time in the file name: note that Directory.GetFiles does not load metadata about files; it simply returns a string array with file names (DirectoryInfo.GetFiles on the other hand returns an array of FileInfo objects). So, if you have date and time in your file names (preferably in a format that lends itself to sorting, such as yyyyMMdd-HHmmss or something like that) you can use Directory.GetFiles to get the file names, sort descending and then pick the 50 first from the list:

string[] files = Directory.GetFiles(pathToLogFiles);
IEnumerable<string> recentFiles = files.OrderByDescending(s => s).Take(50);
List<string> recentFiles = recentFiles.ToList();

Then loop over the list and load whatever data you need from each file.

Related Solutions

C# – How to enumerate an enum

foreach (Suit suit in (Suit[]) Enum.GetValues(typeof(Suit)))
{
}

Note: The cast to (Suit[]) is not strictly necessary, but it does make the code 0.5 ns faster.

C# – How to create an Excel (.XLS and .XLSX) file in C# without installing Microsoft Office

You can use a library called ExcelLibrary. It's a free, open source library posted on Google Code:

ExcelLibrary

This looks to be a port of the PHP ExcelWriter that you mentioned above. It will not write to the new .xlsx format yet, but they are working on adding that functionality in.

It's very simple, small and easy to use. Plus it has a DataSetHelper that lets you use DataSets and DataTables to easily work with Excel data.

ExcelLibrary seems to still only work for the older Excel format (.xls files), but may be adding support in the future for newer 2007/2010 formats.

You can also use EPPlus, which works only for Excel 2007/2010 format files (.xlsx files). There's also NPOI which works with both.

There are a few known bugs with each library as noted in the comments. In all, EPPlus seems to be the best choice as time goes on. It seems to be more actively updated and documented as well.

Also, as noted by @АртёмЦарионов below, EPPlus has support for Pivot Tables and ExcelLibrary may have some support (Pivot table issue in ExcelLibrary)

Here are a couple links for quick reference:
ExcelLibrary - GNU Lesser GPL
EPPlus - GNU (LGPL) - No longer maintained
EPPlus 5 - Polyform Noncommercial - Starting May 2020
NPOI - Apache License

Here some example code for ExcelLibrary:

Here is an example taking data from a database and creating a workbook from it. Note that the ExcelLibrary code is the single line at the bottom:

//Create the data set and table
DataSet ds = new DataSet("New_DataSet");
DataTable dt = new DataTable("New_DataTable");

//Set the locale for each
ds.Locale = System.Threading.Thread.CurrentThread.CurrentCulture;
dt.Locale = System.Threading.Thread.CurrentThread.CurrentCulture;

//Open a DB connection (in this example with OleDB)
OleDbConnection con = new OleDbConnection(dbConnectionString);
con.Open();

//Create a query and fill the data table with the data from the DB
string sql = "SELECT Whatever FROM MyDBTable;";
OleDbCommand cmd = new OleDbCommand(sql, con);
OleDbDataAdapter adptr = new OleDbDataAdapter();

adptr.SelectCommand = cmd;
adptr.Fill(dt);
con.Close();

//Add the table to the data set
ds.Tables.Add(dt);

//Here's the easy part. Create the Excel worksheet from the data set
ExcelLibrary.DataSetHelper.CreateWorkbook("MyExcelFile.xls", ds);

Creating the Excel file is as easy as that. You can also manually create Excel files, but the above functionality is what really impressed me.

Best Answer

Related Solutions

C# – How to enumerate an enum

C# – How to create an Excel (.XLS and .XLSX) file in C# without installing Microsoft Office

Related Topic