Filesystem notes from the real world–C#

These are just some quick tidbits about file processing in C#.  Reading in files, reading in LARGE files, Reading and Processing Large Files.

 

Reading a file (use a StreamReader)

if (File.Exists(path))
               {
                   using (StreamReader sr = new StreamReader(path))
                   {
                       while (sr.Peek() >= 0)
                       {
                           string line = sr.ReadLine();
                       }
                   }
               }


 

Fastest way to read a file (Reading a large file) – Use BufferedStream

Using (FileStream fs = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{  using (BufferedStream bs = new BufferedStream(fs))  {
    using (StreamReader sr = new StreamReader(bs))
    {
      string line;
       while ((line = sr.ReadLine()) != null)
       {

       }
     }   
  }
} 

For large files, split up the Processing of the file – and the Reading of the file

Use a producer – consumer pattern. The producer task read in lines of text using the BufferedStream and handed them off to a separate consumer task that did the searching.

Specializing in high volume web and cloud application architecture, Anuj Varma’s customer base includes Fortune 100 companies (dell.com, British Petroleum, Schlumberger).
Anuj’s training as a mathematical physicist followed by years of advanced computer programming is unique in the industry.

For Anuj’s popular technology seminars and science and scientific computing seminars, please visit ANUJ.COM

For Anuj’s Mathematical Models and Math Modeling related consulting , please visit anuj.com.

All content on this site is original and owned by AdverSite Web Holdings, Inc. – the parent company of anujvarma.com. No part of it may be reproduced without EXPLICIT consent from the owner of the content.

Anuj Varma – who has written posts on Anuj Varma, Technology Architect.


Leave a Reply

Your email address will not be published. Required fields are marked *