large file read c# Archives - Anuj Varma, Hands-On Technology Architect, Clean Air Activist https://www.anujvarma.com/tag/large-file-read-c/ Production Grade Technical Solutions | Data Encryption and Public Cloud Expert Thu, 09 Apr 2020 16:29:24 +0000 en-US hourly 1 https://wordpress.org/?v=7.0.2 https://www.anujvarma.com/wp-content/uploads/anujtech.png large file read c# Archives - Anuj Varma, Hands-On Technology Architect, Clean Air Activist https://www.anujvarma.com/tag/large-file-read-c/ 32 32 Filesystem notes from the real world–C# https://www.anujvarma.com/filesystem-notes-from-the-real-worldc/ https://www.anujvarma.com/filesystem-notes-from-the-real-worldc/#respond Tue, 10 Jun 2014 01:53:23 +0000 http://www.anujvarma.com/?p=2573 These are just some quick tidbits about file processing in C#.  Reading in files, reading in LARGE files, Reading and Processing Large Files. Reading a file (use a StreamReader) if […]

The post Filesystem notes from the real world–C# appeared first on Anuj Varma, Hands-On Technology Architect, Clean Air Activist.

]]>
These are just some quick tidbits about file processing in C#.  Reading in files, reading in LARGE files, Reading and Processing Large Files.

Reading a file (use a StreamReader)

if (File.Exists(path))
               {
                   using (StreamReader sr = new StreamReader(path))
                   {
                       while (sr.Peek() >= 0)
                       {
                           string line = sr.ReadLine();
                       }
                   }
               }


 

Fastest way to read a file (Reading a large file) – Use BufferedStream

Using (FileStream fs = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{  using (BufferedStream bs = new BufferedStream(fs))  {
    using (StreamReader sr = new StreamReader(bs))
    {
      string line;
       while ((line = sr.ReadLine()) != null)
       {

       }
     }
   }
}

For large files, split up the Processing of the file – and the Reading of the file

Use a producer – consumer pattern. The producer task read in lines of text using the BufferedStream and handed them off to a separate consumer task that did the searching.

The post Filesystem notes from the real world–C# appeared first on Anuj Varma, Hands-On Technology Architect, Clean Air Activist.

]]>
https://www.anujvarma.com/filesystem-notes-from-the-real-worldc/feed/ 0