Importing a huge CSV file

2 posts, 0 answers
  1. Patrick
    Patrick avatar
    372 posts
    Member since:
    Aug 2012

    Posted 16 Feb 2015 Link to this post

    Hello,
    I have a huge CSV file (about 3.3M lines), that I had to import for filtering and processing part of it.
    I've tried to looked at your CsvFormatProvider class to do the work. Unfortunately, the only possibility to work with it is to create a Workbook and the workbook is limited to about 1M lines.

    Is there any possibility to have a "low-level" importing of the CSV file?
    For example to import one line of data per call. The import returning an array of String.
    Or do you have another solution for this problem?

    Patrick
  2. Nikolay Demirev
    Admin
    Nikolay Demirev avatar
    103 posts

    Posted 17 Feb 2015 Link to this post

    Hello Patrick,

    By default the RadSpreadProcessing creates workbooks that have 1048576 rows and 16384 columns and there is no way it can import the entire CSV file. If you split the file in smaller CSV files the RadSpreadProcessing will be able to import them but it might not be able to handle that large amount of data fast enough.

    If you do not need any specific functionality that RadSpreadPorcessing provides I suggest you parse the CSV file with you own CSV parser implementation. It would be faster. Here you can find more information on Parsing CSV files in c#.

    Regards,
    Nikolay Demirev
    Telerik
     

    Check out the Telerik Platform - the only platform that combines a rich set of UI tools with powerful cloud services to develop web, hybrid and native mobile apps.

     
Back to Top