RenderReport very slow for large reports

[Bob]

We recently started receiving complaints from clients who have been creating increasingly large reports, in the area of 50,000 to 75,000 records.  In order to rule out our data, I created a test report using the same code we have to create our actual reports, but only create data from a POCO class.  When running this test report, the output from RenderReport is extremely fast for record sets under a few thousand rows, but becomes very slow for record sets 50,000 to 75,000 rows.  With our data, a 75,000 record report will take over an hour.  Here are the times observed processing CSV and PDF report types with the test data and code below:

1000:  *** Render Report #0 DONE in 00:00:00.1581774 ***

5000:  *** Render Report #0 DONE in 00:00:02.6023945 ***

10000:  *** Render Report #0 DONE in 00:00:11.0991098 ***

50000:  *** Render Report #0 DONE in 00:10:20.2986525 ***

75000:  *** Render Report #0 DONE in 00:21:37.1953330 ***

Here is the code I am using to run the tests.  Please advise if there is a better alternative, but please keep in mind that this code currently ties into our user interface to produce a variety of different report types.  Thanks for any insight.

    private Table _MainTable;
    private Unit PageWidth = new Unit(11, UnitType.Inch);
    private Unit PageHeight = new Unit(8.5, UnitType.Inch);
    private Unit ItemHeight = new Unit(.25, UnitType.Inch);
    private Unit TenPointFont = new Unit(10, UnitType.Point);
    private Unit TableWidth;

    public class TestReportObject
    {
        public int? RowID { get; set; }
        public string Description { get; set; }
        public string LongDescription { get; set; }
    }

        private void AddTestColumn(int colIndex, string header, string fieldName, Unit colWidth)
        {
            TableGroup tableGroupColumn = new TableGroup();
            _MainTable.ColumnGroups.Add(tableGroupColumn);
            _MainTable.Body.Columns.Add(new Telerik.Reporting.TableBodyColumn(colWidth));

            Telerik.Reporting.TextBox columnHeaderTextBox;
            Telerik.Reporting.TextBox cellTextBox;

            columnHeaderTextBox = new Telerik.Reporting.TextBox();
            columnHeaderTextBox.Style.BackgroundColor = Color.LemonChiffon;
            columnHeaderTextBox.Style.BorderStyle.Default = BorderType.Solid;

            string name = header;
            columnHeaderTextBox.Value = name;
            columnHeaderTextBox.Name = name;

            columnHeaderTextBox.Size = new SizeU(colWidth, ItemHeight);
            columnHeaderTextBox.Style.Font.Size = TenPointFont;
            columnHeaderTextBox.Style.TextAlign = HorizontalAlign.Center;
            tableGroupColumn.ReportItem = columnHeaderTextBox;
            columnHeaderTextBox.TextWrap = true;
            columnHeaderTextBox.Height = ItemHeight;

            _MainTable.Items.Add((ReportItemBase)columnHeaderTextBox);

            cellTextBox = new Telerik.Reporting.TextBox();
            cellTextBox.Style.BorderColor.Default = Color.Black;
            cellTextBox.Style.BorderStyle.Default = BorderType.Solid;

            cellTextBox.Value = $"=Fields.{fieldName}";
            cellTextBox.Name = name;

            cellTextBox.Size = new SizeU(colWidth, ItemHeight);
            cellTextBox.Style.TextAlign = HorizontalAlign.Left;
            cellTextBox.Style.Font.Size = TenPointFont;
            cellTextBox.Style.VerticalAlign = VerticalAlign.Middle;
            cellTextBox.Height = ItemHeight;
            cellTextBox.TextWrap = true;
            _MainTable.Body.SetCellContent(0, colIndex, cellTextBox);

        }

        private Telerik.Reporting.Report BuildTestReport()
        {
            Telerik.Reporting.Report TestReport = new Report();

            #region page settings (making the assumption of landscaped 8.5 x 11 paper)

            TestReport.PageSettings.Landscape = true;
            TestReport.PageSettings.PaperSize = new SizeU(PageWidth, PageHeight);

            double BottomMarginSize = ExportParameters.BottomMargin == null ? .5 : ExportParameters.BottomMargin.Value;
            TestReport.PageSettings.Margins.Bottom = new Unit(BottomMarginSize, UnitType.Inch);

            double LeftMarginSize = ExportParameters.LeftMargin == null ? 1 : ExportParameters.LeftMargin.Value;
            TestReport.PageSettings.Margins.Left = new Unit(LeftMarginSize, UnitType.Inch);

            double RightMarginSize = ExportParameters.RightMargin == null ? 1 : ExportParameters.RightMargin.Value;
            TestReport.PageSettings.Margins.Right = new Unit(RightMarginSize, UnitType.Inch);

            double TopMarginSize = ExportParameters.TopMargin == null ? .5 : ExportParameters.TopMargin.Value;
            TestReport.PageSettings.Margins.Top = new Unit(TopMarginSize, UnitType.Inch);

            TableWidth = PageWidth - (new Unit(LeftMarginSize + RightMarginSize, UnitType.Inch));

            #endregion

            Unit remainingPageWidth = TableWidth;
            Unit fullTableWidth = new Unit(11, UnitType.Inch);

            _MainTable = new Table();
            _MainTable.Location = new PointU(new Unit(0, UnitType.Inch), new Unit(0, UnitType.Inch));
            _MainTable.Size = new SizeU(fullTableWidth, ItemHeight);

            // Body
            DetailSection ExportDetail = new DetailSection();


            int RowCountTotal = 50000;
            var rowList = new List<TestReportObject>();
            for (int rows = 0; rows < RowCountTotal; rows++)
            {
                var reportObj = new TestReportObject();
                reportObj.RowID = rows;
                reportObj.Description = $"Row {rows}";
                reportObj.LongDescription = $"This is the long description for row {rows}";
                rowList.Add(reportObj);
            }

            _MainTable.DataSource = rowList;
            _MainTable.Style.VerticalAlign = VerticalAlign.Middle;
            _MainTable.ColumnHeadersPrintOnEveryPage = true;
            _MainTable.Name = "mainTable";

            // default grouping has to be added to allow data to appear
            Telerik.Reporting.TableGroup tableGroup = new Telerik.Reporting.TableGroup();
            tableGroup.Groupings.AddRange(new Telerik.Reporting.Grouping[] { new Telerik.Reporting.Grouping("") });
            tableGroup.Name = "DetailGroup";
            _MainTable.RowGroups.Add(tableGroup);

            #region Define columns and cells within the main table

            Unit theColumnWidth = _MainTable.Width / 3;  // number of columns
            AddTestColumn(0, "Row ID", "RowID", theColumnWidth);
            AddTestColumn(1, "Description", "Description", theColumnWidth);
            AddTestColumn(2, "Long Description", "LongDescription", theColumnWidth);


            #endregion

            ExportDetail.Items.AddRange(new ReportItemBase[] { _MainTable });
            TestReport.Items.Add((ReportItemBase)ExportDetail);

            return TestReport;
        }


        /// <summary>
        /// Generates the bytes of the export file
        /// </summary>
        private byte[] ExportBytes()
        {
            Report rpt = BuildTestReport();

            InstanceReportSource rptSource = new InstanceReportSource();
            rptSource.ReportDocument = rpt;

            using (new ContextScope(new Hashtable()))
            {
                Hashtable deviceInfo = new Hashtable();
                deviceInfo["FontEmbedding"] = "Subset";

                Telerik.Reporting.Processing.ReportProcessor RP = new Telerik.Reporting.Processing.ReportProcessor();

               // ExportFileTypeString is UI selection, pdf, csv, etc.
                byte[] buffer = RP.RenderReport(ExportFileTypeString, rptSource, deviceInfo).DocumentBytes;

                return buffer;
            }
        }

[Todor]

Hello Bob,

I tested the provided code in a Console Application in CSharp, exporting to PDF. Here are the timings I got:

10000/690p		- *** Render Report #0 DONE in 00:00:07.3474767 ***
50000/3450p		- *** Render Report #0 DONE in 00:02:26.0321617 ***
75000/5275p		- *** Render Report #0 DONE in 00:05:08.2914986 ***
100000/6898p		- *** Render Report #0 DONE in 00:08:57.1850148 ***

I also exported the report to TRDP to check it visually. I noticed that the table is wider than the page, which introduced horizontal page breaking when generating the PDF document. I suspect this was due to the hard-coded values I used for page margins (1in) and for the table width (11in). Horizontal pagination is one of the factors that may slow down the rendering in physical formats like PDF. For that reason, I reduced the table width and page margins to fit in the 11in of the Letter Landscape page width.

This substantially reduced the above times. Note that they are less than a minute, even with 100 000 records:

50000/1725p		- *** Render Report #0 DONE in 00:00:18.9590838 ***
75000/2587p		- *** Render Report #0 DONE in 00:00:29.7250274 ***
100000/3449p		- *** Render Report #0 DONE in 00:00:48.7737648 ***

My recommendation is to fit the report within the available page width in order to avoid the horizontal page break.

Regards,
Todor
Progress Telerik

[Bob]

Todor, thank you for this excellent information.  I removed the long description column and adjusted widths to fit and ran a PDF with 100,000 rows, and I was also able to run this in 49 seconds.

However, I then tried the exact same report as a CSV, and this took 5 minutes and 39 seconds.  Since our clients tend to use this report more for CSV output, is there any way I can speed up CSV?

Also, please know that the primary use of this report is to export the data of a RadGridView with dynamic columns, so we cannot guarantee that the columns are going to fit.  We try to adjust the column widths as best we can, but the user can select as many columns as they want from a large selection. It would be helpful to know how we could provide the best output without causing the slowness when the width is overrun.

[Todor]

When it comes to the CSV rendering, note that the Reporting engine flattens the tables - see Csv Rendering Design Considerations. I suspect that is the reason for the slow rendering. You may try to remove the duplicated static information by setting NoHeader and/or NoStaticText of the CSV Device Information Settings to True (default values are False). The CSV rendering is a pure text export that doesn't use pagination and the page settings do not affect its performance.

The performance issue when using export formats with physical paging like PDF comes from the horizontal paging. If the report layout is changed dynamically, you need to take care to avoid horizontal page breaks. For example, you may display a maximum number of columns per page and move the rest to another table that is below the first one. The exact approach would depend on the expected number of columns and the design. You may also set the Page Width in the PageSettings to a high value that would be able to accommodate the entire table.