Large data sets

[Edward Pauley]

I'm at my whits end..   i come from the Crystal/SAP world.  what ever we do we can't get a large report (150,000 records, 3000 pages) to work.  I have to tell my clients, i will build them a desktop Crystal solution.  why can't telerik reporting generate reports this large..  every other reporting product does.   

is there any way i can keep telerik reporting and have it work with large datasets or do i have to use Crystal?  please help.   my clients are screaming..   

[Milen | Product Manager @DX]

Hi Edward,

Please, find our response in the support ticket you have created. Let's continue tackling the issues you are experiencing in the support thread.

If you don't mind, I would like to use this thread to start a conversation on your (and our clients') exact needs and use cases. To anyone who reads this thread, please share your comments and needs. We would like to understand your requirements in order to provide you with a better solution in the long run.

Can you let us know why would you need such a large report? What are the requirements?

Our reporting is a BI solution with a main purpose to represent data in a human-readable manner. By this definition, a 3000 page document is more of an archived data document than an analytical one, especially if there is no requirement of the document to be rendered on a client. We also suspect that no PDF viewer will handle/open/print 3000 pages gracefully.

A few extra suggestions we can provide at the moment are:

• Try exporting to Excel 2007 rather than PDF where paging is ignored (it is not a page-oriented format) and let us know of the results
• If you are running a monthly report/data dump, try exporting the report in filtered batches, for example three 10-day reports.

Once we gather some understanding on your needs, we will try to provide you with other solutions and/or think about revising our report engine and may be improve the memory consumption with large data sets.

Let us know what you think.

All the best,
Milen
the Telerik team

[John S.]

Hi Milen,

To provide some information for your question....

My clients frequently need to create reports that are frequently a couple hundred pages long and have 2-4 levels of groupings. I am having issues in these case that are causing either 2-3 minute reports or in many cases things just fail (this is on a reasonably robust server with adequate resources). This is a serious problem in which an answer of 'your not filtering data enough' is not seen as the 'correct' answer.
 
I understand an ideal situation woud be that report user are able to filter down data to smaller datasets and produce only the part they need. In reality reports are printed and flipped through or the user 'wants' a pdf with alot of information to send someone. Data exports are not always seen as acceptable because users frequently don't want to supply easily manipulatable datasets or insist on a defined format (with page numbers, headers on each page, etc). They want reports that look nice and are consistant. If what was necessary is to export data only then I don't know if I would be using a reporting solution to do so.

Any help with this is appreciated.


Thanks,
John S.

[Peter]

Hello John,

Thank you for your input.

Currently our recommendation is to follow the Performance Considerations help article advices and avoid HtmlTextBox item when possible.

Still we do understand how important is the report rendering performance and it's in our TO DO list to improve it.

Regards,
Peter
Telerik

[Holly]

Hello,
We have sort of a similar situation - in our case we have small to very large reports that need to be run for pharmaceutical customers.

Our customers want to be able to run and save to PDF, or in some cases actually print out, potentially very, very large reports in order to document their manufacturing conditions, to meet FDA regulations.

We can easily have to generate a report that writes out 1,000,000+ rows and generates 18,000+ pages. And - we need to do it fast. Compressing or 'summarizing' the data is not a possibilility because some reports need to report the actual data point(s).

In our SCADA-based system we used Crystal Reports to do this - but often times run 40+ minutes trying to gererate these reports. We'd like to be able to write and run the reports using Telerik - but they will need to be written faster than Crystal Reports for us to justify switching.

Do you have any comparison figures between crystal reports and Telerik?  We have played wih Q1 2013 and Q2 2013 -- and they are running slower than Crystal... any additional tips or hints? we use a data source object (with SQL in it) to query the data for the report, and then a reportviewer on our sample C#/Winforms app to display the report.

Compressing or summarizing the data is not an option - some reports must print out all the 'raw' or actual data.

A page pre-view or the ability to display just the first 1 or 5 or xx pages would be awesome.
thank you!
Holly

[Peter]

Hi Holly,

Thank you for elaborating on your scenario. Currently we can only provide you with the recommendations in the Performance Considerations help article about improving your reports rendering. Still we are constantly looking for a ways to improve the Telerik Reporting performance and even now we are testing some new approaches that may lead to major improvement in subsequent Telerik Reporting release. Thus we encourage you to follow our release notes where we will duly note any performance improvements.

Regards,
Peter
Telerik

[Nicole]

Performance hints or not: Sure, if I were writing reports for my own consideration I can keep data sizes down.  However, our clients are government clients and their requirements motivate them to request large data sets in each reports in order to enable their sworn functions, fulfill legal requirements, and to support auditing.  SQL Server .RDLC client-side reporting, using the same SQL SELECT statements, showing the same columns, and all that, completes in a flash, about two seconds.  The Telerik report requires at best, using a .TRDX report definition, two and a half minutes.  If we build our reports using the Visual Studio designer, it may not even complete without throwing an "out of memory" exception and may require at least 12 minutes to complete (if it completes at all).

The data size requirements are not in our control.  Rather, what controls the data set sizes is ultimately the U.S. Constitution in the case of our particular customers.

[Nasko]

Hello Nicole,

In addition to following the Performance Considerations help article, we can advise you to prepare a runnable sample report with test data exhibiting the 12 minutes delay to render the report and send it to us in a support ticket. We will review the report configuration and try to provide further suggestions for optimization.

Regards,
Nasko
Telerik

[robert]

Hi,

Is this issue resolved now?
Our customer's also facing the same problem with telerik reports Q3 2015 (9.2.15.1216).

Query able to get data easily withing some seconds, but report never load and causing browser things hanged-up.

[Yana]

Hello Robert,

As in the previous case, we'd ask you to send us a sample project demonstrating your setup, so we to be able to research the exact scenario and provide further suggestions for optimization.

Regards,
Yana
Telerik by Progress

[Dilip]

Hello,

I am also facing the similar issue.  I am using 16.2.22.914 for Telerik Reporting and trying to generate 1000000 records in the report.

According to my format, I am showing only 2 records on a single page which shows user details. For getting the data from Database, I am using MS SQL Server. My stored procedure hardly takes 20 seconds to generate all 1000000 records. But when It's time to display it on the report, it takes forever and ultimately session times out and I am logged out of my system.

If there's any way I can generate this report efficiently, I'd love to know.

[Todor]

Hello Dilip,

We did performance improvements in our latest releases, and I recommend upgrading. More are coming, and we will hopefully have faster processing and rendering with less memory consumption in 2025.

Meanwhile, you may check the forum post Telerik reporting RAM usage for more details on the performance and how you may optimize it.

If you render the large reports in a viewer, you may switch to PrintPreview mode which applies the PageSettings and loads the generated physical pages one by one. In the Interactive view, the Reporting Engine applies soft pagination. This means that the tables are rendered on a single page, which requires the entire table to be rendered before displaying it. This may worsen the user experience.