Performance for huge data sets

[William]

Hello,

I have an application that has many different tables that need to be presented to the user - some of these have over 200k rows of data.

We currently use a very bad and basic search feature, along with .skip(x), .take(x) to only get a certain amount of data per page.

What I do above works, but lacks more advanced filtering.

I'm currently evaluating and looking at grid, but, I am very confused - it supports paging and by increasing the sample app to return 200k items, I can see through network monitoring that it only pulls the single page at a time, however, I'm getting lost trying to evaluate this product...

... The docs don't seem to go in to that much details and the demos page seems to be better, but neither give any information about performance tuning, limiting the result set and similar - I'm worried that if I link this to the live data, I'm going to introduce a lot of problems.

Are there any other documentation bits that I am missing?

[Vincent]

In my application, a page return 90K rows. I transfer data from server via json. store the data in page kendodatasoure.  In my gird, show first 2k row (within 3 seconds). I use <scrollable endless="true"/> to load data from ds. 

[Georgi]

Hello Vincent,

Have in mind that the performance of the grid mainly depends on the configuration used and the machine of the client. Paging is essential for improving the performance of the grid. Best performance is achieved with server-side paging, which loads only the currently visible page of data and with virtual and endless scrolling, which load data on demand. Here are some demos, which you can see:

• https://demos.telerik.com/aspnet-core/grid/remote-data-binding
• https://demos.telerik.com/aspnet-core/grid/endless-scrolling-remote
• https://demos.telerik.com/aspnet-mvc/grid/virtualization-remote-data

Other things that matter for the performance:

• Number and content of columns: if there are many columns or if they contain templates with other Kendo UI widgets, the Grid page size needs to be set to as small a number as your design allows.
• Respectively, the page size. It shouldn't be too big, to avoid loading to much DOM content in the page, slowing down its performance. I would recommend a page size not bigger than 50, unless you are using virtualization, when 100 is also ok. Respectively, the page size. It shouldn't be too big, to avoid loading to much DOM content in the page, slowing down its performance. I would recommend a page size not bigger than 50, unless you are using virtualization, when 100 is also ok.Respectively, the page size. It shouldn't be too big, to avoid loading to much DOM content in the page, slowing down its performance. I would recommend a page size not bigger than 50, unless you are using virtualization, when 100 is also ok.
• Respectively, the page size. It shouldn't be too big, to avoid loading to much DOM content in the page, slowing down its performance. I would recommend a page size not bigger than 50, unless you are using virtualization, when 100 is also ok. Respectively, the page size. It shouldn't be too big, to avoid loading to much DOM content in the page, slowing down its performance. I would recommend a page size not bigger than 50, unless you are using virtualization, when 100 is also ok.Respectively, the page size. It shouldn't be too big, to avoid loading to much DOM content in the page, slowing down its performance. I would recommend a page size not bigger than 50, unless you are using virtualization, when 100 is also ok.

Regards,
Georgi
Progress Telerik

[William]

Hi Georgi,

What I can't understand at the moment is that I have a very complex data structure that requires several advanced queries before I call .ToList() and return it to my view and it gets rendered.

I have a very basic paging system that simply feeds the current page count, and I skip that amount of records - so, I know that Entity Framework does it's "Magic" in the background, and combined with my .skip() and similar commands, I limit the query a lot.

I guess my question is, I'm impressed so far with the performance locally (only dealing with ~1k rows) and I clearly see that you have done all the amazing work to make paging/ajax loading work efficiently... but, when I place this with live data and do much more complex queries on 200k data point, will the performance suffer?

I'm not really finding any documentation or information on the internals of your datasource feature - Does it automatically do the filtering / .Skip() etc. on what it is passed, or, is there anything more I need to do in order to ensure high performance?

Thanks

[Georgi]

Hello William,

Generally speaking, if the service takes more time to fetch the data, it is expected that the grid will take more time to load. However, this also depends on the clients configuration (internet speed, machine, etc.). If there is no significant difference in loading the data on the back end, you should not experience any major performance impact.

Furthermore, the ToDataSourceResult method, internally uses LINQ to sort, page, filter and group the data.

Finally, you can find further information regarding the configuration of an AJAX grid in the following article:

• https://docs.telerik.com/aspnet-mvc/helpers/grid/binding/ajax-binding

Although, the above article targets .Net MVC, the very same applies for .Net Core.


Regards,
Georgi
Progress Telerik