Need help understanding virtualization/paging

[Daniel]

I've looked through the demos and docs but am still scratching my head.

My goal is to present the user with a filterable drop down list which fetches "paged" data from the server as the user scrolls down.

The scenario is that I don't want to retrieve 1000's of rows from the database at once, instead I want to fetch a page (page size = 100) of data at a time, and stream that data as needed as the user scrolls down.  But the example I see here (http://demos.telerik.com/aspnet-mvc/dropdownlist/virtualization) doesn't appear to do this, nor can I find anything to explain how this would be done.

Indeed, that demo (http://demos.telerik.com/aspnet-mvc/dropdownlist/virtualization) uses a Controller that appears to fetch ALL rows from the database multiple times, is that correct? I'm lost when it comes to the value mapper function as it accepts a parameter of int[] values, then AGAIN fetches ALL rows from the database, loops through ALL those rows, attempting to find and generate an index for each passed-in value...

Isn't the point of implementing virtualization and paging to make it so that we're limiting the amount of data returned from the database?  Or is the actual/only benefit that we're not delivering a huge payload to the browser (ignoring any performance issue with the database)?

[Georgi Krustev]

Hello Daniel,

The virtualization functionality is designed to fetch data chunks with calculated page size based on the height of the popup element. I would suggest you check this help topic, which explains how the virtualization works (it is applicable to ASP.NET MVC too).

The mentioned demo, actually shows that functionality. If you examine the requests, (using browser's dev tool) you will see how only a particular page is requested. The server paging is "hidden" in the ToDataSourceResult static method. Let's focus on this Action method:

public ActionResult Virtualization_Read([DataSourceRequest] DataSourceRequest request)
{
    return Json(GetOrders().ToDataSourceResult(request));
}

The send request is converted to DataSourceRequest, which holds the information about the server paging, filtering, sorting and etc. Then this information is passed to the ToDataSourceResult method, which builds the correct LINQ expression and retrieves the proper data chunk.

The idea of valueMapper is to retrieve the row index that matches particular value.  I would suggest examine the valueMapper documentation. The Orders_ValueMapper Action method, indeed, uses all Orders data items to get the specific index, but this implementation is done for simplicity. If you would like to optimize this process, then you can use specific SQL query that will return the row index based on the passed value.

Let me know if something is not clear.

Regards,
Georgi Krustev
Telerik

 

 

[Daniel]

Hi Georgi, 

Thanks for your response.  I did look at that help topic previously (and looked at it again now) and was about to respond that I'm still confused, but I think I figured out where my confusion is coming from.

I believe I am incorrectly assuming that virtualization is necessary to implement fetching data on a page by page basis.  When really, it's sort of a separate optimization option to limit the size of the html when dealing with very large data sets.

In other words, I believe I can implement a paged data-source without using virtualization....is my understanding correct?

Thanks,
Dan

[Georgi Krustev]

Hello Dan,

The DropDownList supports binding on a page by page basis only when virtualization functionality is enabled. When it is disabled (Virtual settings are not defined), then widget will display the whole data without braking it into pages.

With regards to this question - "In other words, I believe I can implement a paged data-source without using virtualization....is my understanding correct?"

The DataSource, as a standalone component, supports paging (client or server). If you, however, would like to incorporate a datasource with enabled paging into DropDownList widget then virtualization is the only way to go.

Regards,
Georgi Krustev
Telerik

 

 

[Daniel]

Hi Georgi,

Thanks again for the response.

Unfortunately, ​I'm still struggling how I can utilize the DropDownList's virtualization in a beneficial way for my situation, which is that I need to bind the drop down list to the results of a database query that can return 10,000+ rows.  

My previous experience with paging data usually involved sending the desired page number to the server along with the page size, and having the database query return only those rows, along with a count of the total number of rows which would have been returned had paging not been used.  This way the client has enough information to display the appropriate page number links, yet only the current page's rows are processed on the client.  

But with the DropDownList's virtualization implementation, if I'm understanding things correctly, the above is not enough, and in addition to returning the current page's data, I also need to supply a means of finding the index for a given value, which means that I need to re-query those 10,000+ rows just to find the row index for the passed-in value, and this would have to be done numerous times. 

I guess now I'm interested in learning how the DropDownList determines when it needs to call the ValueMapper function?  How often does it call it, and under what scenarios?  It's seems strange to me because when calling the ValueMapper, the DropDownList knows the value already, which means the value was returned via a previous call to DataSource.Read....so why would it need to find the row-index for a value that was retrieved previously when it presumably would have already known the index at that time?

Thanks,
Dan

[Georgi Krustev]

Hi Dan,

Your understanding about the data virtualization process is correct. The consumer requests only a particular data chunk, in this case page, instead of requesting the whole data at once.

The DropDownList virtualization uses exactly the same principle. The virtual list request the first page on load, when user starts to scroll, we pre-fetch next pages that should be displayed. This is done exactly with server requests that contains the page number and its size. You can easily see this in the requests that the widget makes.

The ValueMapper was introduced, because unlike simple Data + UI virtualization, DropDownList needs to maintain the selected item and also to display the selected data item based only on value. In order to display the selected text widget needs to retrieve the selected data item, which is part of a particular data page that is unknown to us. The required information is gathered exactly with the valueMapper, that passes value and requests the corresponding row/dataitem index of that value. From this index, we can calculate the page number and thus pre-fetch only that page with additional Ajax request.

To answer you specific question, the valueMapper will be called when we need to select a dataitem that is not present in the data source. To depict more clearly, let's image the following case:

- widget has one page of 50 items
- the selected value is "1250"
- widget will call the valueMapper, passing this "1250" value asking the developer to return the index of that value (or data item).
- the valueMapper returns index 1250 (I am using same index and value to be easier for understanding)
- this index corresponds to 25th page (1250 / 50 = 25).
- Once the page is calculated the widget will retrieve that page and will select the correct data item.

In this particular case, the valueMapper will be called only once, because widget needs to retrieve the data page of the selected data item. In short, the valueMapper is called every time when the widget needs to find the data page that corresponds to a selected value.

I hope that this makes things clearer now.

Regards,

Georgi Krustev
Telerik

 

 

[Daniel]

Hi Georgi,

Ah, ok now I see.  Yes, much clearer now.  Thanks for your help!

Thanks,
Dan

[Kevin Smith]

 

Hello Georgi Krustev,

I don't want to show the page of selected value, Is there virtualisation of scroll can work without valueMapping function  ?

 

 

[Georgi Krustev]

Hello Kevin,

The valueMapper callback is required, because the widget needs at least the selected data item to ensure its proper work. What I would suggest is to use the `mapValueTo: 'dataItem'` configuration to avoid calculating the value index:

http://docs.telerik.com/kendo-ui/controls/editors/combobox/virtualization#recent-updates

Regards,
Georgi Krustev
Telerik by Progress