Grid with remote virtual scrolling fires mutiple read requests, when scrolling fast.

[KorsG]

Hi,

When using the Grid with remote datasource and virtual scrolling, the grid often fires 2 or more of the same read request, when scrolling by either mousescroll or dragging the scrollbar. 

I reproduced the error with your demo at: http://demos.kendoui.com/web/grid/virtualization-remote-data.html 
The problems exists in the newest Firefox, Chrome and IE9 (on windows). 
Attached is screenshots of FF and IE9, where it illustrates multiple server requests for the same "page".

The easiest way to reproduce the error is scroll as fast as possible.

Let me know if you need more examples.

[Alexander Valchev]

Hi Morten,

Thank you for bringing this to our attention.
Generally speaking the DataSource has a method that prevents such behaviour, however when the user is scrolling too fast it is possible multiple requests to be send.

I will forward this case to the team for further investigation, but currently I cannot promise if and when it will be fixed. Please accept my apology for the inconvenience caused.

Regards,
Alexander Valchev
the Telerik team

[srikanth]

I solved it using  async:false in transport:read.

[Marco]

Same problem here, using async:false does not duplicate request anymore but there is another side effect, the loading panel animation is freezed.

[Alexander Valchev]

Hello Marco,

This is expected. Synchronous requests to the server will prevent the normal flow of JavaScript execution and as a result the UI of the page will become not responding. This is because synchronous requests have "blocking" character.
The standard asynchronous requests are "non-blocking" and does not give such side effects.

Regards,
Alexander Valchev
Telerik

[Simon]

Hello,

2 years later, the problem still exist.  The user doesn't have to scroll too fast... the problem seems to occur at random even when you scroll very slowly.

Is there any update about this issue?

[Alexander Valchev]

Hi Simon,

Could you please specify which browser and OS you are testing with? Does the issue occurs on the demo page?
Can you please make a screen cast so I can forward the issue to the corresponding staff members? Thank you in advance for the cooperation.

Regards,
Alexander Valchev
Telerik

 

 

[Simon]

Hello Alexander,

I'm using Chrome and I did create a screen cast from your own Demo page.

Best regards,

Simon

[Dimo]

Hi all,

Let me provide some in-depth information about the observed behavior.

Virtual scrolling makes two types of requests - "prefetches" and "fetches".

"Prefetches" take care of preloading data while the user scrolls slowly (e.g. while reading). These are background requests, which cache data, so that it is available when needed. This mechanism ensures smooth user experience and thanks to it, the user may even never see the animated loading indicator.

"Fetches" take care of loading data when the user scrolls fast to a place where there is no cached loaded data.

Now, it is theoretically possible that a "prefetch" and a "fetch" request take place shortly after each other and create the impression of a duplicate request. This depends on the following:

- the scrolling speed
- the Grid height
- the page size
- the server response speed

All the requests are asynchronous, and the Grid dataSource is not aware whether it has already sent a "prefetch" request for certain data. As a result, if the user reaches a place with missing data, the dataSource will check if this data is already cached or not and if not, unconditionally send a "fetch" request for it.

Regards,
Dimo
Telerik

[Simon]

Thank you Dimo for the details about this behavior.

In a near future, can we expect the Prefetches / Fetches to notify each other by tracking each asynchronous requests sent to the server?  That way, we could avoid sending 2 identical request to the server in a short period of time and increase the global performance of every user.

[Dimo]

Hi Simon,

I am afraid we have no short-term plans to change the described implementation and behavior.

Regards,
Dimo
Telerik

[Simon]

Hello Dimo,

I already asked for more control over the prefetch in this post.  Meanwhile, I would like to know if there a way to have a distinction between a Prefetch and a Fetch call. (in the transport.read and the dataBound callback)

That would be very useful.

Best regards,

Simon

[Dimo]

Hi Simon,

I am afraid there is no way to distinguish between fetches and prefetches in the exposed Kendo UI events.

Recently we implemented an option to stop prefetches for specific internal usage of our own. I will show you how to achieve this, but please keep in mind that this is an internal API, which is not intended for the public. We do not support it, do not claim that it will exist or work in the same way in the future, and you can use it at your own risk. In addition to disabling prefetches, the option also prevents the Grid from making any requests while the user is still scrolling.

Kendo UI 2014.3.1316 is required.

kendo.ui.VirtualScrollable.fn.options.prefetch = false;

$("#grid").kendoGrid({
    scrollable: {
        virtual: true
    }
});

Regards,
Dimo
Telerik

[Simon]

Hello Dimo,

I tried your solution and with Kendo UI 2014.3.1316 and I'm afraid this cause an other problem.

If I set the prefetch to false, the DataBound callback is called on every scroll (even when it's just a line).  Can you please check for an other solution because those prefetch calls does have an impact on the server resources.

Regards,

Simon

[Dimo]

Hi Simon,

The observed behavior is by design - in this unofficial mode the Grid is rebound on every scroll-stop (and only then). If there is no cached data, a remote request will occur. I am afraid I have no other suggestions.

Regards,
Dimo
Telerik

[Simon]

Hello Dimo,

You already mentioned it wan't possible at this moment, but could you simply add an indicator telling if the request is a prefetch or a fetch call?  That would be a simple way to give us control over those requests involving minimal modification on kendo's side.

I really want to help and contribute (not just asking for feature) with this one so I took a look at the code.

In kendo.data.js, you only need to add 1 lines of code:

prefetch: function(skip, take, callback) {
            var that = this,
                size = math.min(skip + take, that.total()),
                options = {
                    take: take,
                    skip: skip,
                    page: skip / take + 1,
                    pageSize: take,
                    sort: that._sort,
                    filter: that._filter,
                    group: that._group,
                    aggregate: that._aggregate,
                    prefetch, true
                };

The prefetch value could also be set to false as the default value in the options for a regular request, but that's also easy to do.

Best regards,

Simon

[Simon]

prefetch: true

To bad we can't edit our post...

[Dimo]

Hi Simon,

The Grid doesn't know if a request is a prefetch or a fetch, because the Grid is not aware what data has already been loaded and cached by the DataSource. Both types of requests use the prefetch() method of the Kendo UI DataSource. It is theoretically possible to introduce some flags to distinguish between "background requests" which occur during slow scrolling and "direct requests", which occur during fast scrolling, but these flags need to be injected at an earlier stage in the code than suggested, and be passed through several steps in the algorithm, which will not represent good coding practice. I consulted our Grid developers about this matter and I am afraid we do not intend to implement the discussed request flag.

Regards,
Dimo
Telerik

[Jared]

This same behavior occurs with the DropDownList widget... is there any word on this being fixed? We have like a million records (no joke) so this is VERY bad if a user scrolls to the bottom. The browser queues like 10,000 AJAX requests.

[Alexander Valchev]

Hi Jared,

DropDownList virtualization implementation differs from Grid's one.
The control should not make 10,000 requests for the same data range. Did you checked if the requests are for the same data range? Could you please share your DropDownList and DataSource configuration so we can take a look and assist you further?

Regards,
Alexander Valchev
Telerik

 

 

[Jared]

Hi Alexander, the requests are not for the same data range, they are for sequential data ranges. For example, if anyone scrolls from the top to the bottom, the control will queue up thousands of async AJAX requests to the controller, each for the next sequential range of data. I can see the queued requests in my browser's development tools.

[Alexander Valchev]

Hi Jared,

If the requests are for a different range the behaviour is normal. While the user scrolls down, the widget is making requests for new data pages. There is throttling but our tests shows that if we increase the timeout the user experience is not smooth.

You can be sure that although the data is retrieved only a small chunk of it is rendered in the DOM tree.

Regards,
Alexander Valchev
Telerik

 

 

[Jared]

You are correct that only a small chunk is rendered, but the problem is that when scrolling through a list of millions, all of the million records are retrieved and transferred over the network - which takes a long time while the list says "loading..." for each of the items it's waiting for in the current "page" of the dropdownlist. 

 

Retrieving and transferring hundreds of thousands of records is VERY slow!

[Simon]

Jared is right...  every kendo demo are made with few hundred records only.  In those case, yes, the data prefetch may give a smoother user experience.

However, if you have many thousands of records to display, the virtual scrolling will send hundreds of requests while scrolling.  Knowing that large queries paging is slower and that this behaviour will be replicated for every users working simultaneously, your DB server will end up spending most of its time executing requests that won't even be displayed to the user as he already scrolled somewhere else by the time the data gets back to the client side.

All those small requests put together have a significant impact by decreasing the global user experience.  Please, consider adding an optional parameter that would allow us to prevent all those unwanted requests.

Best regards,

Simon

 

[Alexander Valchev]

Hi guys,

We will investigate various ways to improve the data request throttling. We will consider adding a 'delay' configuration option to the virtual object that can be used by the developer to configure how often the dataSource will send requests.

I logged this as an enhancement. You may follow its status here.

Regards,
Alexander Valchev
Telerik

 

 

[Frank]

According to the last link, https://github.com/telerik/kendo-ui-core/issues/916, this fix was only made for AutoComplete/ComboBox/DropDownList/MultiSelect.

Why wasn't this added for grid as well?

[Alexander Valchev]

Hi Frank,

The enhancement issue that you refer to was created only for the DropDown editor widgets. As explained earlier in the topic, the DropDown editor widgets (DropDownList, ComboBox, AutoComplete, MultiSelect) use different virtualization algorithm compared to the Grid widget.

Regards,
Alexander Valchev
Telerik

 

 

[Anthony]

Hi

  I've got to research implementing an endlessly scrolling grid, the first thing I did was google for issues you might face if you have a slow server, which we will, which brought me here.

 The last post suggests there is still an issue with grids where the server is slow to return, however I took your example, set the page size to 10, and even when I scroll through right to the bottom it seems to work as I would expect, i.e. it doesn't load all of the intermediate stages.

http://runner.telerik.io/fullscreen/igUVe

 

Is this an issue that's been fixed? The last post is from the 9th of May and our Kendo UI version is older than that. If not do you have an example that demonstrates the problem I'm likely to face?

 

If there is an issue I was thinking of making it page client side, insert a dummy row, then set a polling bit of javascript to check the grid and get the data every 400ms or so.

thanks

[Dimo]

Hi Anthony,

We have not changed the Grid and DataSource behavior with regard to the current discussion. The following information still applies:

http://www.telerik.com/forums/grid-with-remote-virtual-scrolling-fires-mutiple-read-requests-when-scrolling-fast#CsEuBWOjZkmCwbBUYsDNYQ

Depending on the scrolling speed, two requests for the same data can be observed on our online demo too:

http://demos.telerik.com/kendo-ui/grid/virtualization-remote-data

Here is a video:

http://screencast.com/t/gDQAhobFrtB

Regards,
Dimo
Telerik by Progress

[Dave]

Two years have gone by and still with the latest grid this problem happens. And its not theoretical, i can prove that it happens the overwhelming majority of the time as scrolling speed increases. This causes a tremendous amount of increased traffic on the network as the server gets pounded with page requests for the same data. With many users, this is not a trivial problem. When will there be a solution to this?

[Stamo Gochev]

Hi Dave,

I have already answered the original support ticket that you sent on the same topic and I am posting the answer here for reference purposes only:

The forum post that I suppose you are referring to:

https://www.telerik.com/forums/grid-with-remote-virtual-scrolling-fires-mutiple-read-requests-when-scrolling-fast#CsEuBWOjZkmCwbBUYsDNYQ

provides the explanation of the behavior, which is still true, so what I can suggest is to share your ideas in the Feedback Portal, so we can better prioritize the items for future releases based on the received feedback:

http://kendoui-feedback.telerik.com/forums/127393-kendo-ui-feedback

Regards,
Stamo Gochev
Progress Telerik