Updating grid with huge live feed

[Yoav]

Hi,

I want to use the Telerik grid to show a read only live feed of millions of records (while still supporting sorting, filtering, etc.).
Is it supported (I will also want to use the DataFilter to filter the live feed)? 

I have two questions:
1. I went over the documentation and couldn't find any performance guidance on how to populate the grid with millions of records of live data.
Should I feed the grid one row at a time by adding it to the Items collection? 
Should I use ObservableCollection and add rows to it when they are available?
Should I add single rows or should I add batches of rows?
If I'm using an ObservableCollection, I can't add records to it from a worker thread. I need to use the Dispatcher to actually feed the collection, which seems to slow down everything... What is the right way to do so?
I tried implementing the ObservableCollection approach, adding 3000 rows every second (while using the Dispacher and DeferRefresh), but it is a bit slow and not so responsive. 

2. When implementing the ObservableCollection and the Dispatcher, I noticed a strange scrolling issue. Every time there is an update (adding rows to the collection) the grid scrolls back to the first row.
I actually want a different behavior and I can't figure out how to do that.
I want the grid to automatically scroll to the last row, but if the user is manually scrolling, I want the automatic scrolling to stop and the grid to stay where the user scrolled to.
I can probably implement the auto scrolling myself, but I can't find how to tell the grid not to scroll to the first line. How can I do that?

Thank you,
Yoav.

[Yoan]

Hi Yoav,

About your first question:

By defaut, RadGridView uses UI virtualization for its visual elements. Only the rows that are currently visible have corresponding visual elements. Upon scrolling they are reused. You can find more about it here. You can find some tips and tricks about improving the performance here.

You could check the RadObservableCollection to add a range of data. You can use its SuspendNotifications() method to suspend the notifications, perform desired changes in the collection and then call ResumeNotifications() method which will trigger Reset CollectionChanged action.

I suggest you to check the GridView's "Real-time Update" example in your WPF demos. You may find it helpful.

About your second question:

If you add an item to the bound ObservableCollection, then the CollectionChanged will be raised with an Add action and the new record will be shown in the GridView. This means that the GridView will get updated every time a new record is added to the bound collection. Maybe in your case, your collection gets Reset and that is the reason for resetting the scrollbar. If you edit the existing items instead of Resetting the collection, then the scrollbar will remain at the same position.

Regards,
Yoan
the Telerik team

[Yoav]

Hi Yoan,

I will check the RadObservableCollection soon.
Regarding the second part, I need a way to tell the grid not change the scrolling position.
I'm using the grid in read only mode, so I will never edit existing items. I'm just adding new items all the time (and always to the end of the grid).
I want to be able to add items without moving the scroll bar.
Is there a way for me to tell the grid to not reset the scroll bar when adding items?
Is there a way for me to maybe use the SuspendNotifications, remember the scroll bar position, add rows, restore the scroll bar position and ResumeNotifications?

How do I get the current value of the grid scroll bar and how do I set this value? How can I get the maximum value allowed for the scroll bar (to later implement the scroll to the end)?

Thank you,
Yoav.

[Pavel Pavlov]

Hi,

You can get hold of the ScrollViewer instance inside RadGridview the following way :

GridViewScrollViewer scrollViewer = this.MyRadGridView.ChildrenOfType<GridViewScrollViewer>();

*The Children of type extension method is located in the Telerik.Windows.Controls namespace.

The Scrollviewer itself provides all the properties and methods to get the current position,  get the view port and extent values and scroll to a specific position


Greetings,
Pavel Pavlov
the Telerik team

[Yoav]

Regarding the GridViewScrollViewer, it is not available in my window CTor. I was able to aquire a reference to in the Loaded event. Is this the right place to initialize the reference to it (I don't want to look it up on every change)?

To lock the scroll bar in place, I'm doing this:

var x = scrollViewer.VerticalOffset;
this.items.ResumeNotifications();
scrollViewer.ScrollToVerticalOffset(x);

It seems like it is working, but I'm afraid that on faster machines, I will see the scrollbar jumping to 0 and back to the current position.
Is there a way to tell the scroll bar not to update?
I tried using the grid DeferRefresh(), but it prevents from the VerticalOffset to get updated as well.

Regarding the RadObservableCollection, I'm trying to use it, but every time I do a ResumeNotifications, the grid freezes for a second, while adding the bulk of rows.
How can I add a bulk of rows without freezing my app?

Thanks,
Yoav.

[Pavel Pavlov]

Hello,
On your first question - Yes the Loaded or even the first LayoutUpdated event handlers are good places to get the ScrollViewer.The constructor is too early as the visual tree has not yet been built.
RadGridView would always reset the scrollbar when adding items and so far other mode is not supported.
This and the bulk that load time can be partially solved with adding some visual busy indicator to mask the effect.

All the best,
Pavel Pavlov
the Telerik team

[Yoav]

This is really bad news.
Adding busy indicator will make my app useless.
My app is expected to have a constant flow of trace messages appearing on the UI. This means that it will always be in a "busy" state.
Does this mean that I need to drop the Telerik grid and look for other alternatives more suitable for live stream of data?

Thanks,
Yoav.

[Pavel Pavlov]

Hi Yoav,

Can you please share a bit more details on the nature of your data e.g. number of columns , number of rows, time intervals between the data updates and the size of the batch updates (e.g how many records are being periodically added) .

Based on this we can build a prototype here and  analyze the issue deeper and eventually propose a descent solution for your specific scenario.

I have just talked to the team and they had some ideas ( alternatives to using the RadObservableCollection)  which could solve the problem with the jumping scrollbar as well .

Please let me know the type and the size of your data , so we can address the issue directly .

Regards,
Pavel Pavlov
the Telerik team

[Yoav]

Hi,
Thank you for your reply.
My app is a live trace log viewer.
Each event will have roughly 10-20 properties, out of which I would assume only up to 10 will be visible (the others will be displayed in a detailed section).
I'm expecting a very high load of events. The events can come as single events, or I can do some grouping and batch them, if it helps.
Let's assume 200 events every second (sometimes it will be lower than that, I hope it will never be more than that).

It will be great if you can come up with a good performing WPF solution.

Thank you very much!
Yoav.

[Pavel Pavlov]

Hello Yoav ,

I am attaching a small sample with 200 items added per second , while keeping the scroll position at the end.

This is as fast as RadGRidView can perform currently . Please have a look and let me know if  such performance is ok for you .


All the best,
Pavel Pavlov
the Telerik team