This is a migrated thread and some comments may be shown as answers.

Unable to bind to greater than 3000 rows in Self Referencing Hierarchy

5 Answers 309 Views
Grid
This is a migrated thread and some comments may be shown as answers.
Chris
Top achievements
Rank 1
Chris asked on 11 Jan 2010, 06:36 PM
Hello,
I've been using the self referencing capability of the RadGrid for some time now. Today I was loading data in to the DB that the grid is pulling from and the grid starting throwing out of memory exceptions.

This grid is fully functional, and I had been using test data sets of up to several hundred rows. When I loaded the real data set of just over 6500 rows the grid started throwing the error. I was able to get it work up to 3400 rows, after that the error is thrown. Page processing takes a couple of minutes when it's trying to load the data, during which you can see it taking up memory little by little until .NET tells it to take a hike. The system has 8GB of memory installed, and a test server has 4GB. Fails on both.

This is clearly a bug to me. This is basic text, each row contains a couple of GUIDs and a single text sentence, and it can't handle a few thousand rows? DataTables, GridViews, et all, work without issue.

There is no paging enabled on this grid nor can there be, as it is a grid of a hierarchy that needs to keep expanding.

Please let me know if there is something wrong with my setup, or if there is a known fix. My markup is below, as well as the error message.

<telerik:RadGrid runat="server" ID="TaxonomiesRadGrid" AllowMultiRowSelection="true" Visible="false"  
    OnNeedDataSource="TaxonomiesRadGrid_NeedDataSource" Skin="WebBlue" AutoGenerateColumns="false"  
    Width="100%" GridLines="None" MasterTableView-GridLines="None" ClientSettings-AllowGroupExpandCollapse="true"  
    OnItemCreated="TaxonomiesRadGrid_ItemCreated" ShowStatusBar="true" OnInsertCommand="TaxonomiesRadGrid_InsertCommand"  
    OnUpdateCommand="TaxonomiesRadGrid_UpdateCommand" OnDeleteCommand="TaxonomiesRadGrid_DeleteCommand">  
    <MasterTableView HierarchyDefaultExpanded="false" HierarchyLoadMode="Client" CommandItemDisplay="Top"  
        CommandItemSettings-AddNewRecordText="Add a new Taxonomy Parent" EditMode="InPlace"  
        NoMasterRecordsText="No taxonomy items to display. Please add an item using the link above."  
        DataKeyNames="taxonomyItemId, taxonomyParentId">  
        <SelfHierarchySettings ParentKeyName="taxonomyParentId" KeyName="taxonomyItemId"  
            MaximumDepth="100" />  
        <Columns>  
            <telerik:GridBoundColumn DataField="taxonomyText" DataType="System.String" HeaderText="Taxonomy Description"  
                SortExpression="taxonomyText" UniqueName="taxonomyText">  
            </telerik:GridBoundColumn>  
            <telerik:GridEditCommandColumn ButtonType="PushButton" ItemStyle-Width="120px" ItemStyle-HorizontalAlign="Right">  
            </telerik:GridEditCommandColumn>  
            <telerik:GridClientDeleteColumn ButtonType="PushButton" ItemStyle-Width="50px" ItemStyle-BorderWidth="0px"  
                ItemStyle-HorizontalAlign="Center" ConfirmText="Are you sure you want to delete this taxonomy item and any items beneath it?">  
            </telerik:GridClientDeleteColumn>  
        </Columns>  
    </MasterTableView>  
    <ClientSettings AllowExpandCollapse="true" AllowGroupExpandCollapse="true">  
    </ClientSettings>  
</telerik:RadGrid>  


Server Error in '/NTTR' Application.  
  
Exception of type 'System.OutOfMemoryException' was thrown.  
  
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.   
  
Exception Details: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.  
  
Source Error:   
  
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.  
  
Stack Trace:   
  
  
[OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.]  
   System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity) +29  
   System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength) +69  
   System.Text.StringBuilder.Append(String value) +97  
   System.Web.UI.ScriptControlManager.RegisterScriptsForScriptDescriptors(IEnumerable`1 scriptDescriptors, Control control) +240  
   System.Web.UI.ScriptControlManager.RegisterScriptDescriptors(IScriptControl scriptControl) +113  
   System.Web.UI.ScriptManager.RegisterScriptDescriptors(IScriptControl scriptControl) +21  
   Telerik.Web.UI.RadCompositeDataBoundControl.RegisterScriptDescriptors() +37  
   Telerik.Web.UI.RadCompositeDataBoundControl.RenderContents(HtmlTextWriter writer) +38  
   Telerik.Web.UI.RadGrid.RenderContents(HtmlTextWriter writer) +56  
   Telerik.Web.UI.GridBaseDataList.Render(HtmlTextWriter writer) +88  
   Telerik.Web.UI.RadGrid.Render(HtmlTextWriter writer) +125  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25  
   Telerik.Web.UI.ControlRenderer.Render(HtmlTextWriter writer) +84  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25  
   System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +134  
   System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19  
   System.Web.UI.Control.Render(HtmlTextWriter writer) +10  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25  
   System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +134  
   System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19  
   System.Web.UI.UpdatePanel.RenderChildren(HtmlTextWriter writer) +234  
   System.Web.UI.Control.Render(HtmlTextWriter writer) +10  
   System.Web.UI.UpdatePanel.Render(HtmlTextWriter writer) +34  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25  
   Telerik.Web.UI.PreControlToAjaxify.Render(HtmlTextWriter writer) +112  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25  
   System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +134  
   System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19  
   System.Web.UI.Control.Render(HtmlTextWriter writer) +10  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25  
   System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +134  
   System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19  
   System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer) +163  
   System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer) +32  
   System.Web.UI.HtmlControls.HtmlForm.Render(HtmlTextWriter output) +51  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.HtmlControls.HtmlForm.RenderControl(HtmlTextWriter writer) +40  
   System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +134  
   System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19  
   System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer) +32  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25  
   System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +134  
   System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19  
   System.Web.UI.Control.Render(HtmlTextWriter writer) +10  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25  
   System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +134  
   System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19  
   System.Web.UI.Page.Render(HtmlTextWriter writer) +29  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25  
   Telerik.Web.UI.RadAjaxControl.RenderPageInAjaxMode(HtmlTextWriter writer, Control page) +594  
   System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +256  
   System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19  
   System.Web.UI.Page.Render(HtmlTextWriter writer) +29  
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99  
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25  
   System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +1266  
  
Version Information: Microsoft .NET Framework Version:2.0.50727.4200; ASP.NET Version:2.0.50727.4016  

5 Answers, 1 is accepted

Sort by
0
Sebastian
Telerik team
answered on 12 Jan 2010, 08:28 AM
Hello Chris,

My colleague Veli already responded in the support ticket you opened with regards to this subject. I am pasting his reply in this forum thread for further reference:

Thank you for contacting us. Please note that 3000 rows is quite a number of records RadGrid needs to build hierarchy on. When using self-referencing hierarchy, RadGrid creates and maintains data relations for all your data items, involving multiple loops over multiple sets of data. Additionally, if you have relations resulting in multiple nested levels, this additionally slows execution down, as numerous nested tables need to be initialized and databound.

With such a large number of data records for self-referencing hierarchy, we advise you to consider alternative visualization options or try to reduce the number of data RadGrid binds to. If other hierarchical visualization options like nested tables or nested view template are applicable, consider testing such setups too. Where possible, enable server-on-demand or mixed hierarchy loading or paging.


Kind regards,
Sebastian
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.
0
Chris
Top achievements
Rank 1
answered on 12 Jan 2010, 07:34 PM
Hi Sebastian,
Thank you for your replay. I did get the response to my support ticket as well and replied there, but wanted to copy it here for reference as well. But to make a long story short, I have proof that this is a bug, see below:

Hi Veli,
Thanks for your prompt reply. I've been doing a little research and testing on this issue. As it turns out, the self-referencing feature effectively becomes unusable at 1350 rows. If you look at the attached image, I have charted the load time for the data from the database and the render of the grid, and it is very interesting. The time it takes to get the data from the database never exceeds 35 ms even at 3500 rows, and progresses at a near linear rate - as you would expect. 

However, the results for the grid are very different. Up until 1300 rows, the render time varies linearly from 151 ms to 206ms - all acceptable load times for a page being served to users. However, at 1350 rows the load time spikes to over one full second, and quickly escalates to nearly six seconds for 4000 rows, culminating in a plateau around nine seconds for up to 2750 rows. At 3000 it jumps to over 18 seconds, then to over 52 seconds for 3250 rows, and nearly 85 seconds for 3500 rows. After this it throws either an out-of-memory exception or the following: Error during serialization or deserialization using the JSON JavaScriptSerializer. The length of the string exceeds the value set on the maxJsonLength property. 

Clearly, this is a major issue. First and foremost, if this was simply an issue with performance, then I would think the grid load times should scale linearly with the number of rows, or likewise be consistently exponential. I would not expect it to have a quick surge, then plateau, and then exponential growth, as is shown on the chart. In addition, 1350 rows, or 3000, or 6000, or even 10,000, are not that many rows when the data within is simply two GUIDs and a single text sentence. If the database server can load it, order it, and send it to the web server in a only a few milliseconds, then I can't believe it's reasonable that the grid needs 85 seconds to process all that. 

I have a purely self referencing table that contains about 7,000 rows (will grow to 10-20k tops); the rows themselves contain single sentence descriptions of each level of the hierarchy. The rows are organized simply using itemId and parentItemId fields. 

It seems to me, as it currently stands, your self referencing hierarchy feature is all but useless. Paging will not work as it won't know which row belongs on which page without processing the whole of the data.

Edit, I wrote what is above this morning, then spent the rest of the time until now looking through Telerik source code and trying different things. The biggest item of note, which was my latest test, I replaced the self referencing RadGrid with a self referencing RadTreeView, connected to all 7,000 rows, and the total load time is just 550 mson average!!!

Secondly, I went back and reran some of the tests to check for memory usage rather than load times. Like the load times, memory usage is stable up until 1300 rows. After that, it starts to spike, culminating with maxing out at 1.5 GB at 3500 rows. It wanted more, .NET wouldn't let it have it. 

What does this mean? I think it is OBVIOUSLY a substantial bug. It was pretty confident based on load times, very confident based on memory usage, and now without a doubt confident about it considering the RadTreeViews efficiency. The only thing these two controls should be doing differently is their rendering of the items, and I'm sorry but you cannot convince me rending child elements in the grid takes 85 seconds and 1.5 GB of memory. 

Where do we stand now? Well, I can replace all my uses of the RadGrid with the TreeView for my search and select functionality, however, I need to have the Insert, Update, and Delete capabilities on the nodes that the grid offers. Not to mention all these features have been completed and tested, I was only doing load testing when I noticed this problem. So I need this RadGrid to work as advertised.

So I respectfully request that you escalate this ticket to the responsible product development team, and have them get an estimate of what is wrong and how long it will take to fix. This was supposed to go to production in two weeks, but because of this bug in the Telerik controls we will now have to delay it. 

Thank you,
Chris Cicchitelli
0
Veli
Telerik team
answered on 13 Jan 2010, 09:53 AM
Hi Chris,

Here is my reply in the support ticket.

Hi Christopher,

I am sorry for your disappointment. Even though the numbers speak for themselves, note that RadGrid goes through a considerably larger life cycle and data binding logic than RadTreeView. Allow me to disagree with your statement that the only thing these two controls should be doing differently is their rendering of the items. The mere fact that you can benefit from extensive RadGrid functionality like paging, sorting, filtering and data editing speaks for itself in terms of how much more time it takes for RadGrid to prepare its structure, in comparison to RadTreeView. I believe there is no much ground in comparing load times of a control that only binds and visualizes a set of hierarchical data items, with a control that additionally enables you to sort,page and filter through the data, as well as edit and insert new items with a few clicks.

Let me also mention a few words on how self-referencing hierarchy is built in RadGrid. At its core, a recursive function goes over each hierarchy level and builds the GridTableView objects with their data and functional items. The larger portion of your memory and CPU usage goes to maintain the stacking context of this recursion and it is only natural with a deep hierarchy.

As for the chart you have been discussing, here is the expected complexity. Given an evenly distributed sample, i.e. a hierarchical set containing nearly equal number of items in each level, you can expect the complexity of the recursion to be exponential. Every detail table at every level filters all the data items to extract the set of items belonging to the current hierarchy levels.

This means that if you have 10 levels of hierarchy (the default RadGrid.MasterTableView.SelfHierarchySettings.MaximumDepth value), your computational complexity would be O(N^10), where N is the number of data records fetched from the database. If you have an arbitrary M levels of hierarchy, the complexity of the hierarchical recursion would be O(N^M).

Addtionally, consider the renderd HTML and the time it takes for the browser to process the markup and render it. You are loading all your hierarchy levels at once.

That being said, I can assure you that our development team has been notified about this issue and will consider it for research. However, we cannot, at the moment, give any specific schedules for fixes or any guarantees that action will be taken.

You can addtionally consider RadTreeView as an alternative for hierarchical rendering, if the control's performance is within reasonable limits. If you need advanced data operations like data editing, however, you will need to implement them yourself.

Kind regards,
Veli
the Telerik team


Kind regards,
Veli
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.
0
Raji
Top achievements
Rank 1
answered on 29 Jan 2013, 11:46 PM

Hi,

I am getting System out of memeory error. When I try to use the property HierarchyLoadMode="Client" to the detailstable. I have a radgrid with 7 nested tables.

Is this a known defect?

Thanks,
Raji

0
Pavlina
Telerik team
answered on 01 Feb 2013, 06:41 PM
Hello Raji,

Can you specify if the error appears only when you set HierarchyLoadMode to Client? I am afraid that we can not determine what is causing this error without a working project which demonstrates it. Therefore, I suggest you isolate the problem in a sample project and send it to us via support ticket, so we can debug it and advice you further.

Kind regards,
Pavlina
the Telerik team
If you want to get updates on new releases, tips and tricks and sneak peeks at our product labs directly from the developers working on the RadControls for ASP.NET AJAX, subscribe to their blog feed now.
Tags
Grid
Asked by
Chris
Top achievements
Rank 1
Answers by
Sebastian
Telerik team
Chris
Top achievements
Rank 1
Veli
Telerik team
Raji
Top achievements
Rank 1
Pavlina
Telerik team
Share this question
or