Filter Optimization

[Michael]

I'm building a module for DNN Pro 5.6.1 using the built in Telerik (version 2010.1.309.35) controls.  The intention of the app is to filter a List(of T) using the radFilter control.  I have the mechanics worked out and everything is working as designed.  I'm building the list of filtered columns dynamically in the page_init method.

Protected Sub Page_Init(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Init
    Try
        BindFilterAttributes()
    Catch ex As Exception
        ProcessModuleLoadException(Me, ex)
    End Try
End Sub
Private Sub BindFilterAttributes()
    For Each thisFieldEditor As RadFilterDataFieldEditor In Me.FieldEditors
        rflFilter.FieldEditors.Add(thisFieldEditor)
    Next
End Sub

With each click on the control, whether to add or remove filter conditions or to change the filter in any way the control gets progressively slower.  At first the control completes it's postback in less than one second.  After twenty clicks, the control takes about three seconds to return for each click.   

Has anyone else experienced this behavior?  Are there methods I could use to reduce the number of postbacks, reduce the amount of data going back and forth to the server, or in any other way improve the performance?

[Michael]

I've continued to experiment to try to find the cause of the deteriorating performance.  I've tried four things and learned two.

1. Changed the Ajax solution for the control from an radAjaxPanel to radAjaxManager with no effect.
2. Commented out the code behind and built 32 FieldEditors of different types in the markup.  This removed the performance deterioration entirely
3. Changed back to the dynamic population scheme and changed the code to read:

   Private Sub BindFilterAttributes()
       rflFilter.FieldEditors.Clear()
       For Each thisFieldEditor As RadFilterDataFieldEditor In Me.FieldEditors
           rflFilter.FieldEditors.Add(thisFieldEditor)
       Next
   End Sub

   There was no change in the behavior (performace still deteriorated to 3 second for the 21st click).
4. Changed the code again, this time to limit the number of FieldEditors added to the filter:

   Private Sub BindFilterAttributes()
       Dim thingy As Integer = 0
       For Each thisFieldEditor As RadFilterDataFieldEditor In Me.FieldEditors
           If thingy < 5 Then
               rflFilter.FieldEditors.Add(thisFieldEditor)
               thingy += 1
           Else
               Exit For
           End If
       Next
   End Sub

   This reduced the impact to performance, but there was still a cumulative effect for each click on the control.

Any ideas at all would be most welcome.

[Tsvetina]

Hello Michael,

How are the FieldEditors created before you add them to the FieldEditors collection of RadFilter? The general approach to create editors on Page_Init would be like:

Dim editor As New RadFilterTextFieldEditor()
editor.FieldName = "Field1"
editor.DataType = GetType(Integer)
RadFilter1.FieldEditors.Add(editor)

All the best,
Tsvetina
the Telerik team

[Michael]

Thank you for your response.  That is essentially what I'm doing.  In whole, I First check to see if my lst of FieldEditors has already been created and stored in the server cache.  If not, I loop through the fields available for filtering, create the appropriate FieldEditor object for the given data type, give the FieldEditor a name and add it to the list.

I have created a sample project that reproduces the behavior.  I would be happy to share that with you.

Private ReadOnly Property FieldEditors() As RadFilterDataFieldEditorCollection
    Get
        If GetCache("shl_FieldEditors") Is Nothing Then
            Dim objList As New ListController
            Dim thisDataTypeList As ListEntryInfoCollection = objList.GetListEntryInfoCollection("DataType")
            Dim thisFieldEditor As RadFilterDataFieldEditor = Nothing
            Dim thisFieldEditorCollection As New RadFilterDataFieldEditorCollection(rflFilter)
            For Each thisAttribute As ProfilePropertyDefinition In Me.InterestingProfileProperties
                Dim tempAttribute As ProfilePropertyDefinition = thisAttribute
                If Not tempAttribute.PropertyCategory.EndsWith("_NS") Then
                    If Not rflFilter.FieldEditors.Any(Function(_thisAttribute As RadFilterDataFieldEditor) _thisAttribute.FieldName = tempAttribute.PropertyName) Then
                        For Each thisDataType As ListEntryInfo In thisDataTypeList
                            thisFieldEditor = Nothing
                            If thisAttribute.DataType = thisDataType.EntryID Then
                                Select Case thisDataType.Value
                                    Case "Date"
                                        thisFieldEditor = New RadFilterDateFieldEditor
                                    Case "Text", "Region"
                                        thisFieldEditor = New RadFilterTextFieldEditor
                                    Case "Integer"
                                        thisFieldEditor = New RadFilterNumericFieldEditor
                                        thisFieldEditor.DataType = GetType(System.Double)
                                    Case "TrueFalse"
                                        thisFieldEditor = New RadFilterBooleanFieldEditor
                                End Select
                                Exit For
                            End If
                        Next
                        If Not thisFieldEditor Is Nothing Then
                            thisFieldEditor.FieldName = thisAttribute.PropertyName
                            thisFieldEditorCollection.Add(thisFieldEditor)
                        End If
                    End If
                End If
            Next
            SetCache("shl_FieldEditors", thisFieldEditorCollection, DateTime.Now.AddHours(72))
        End If
        Return CType(GetCache("shl_FieldEditors"), RadFilterDataFieldEditorCollection)
    End Get
End Property

[Iana Tsolova]

Hello Michael,

Where is this property called? Can you send the whole page code so we can test it?

Greetings,
Iana
the Telerik team

[Michael]

Thank you very much for your help on this!  Here is both the markup and the code behind of the test module I created as an example of this behavior.

The commented section at the bottom of this clip is the markup I used as a test.  It's only there for reference.

<%@ Control Language="vb" AutoEventWireup="false" CodeBehind="radFilterTest.ascx.vb" Inherits="SHL.Modules.radFilterTest.radFilterTest" %>
<%@ Register Assembly="Telerik.Web.UI" Namespace="Telerik.Web.UI" TagPrefix="telerik" %>
<telerik:RadCodeBlock ID="RadCodeBlock1" runat="server">
    <script type="text/javascript">
        function pageLoad() {
            var filter = $find("<%=rflFilter.ClientID %>");
            var someStupidShit = filter.get_contextMenu();
            filter.get_contextMenu().add_showing(function(sender, args) {
                var currentExpandedItem = sender.get_attributes()._data.ItemHierarchyIndex;
                var fieldName = filter._expressionItems[currentExpandedItem];
                var allFields = filter._dataFields;
                var dataType = null;
                for (var i = 0, j = allFields.length; i < j; i++) {
                    if (allFields[i].FieldName == fieldName) {
                        dataType = allFields[i].DataType; break;
                    }
                }
                switch (dataType) {
                    case "System.Double":
                        sender.findItemByValue("EqualTo").set_text("Equals");
                        sender.findItemByValue("NotEqualTo").set_text("Does Not Equal");
                        sender.findItemByValue("Between").set_text("Is Between");
                        sender.findItemByValue("NotBetween").set_visible(false);
                        sender.findItemByValue("GreaterThan").set_text("Is More Than");
                        sender.findItemByValue("LessThan").set_text("Is Less Than");
                        sender.findItemByValue("GreaterThanOrEqualTo").set_visible(false);
                        sender.findItemByValue("LessThanOrEqualTo").set_visible(false);
                        sender.findItemByValue("IsNull").set_text("Is Empty");
                        sender.findItemByValue("NotIsNull").set_visible(false);
                        break;
                    case "System.Boolean":
                        sender.findItemByValue("EqualTo").set_text("Equals");
                        sender.findItemByValue("IsNull").set_text("Is Empty");
                        sender.findItemByValue("NotIsNull").set_text("Is Not Empty");
                        break;
                    case "System.String":
                        sender.findItemByValue("DoesNotContain").set_visible(false);
                        sender.findItemByValue("StartsWith").set_text("Starts With");
                        sender.findItemByValue("EndsWith").set_visible(false);
                        sender.findItemByValue("EqualTo").set_text("Equals");
                        sender.findItemByValue("NotEqualTo").set_visible(false);
                        sender.findItemByValue("GreaterThan").set_visible(false);
                        sender.findItemByValue("LessThan").set_visible(false);
                        sender.findItemByValue("GreaterThanOrEqualTo").set_visible(false);
                        sender.findItemByValue("LessThanOrEqualTo").set_visible(false);
                        sender.findItemByValue("Between").set_visible(false);
                        sender.findItemByValue("NotBetween").set_visible(false);
                        sender.findItemByValue("IsEmpty").set_text("Is Empty");
                        sender.findItemByValue("NotIsEmpty").set_visible(false);
                        sender.findItemByValue("IsNull").set_visible(false);
                        sender.findItemByValue("NotIsNull").set_visible(false);
                        break;
                    case "System.DateTime":
                        sender.findItemByValue("EqualTo").set_text("Equals");
                        sender.findItemByValue("NotEqualTo").set_text("Does Not Equal");
                        sender.findItemByValue("GreaterThan").set_text("Is After");
                        sender.findItemByValue("LessThan").set_text("Is Before");
                        sender.findItemByValue("GreaterThanOrEqualTo").set_visible(false);
                        sender.findItemByValue("LessThanOrEqualTo").set_visible(false);
                        sender.findItemByValue("Between").set_text("Is Between");
                        sender.findItemByValue("NotBetween").set_visible(false);
                        sender.findItemByValue("IsNull").set_text("Is Empty");
                        break;
                }
            });
        }
    </script>
</telerik:RadCodeBlock>
<telerik:RadFilter ID="rflFilter" runat="server">
</telerik:RadFilter>
<telerik:RadAjaxManager ID="RadAjaxManager1" runat="server">
    <AjaxSettings>
        <telerik:AjaxSetting AjaxControlID="rflFilter">
            <UpdatedControls>
                <telerik:AjaxUpdatedControl ControlID="rflFilter" />
            </UpdatedControls>
        </telerik:AjaxSetting>
    </AjaxSettings>
</telerik:RadAjaxManager>

<!--
    <FieldEditors>
        <telerik:RadFilterTextFieldEditor DataType="System.String" DisplayName="Text Field 1" FieldName="TextField1" />
        <telerik:RadFilterNumericFieldEditor DataType="System.Int16" DisplayName="Numeric Field 1" FieldName="NumericField1" />
        <telerik:RadFilterDateFieldEditor DataType="System.DateTime" DisplayName="Date Field 1" FieldName="DateField1" />
        <telerik:RadFilterBooleanFieldEditor DisplayName="Boolean Field 1" FieldName="BooleanField1" />
        <telerik:RadFilterTextFieldEditor DataType="System.String" DisplayName="Text Field 2" FieldName="TextField2" />
        <telerik:RadFilterNumericFieldEditor DataType="System.Int16" DisplayName="Numeric Field 2" FieldName="NumericField2" />
        <telerik:RadFilterDateFieldEditor DataType="System.DateTime" DisplayName="Date Field 2" FieldName="DateField2" />
        <telerik:RadFilterBooleanFieldEditor DisplayName="Boolean Field 2" FieldName="BooleanField2" />
        <telerik:RadFilterTextFieldEditor DataType="System.String" DisplayName="Text Field 3" FieldName="TextField3" />
        <telerik:RadFilterNumericFieldEditor DataType="System.Int16" DisplayName="Numeric Field 3" FieldName="NumericField3" />
        <telerik:RadFilterDateFieldEditor DataType="System.DateTime" DisplayName="Date Field 3" FieldName="DateField3" />
        <telerik:RadFilterBooleanFieldEditor DisplayName="Boolean Field 3" FieldName="BooleanField3" />
        <telerik:RadFilterTextFieldEditor DataType="System.String" DisplayName="Text Field 4" FieldName="TextField4" />
        <telerik:RadFilterNumericFieldEditor DataType="System.Int16" DisplayName="Numeric Field 4" FieldName="NumericField4" />
        <telerik:RadFilterDateFieldEditor DataType="System.DateTime" DisplayName="Date Field 4" FieldName="DateField4" />
        <telerik:RadFilterBooleanFieldEditor DisplayName="Boolean Field 4" FieldName="BooleanField4" />
        <telerik:RadFilterTextFieldEditor DataType="System.String" DisplayName="Text Field 5" FieldName="TextField5" />
        <telerik:RadFilterNumericFieldEditor DataType="System.Int16" DisplayName="Numeric Field 5" FieldName="NumericField5" />
        <telerik:RadFilterDateFieldEditor DataType="System.DateTime" DisplayName="Date Field 5" FieldName="DateField5" />
        <telerik:RadFilterBooleanFieldEditor DisplayName="Boolean Field 5" FieldName="BooleanField5" />
        <telerik:RadFilterTextFieldEditor DataType="System.String" DisplayName="Text Field 6" FieldName="TextField6" />
        <telerik:RadFilterNumericFieldEditor DataType="System.Int16" DisplayName="Numeric Field 6" FieldName="NumericField6" />
        <telerik:RadFilterDateFieldEditor DataType="System.DateTime" DisplayName="Date Field 6" FieldName="DateField6" />
        <telerik:RadFilterBooleanFieldEditor DisplayName="Boolean Field 6" FieldName="BooleanField6" />
        <telerik:RadFilterTextFieldEditor DataType="System.String" DisplayName="Text Field 7" FieldName="TextField7" />
        <telerik:RadFilterNumericFieldEditor DataType="System.Int16" DisplayName="Numeric Field 7" FieldName="NumericField7" />
        <telerik:RadFilterDateFieldEditor DataType="System.DateTime" DisplayName="Date Field 7" FieldName="DateField7" />
        <telerik:RadFilterBooleanFieldEditor DisplayName="Boolean Field 7" FieldName="BooleanField7" />
        <telerik:RadFilterTextFieldEditor DataType="System.String" DisplayName="Text Field 8" FieldName="TextField8" />
        <telerik:RadFilterNumericFieldEditor DataType="System.Int16" DisplayName="Numeric Field 8" FieldName="NumericField8" />
        <telerik:RadFilterDateFieldEditor DataType="System.DateTime" DisplayName="Date Field 8" FieldName="DateField8" />
        <telerik:RadFilterBooleanFieldEditor DisplayName="Boolean Field 8" FieldName="BooleanField8" />
    </FieldEditors>
-->

This is the entire code behind for the page, including methods that have no bearing on this issue.

Imports DotNetNuke
Imports DotNetNuke.Services.Exceptions
Imports DotNetNuke.Services.Localization
Imports DotNetNuke.Entities
Imports DotNetNuke.Entities.Users
Imports DotNetNuke.Entities.Profile
Imports DotNetNuke.Common.Utilities
Imports DotNetNuke.Common.Lists
Imports DotNetNuke.Common.Globals

Imports Telerik.Web.UI

Namespace SHL.Modules.radFilterTest
    Partial Public Class radFilterTest
        Inherits Entities.Modules.PortalModuleBase

#Region "Protected Members"
        Protected ReadOnly Property IsSuperUser() As Boolean
            Get
                If PortalSettings.ActiveTab.ParentId = PortalSettings.SuperTabId Then
                    Return True
                Else
                    Return False
                End If
            End Get
        End Property

        Protected ReadOnly Property UsersPortalId() As Integer
            Get
                Dim intPortalId As Integer = PortalId
                If IsSuperUser Then
                    intPortalId = Null.NullInteger
                End If
                Return intPortalId
            End Get
        End Property
#End Region

#Region "Private Properties "
        Private ReadOnly Property ProfileProperties() As List(Of ProfilePropertyDefinition)
            Get
                If Session("shl_ProfileProperties") Is Nothing Then
                    Dim AllProfileProperties As ProfilePropertyDefinitionCollection
                    Dim tempProfileProperties As New List(Of ProfilePropertyDefinition)

                    AllProfileProperties = ProfileController.GetPropertyDefinitionsByPortal(UsersPortalId, False)

                    For Each profProperty As ProfilePropertyDefinition In AllProfileProperties
                        If profProperty.Visible Then
                            tempProfileProperties.Add(profProperty)
                        End If
                    Next

                    'Sort the list by it's ViewOrder property
                    tempProfileProperties.Sort(AddressOf SortProfilePropertyByViewOrder)
                    Session("shl_ProfileProperties") = tempProfileProperties
                End If

                Return CType(Session("shl_ProfileProperties"), List(Of ProfilePropertyDefinition))
            End Get
        End Property

        Private ReadOnly Property FieldEditors() As RadFilterDataFieldEditorCollection
            Get
                If Session("shl_FieldEditors") Is Nothing Then
                    Dim objList As New ListController
                    Dim thisDataTypeList As ListEntryInfoCollection = objList.GetListEntryInfoCollection("DataType")

                    Dim thisFieldEditor As RadFilterDataFieldEditor = Nothing
                    Dim thisFieldEditorCollection As New RadFilterDataFieldEditorCollection(rflFilter)

                    For Each thisAttribute As ProfilePropertyDefinition In Me.ProfileProperties
                        Dim tempAttribute As ProfilePropertyDefinition = thisAttribute

                        If Not tempAttribute.PropertyCategory.EndsWith("_NS") Then
                            If Not rflFilter.FieldEditors.Any(Function(_thisAttribute As RadFilterDataFieldEditor) _thisAttribute.FieldName = tempAttribute.PropertyName) Then
                                For Each thisDataType As ListEntryInfo In thisDataTypeList
                                    thisFieldEditor = Nothing

                                    If thisAttribute.DataType = thisDataType.EntryID Then
                                        Select Case thisDataType.Value
                                            Case "Date"
                                                thisFieldEditor = New RadFilterDateFieldEditor
                                            Case "Text", "Region"
                                                thisFieldEditor = New RadFilterTextFieldEditor
                                            Case "Integer"
                                                thisFieldEditor = New RadFilterNumericFieldEditor
                                                thisFieldEditor.DataType = GetType(System.Double)
                                            Case "TrueFalse"
                                                thisFieldEditor = New RadFilterBooleanFieldEditor
                                        End Select
                                        Exit For
                                    End If
                                Next

                                If Not thisFieldEditor Is Nothing Then
                                    thisFieldEditor.FieldName = thisAttribute.PropertyName
                                    thisFieldEditorCollection.Add(thisFieldEditor)
                                End If
                            End If
                        End If
                    Next


                    Session("shl_FieldEditors") = thisFieldEditorCollection
                End If

                Return CType(Session("shl_FieldEditors"), RadFilterDataFieldEditorCollection)
            End Get
        End Property
#End Region

#Region "Event Handlers"
        Protected Sub Page_Init(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Init
            Try
                BindFilterAttributes()
            Catch ex As Exception
                ProcessModuleLoadException(Me, ex)
            End Try
        End Sub

        Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
            'Nothing to see here
        End Sub

        Protected Sub rflFilter_PreRender(ByVal sender As Object, ByVal e As EventArgs) Handles rflFilter.PreRender
            Dim menu As RadContextMenu = CType(rflFilter.FindControl("rfContextMenu"), RadContextMenu)
            menu.DefaultGroupSettings.Height = Unit.Pixel(200)
            menu.EnableAutoScroll = True

            SetOperatorButtonVisible(rflFilter.RootGroupItem)
        End Sub
#End Region

#Region "Private Methods"
        Private Sub BindFilterAttributes()
            For Each thisFieldEditor As RadFilterDataFieldEditor In Me.FieldEditors
                rflFilter.FieldEditors.Add(thisFieldEditor)
            Next
        End Sub

        Private Function SortProfilePropertyByViewOrder(ByVal x As Object, ByVal y As Object) As Integer
            Return CInt(CType(x, ProfilePropertyDefinition).ViewOrder < CType(y, ProfilePropertyDefinition).ViewOrder)
        End Function

        Public Sub SetOperatorButtonVisible(ByRef GroupItem As RadFilterGroupExpressionItem)
            For Each oControl In GroupItem.Controls
                Dim oHyperLink As HyperLink = CType(GetControlOfType(oControl, GetType(HyperLink)), HyperLink)
                If oHyperLink IsNot Nothing Then
                    If oHyperLink.Text.ToUpper = "AND" Then
                        oHyperLink.Visible = GroupItem.ChildItems.Count > 1
                    End If
                End If

                Dim oLinkButton As LinkButton = CType(GetControlOfType(oControl, GetType(LinkButton)), LinkButton)
                If oLinkButton IsNot Nothing Then
                    If oLinkButton.Text.ToUpper = "ADD EXPRESSION" Then
                        oLinkButton.Text = "Add Row"
                        oLinkButton.ToolTip = "Add Row"
                    End If
                End If
            Next

            For Each oChildItem In GroupItem.ChildItems
                If TypeOf (oChildItem) Is RadFilterGroupExpressionItem Then
                    Dim oGroup As RadFilterGroupExpressionItem = CType(oChildItem, RadFilterGroupExpressionItem)
                    SetOperatorButtonVisible(oGroup)
                End If
            Next
        End Sub

        Public Function GetControlOfType(ByVal ParentControl As Object, ByVal TypeOfControl As Type) As Control
            Dim oLinkButton As Control = Nothing

            For Each oSubControl As Control In CType(ParentControl, Control).Controls
                If oLinkButton Is Nothing Then
                    If oSubControl.GetType Is TypeOfControl Then
                        oLinkButton = oSubControl
                    Else
                        oLinkButton = GetControlOfType(oSubControl, TypeOfControl)
                    End If
                End If
            Next

            Return oLinkButton
        End Function
#End Region

    End Class
End Namespace

Please let me know if there is any addition information I can offer that would be of use to you.  Again, thank you very much for your help.

[Tsvetina]

Hi Michael,

We were not able to replicate the issue in a standalone aspx page (tried this way since debugging and tracking such issue in DNN is not that easy). Therefore, I am not sure if the problem comes from the dynamic creation of the field editors or from DNN itself. Could you please try to replicate the problem in a regular aspx web form and if you manage, send us the full code that you used?

In the meantime, one other thing that you could try is to create the field editors only on initial Page_Load (with a check if !isPostBack), so that they do not get recreated on each postback.

Kind regards,
Tsvetina
the Telerik team

[Michael]

Tsvetina,
Thank you for your response.  Could you confirm for me that you performed your test using version 2010.1.309.35 of the controls?  This is the version that comes with DNN Pro 5.6.1.

Also, since the FieldEditors list is built from a list of ProfileProperties, a DNN specific object, how did you perform the test outside of DNN without rewiting and/or commenting out the majority of my code?  May I have a copy of the rewritten code so that I can anylize the differences?

Because my license allows me to build with Telerik controls only for DNN Pro sites, am I allowed to build a test solution outside of DNN?  If so, doesn't the amount of reconfiguration necessary to pull the Telerik DLL and it's supporting files (if any) out of DNN to create the test solution almost ensure that I will create more issues that I'm able to resolve by doing so?

Thanks again.

Michael

[Tsvetina]

Hello Michael,

I am attaching the project that I used for testing purposes.

The issue that you describe can be replicated only if you remove the line where the FieldEditors are cleared but I see in your code that you also clear them on each postback. Please, review it and let me know if I am missing anything out.

Greetings,
Tsvetina
the Telerik team

[Michael]

Tsvetina,
Thank you again for your help.  I used visual studio to open a new website and pointed it to your test project.  I then ran the new site in debug and performed the same test I've been using within DNN.  I found that the behavior was exactly the same in your project as it was in mine.  I clicked the button to add a new filter item ten times, each time waiting until the new row in the filter control was displayed.  I then clcked the "X" on each row in the filter control to dismiss it, begining from the top.  The control got progessively slower with each click.  The first postback (from adding the very first row) took less than one second.  The twentieth postback (from removing the very last row) took just shy of three seconds to complete.

Do I understand correctly that you are not experiencing this same behavior?

To address your earlier question about using a check for !isPostBack, I took my guidance from this thread and have found no way to persist the list of field editors across postbacks.  If there is a way to use ViewState, Javascript or any other mechanism, I would very much like to learn about it.

Thanks again.
Michael

[Tsvetina]

Hi Michael,

I made a video of how the performance stays one and the same on my side while performing the steps listed by you. Could you take a look at the video and let me know if you are doing anything different?
http://screencast.com/t/GU1cXdQijUHT

All the best,
Tsvetina
the Telerik team