RadChart samplingthreshold & zoom problem

[Dan]

Hello,

I am using Telerik RadControls evaluation for testing, to build an application of generating graphs for industrial monitoring of a big number of sensors.

I ran into a problem with zooming and sampling threshold. The number of points for the graphics is very large, sometimes millions, sometimes thousands, based on the time interval selected, upon which the graph is generated.

The zooming is essential, and without sampling, the application is far too slow or is not responsive.
With zoom, it works very well.
Being an industrial application, it is necessary, to zoom into very small time intervals. The engineers that will use it want to go deep, even to see the difference, between two points that appear in a time interval of one or two seconds (and the number of points contained in a time interval of 2 seconds would be 4, meaning that every half second the sensors sends us data) .

I observed, that if the number of points that exists in the time interval, is smaller than the sampling threshold, the graphic runs wild, showing false data.

This is kind of a problem, because we want a decent sampling rate, to show the spikes in the graphs, and in the same time, this apparently does not allow us to zoom to the time intervals we want.

I tried to change the samplingThreshold on the ChartArea.ZoomScrollSettingsX.PropertyChanged event.

As i suspected, the sampling is calculated on the creation of the chart, and i am not allowed to change this type of property on this kind of event:
"Cannot change ObservableCollection during a CollectionChanged or PropertyChanged event."
This error, actually makes sense.

So the question, is : Could we work around this, or is there any other way to achieve what i described above? (maybe i do not know all possibilities of Radchart, or i am not approaching the problem from the right angle)

.Thank you for your time,

Dan Gheorghe
Spectra Computers
Software Engineer

[Ves]

Hi Dan,

The Sampling in RadChart kicks in only if necessary. That is -- if you configure your SamplingThreshold to 300 and the chart happens to be populated with 200 items, it will just show all of them without any sampling. The same is true when zooming. If you populate the chart with 1000 items, you will get 300 samples. However, when zooming, if at a certain points there are less than 300 points within the area, sampling will not be applied and you will get the actual data. That is -- you do not need to update SamplingThreshold when the user zooms the chart.

Please, find attached a small example, showing RadChart populated with 2000000 items and with SamplingThreshold set to 300. Initially you will see 300 samples, but after several zoom actions you will start to get the real values and eventually you will be able to drill to see just several of them.


Best regards,
Ves
the Telerik team

[Dan]

Hello,

Thank you for your time and answer. 

The problem is that on a deeper zoom the graphics go nuts. What you said in the previous post is reasuring, but the problem still persists. I never actually got to see the actual data on a deeper zoom.

When i have multiple graphics on the same chart, on a deeper zoom it shows random lines.

When i have only one graphic on the chart, the x axis looses gets lost.

I attached some pictures for more details.

(A picture values more than a thousand words ) :P

Thank you,

Dan

[Ves]

Hello Dan,

I would agree some of the images show unexpected results. Please, make sure the data is sorted or add SortDescriptors to RadChart to ensure that. I did test the 100000+ records example with second series and it still behaved as expected. In addition, as the result in zoom2 reminds me of an old bug,  please make sure you use the latest available version of RadControls -- currently 2010 Q3.

Best regards,
Ves
the Telerik team

[Dan]

Hello,

Unfortenately, the problem persists and we have the latest version of radcontrol.
I have attached here a small dataset we used for testing.
http://dl.transfer.ro/transfer_ro-28jan-48fcb27be5afca.zip

The dataset is in columnar format. Every timestamp has a value for the points selected.
I had to change the columnar format in rows format, where every point has a timestamp.
I have created a List of object of type :

public class ObjRaport
    {
        public DateTime timestamp;
        public string point_id;
        public double? value1;
        public double? value2;
        public double? value3;
        public double? value4;
        public double? value5;
        public double? value6;
        public double? value7;
        public double? value8;
        public double? value9;
        public double? value10;
    }

This is a dynamic report. The user can select maximum 10 signals. If he selects 3 signals, for example BB1, BB2, DWATt the list will contain values like ObjRaport.timestamp= x;
                ObjRaport.point_id=bb1;
                Objraport.value1=y;
                (the rest of the values will be null for this object)
ObjRaport.timestamp= x;
                ObjRaport.point_id=bb2;
                Objraport.value2=y;
                (the rest of the values will be null for this object)
ObjRaport.timestamp= x;
                ObjRaport.point_id=dwatt;
                Objraport.value3=y;
                (the rest of the values will be null for this object)
When i generate the report, i check the number of signals picked by the users, and generate the aditional Axes:
   for (int i = 0; i < semnale.Count; i++)
              {
                  radChart1.DefaultView.ChartArea.AdditionalYAxes.Add(new AxisY());
                  radChart1.DefaultView.ChartArea.AdditionalYAxes[i].AutoRange = false;
                  radChart1.DefaultView.ChartArea.AdditionalYAxes[i].AxisName = "Axis" + i;

                  radChart1.DefaultView.ChartArea.AdditionalYAxes[i].Title = semnale[i].Signal_Name;

                  radChart1.DefaultView.ChartArea.AdditionalYAxes[i].LabelRotationAngle = 270;
                  if (usMetric == true)
                  {
                      radChart1.DefaultView.ChartArea.AdditionalYAxes[i].AddRange(Convert.ToDouble(semnale[i].Eng_Min), Convert.ToDouble(semnale[i].Eng_Max), Convert.ToDouble(semnale[i].Eng_Max) / 5);
                      radChart1.DefaultView.ChartArea.AdditionalYAxes[i].Title = semnale[i].Units;
                      if (AxisYDuplicateExists(semnale, i, Convert.ToDouble(semnale[i].Eng_Min), Convert.ToDouble(semnale[i].Eng_Max)) == false)
                      {
                          radChart1.DefaultView.ChartArea.AdditionalYAxes[i].MajorGridLinesVisibility = Visibility.Visible;
                      }
                      else
                      {
                          radChart1.DefaultView.ChartArea.AdditionalYAxes[i].Visibility = Visibility.Collapsed;
                      }
                  }
                  else
                  {
                      radChart1.DefaultView.ChartArea.AdditionalYAxes[i].AddRange(Convert.ToDouble(semnale[i].Disp_Min), Convert.ToDouble(semnale[i].Disp_Max), Convert.ToDouble(semnale[i].Disp_Max) / 5);
                      radChart1.DefaultView.ChartArea.AdditionalYAxes[i].Title = semnale[i].Disp_Unit;
                      if (AxisYDuplicateExistsUE(semnale, i, Convert.ToDouble(semnale[i].Disp_Min), Convert.ToDouble(semnale[i].Disp_Max)) == false)
                      {
                          radChart1.DefaultView.ChartArea.AdditionalYAxes[i].MajorGridLinesVisibility = Visibility.Visible;
                      }
                      else
                      {
                          radChart1.DefaultView.ChartArea.AdditionalYAxes[i].Visibility = Visibility.Collapsed;
                      }
                  }
              }

Then i add the data:

public void createSeriesCompleted(object sender, wcfService.createSeriesEngineer2CompletedEventArgs e)
        {
            SamplingSettings sSettings = new SamplingSettings();
            radChart1.DefaultView.ChartArea.ZoomScrollSettingsX.ScrollMode = ScrollMode.ScrollAndZoom;

            radChart1.DefaultView.ChartArea.AxisY.AutoRange = false;
            radChart1.DefaultView.ChartArea.AxisY.Visibility = Visibility.Collapsed;

            radChart1.DefaultView.ChartArea.AxisX.DefaultLabelFormat = "MMM-dd/HH:mm:ss";
            radChart1.DefaultView.ChartArea.AxisX.LabelRotationAngle = 315;
            radChart1.DefaultView.ChartArea.AxisX.StepLabelLevelCount = 2;
            radChart1.DefaultView.ChartArea.AxisX.StepLabelLevelHeight = 20;
            // radChart1.DefaultView.ChartArea.AxisY.DefaultLabelFormat = "VAL{0:C0}";

            radChart1.DefaultView.ChartArea.AxisY.MajorGridLinesVisibility = Visibility.Visible;
            radChart1.DefaultView.ChartArea.AxisY.MinorGridLinesVisibility = Visibility.Visible;
            radChart1.DefaultView.ChartArea.AxisX.MajorGridLinesVisibility = Visibility.Visible;
            //  radChart1.DefaultView.ChartArea.Padding = new Thickness(5, 10, 20, 5);
            //  radChart1
            radChart1.DefaultView.ChartArea.LabelFormatBehavior = LabelFormatBehavior.None;

            radChart1.SamplingSettings.SamplingThreshold = 200;
        //    radChart1.DefaultView.ChartArea.ZoomScrollSettingsX.PropertyChanged += ZoomScrollSettingsX_PropertyChanged;

            radChart1.SamplingSettings.SamplingFunction = ChartSamplingFunction.KeepExtremes;
            radChart1.DefaultView.ChartArea.EnableAnimations = false;
            radChart1.DefaultView.ChartArea.EnableTransitionAnimations = false;

            radChart1.DefaultView.ChartArea.DataSeries.Clear();
            radChart1.SeriesMappings.Clear();
            radChart1.SeriesMappings = new SeriesMappingCollection();
            for (int i = 0; i < semnale.Count; i++)
            {
                SeriesMapping temperatureMapping = new SeriesMapping();

                LineSeriesDefinition lineDefinition = new LineSeriesDefinition();// LineSeriesDefinition();
                lineDefinition.ShowItemLabels = false;
                lineDefinition.ShowItemToolTips = true;
                lineDefinition.ShowPointMarks = false;
                lineDefinition.InteractivitySettings.HoverScope = InteractivityScope.Series;
                lineDefinition.InteractivitySettings.SelectionScope = InteractivityScope.Series;
                lineDefinition.AxisName = "Axis"+i;



                temperatureMapping.SeriesDefinition = lineDefinition;
                temperatureMapping.ItemMappings.Add(new ItemMapping("point_id", DataPointMember.XCategory));
                temperatureMapping.ItemMappings.Add(new ItemMapping("value" + (i + 1), DataPointMember.Tooltip));

                //temperatureMapping.GroupingSettings.GroupDescriptors.Add(new ChartGroupDescriptor("point_id"));
                //temperatureMapping.GroupingSettings.ShouldCreateSeriesForLastGroup = true;

                temperatureMapping.ItemMappings.Add(new ItemMapping("timestamp", DataPointMember.XValue));
                //temperatureMapping.ItemMappings.Add(new ItemMapping("value" + (i + 1), DataPointMember.YValue));
                ItemMapping yItemMapping = new ItemMapping("value" + (i + 1), DataPointMember.YValue);
                yItemMapping.SamplingFunction = ChartSamplingFunction.KeepExtremes;
                temperatureMapping.ItemMappings.Add(yItemMapping);
                radChart1.SeriesMappings.Add(temperatureMapping);



                radChart1.DefaultView.ChartArea.LegendName = "LegendaComuna";

                Brush brush = new SolidColorBrush(Culori[i]);

                radChart1.DefaultView.ChartArea.PaletteBrushes.Add(brush);

                ChartLegendItem item1 = new ChartLegendItem();
                item1.Label = semnale[i].Signal_Name + " - " + semnale[i].Description;

                item1.MarkerFill = new SolidColorBrush(Culori[i]);
                radChart1.DefaultView.ChartArea.Legend.Items.Add(item1);
            }

            radChart1.DefaultView.ChartArea.ItemToolTipOpening += new ItemToolTipEventHandler(ChartArea1_ItemToolTipOpening);


            //list = e.Result;
             radChart1.ItemsSource = e.Result;
            // object myObject = radChart1.DefaultView.GetType().GetField("MergedLegendItemsCollection", BindingFlags.Instance | BindingFlags.NonPublic).GetValue(radChart1.DefaultView);
             label6.Visibility = Visibility.Collapsed;
             this.busyIndicator.IsBusy = false;



        }

I attached you some pictures again, with the report on the set of data i gave you in the csv from the link.
The "Without zoom. jpg" is the initial report without zoom. It is perfect. Look at the interval on the x axis.
The WithZoom.jpg contains the report after i zoomed in. Observe the x axes again. It has no sense whatsover.
The first axis has an interval on 3rd of January, and after the zoom i have 3 equal values of 30 of december.
I have no ideea from where that value appeared.
Thank you again,

Dan

[Ves]

Hello Dan,

Thanks for the additional details. I reviewed your code and I found that there are multiple ItemMappings added to a single SeriesMapping. While it is possible and expected to have more than one ItemMapping in a single SeriesMapping, note that there should be a single ItemMapping for a DataPointMember. Here is the flow (sampling ignored for simplicity): For each item in the underlying data a DataPoint is created. Then for each ItemMapping a datapoint member is populated. Hence, it is expected that there is a single ItemMapping for YValue, so that the DataPoint.YValue is populated with the value from the corresponding property.

Another note here -- it is expected that a DataPoint is populated with either XValue of XCategory, that is -- datapoints are either placed next to each other along the X axis (XCategory) or at a certain position (XValue). Your code shows adding multiple ItemMappings for YValue as well as ItemMappings for both XValue and XCategory. When you need more than one series, you can add more SeriesMappings, but do use the same X-axis related DataPointMember for all of them, that is either XValue or XCategory.

In addition, zooming in RadChart is performed by filtering out the data, which is not shown. This might lead to  unexpected result if you happen to zoom in too much, like zooming so that there is no datapoint within the zoom range. For this purpose, you can use the ZoomScrollSettings.MinZoomRange property to limit the zoom in range. You can find this explained in more details in our Zooming And Scrolling help topic.

Best regards,
Ves
the Telerik team

[Dan]

Hello,

Thank you for your answer.
I did not add multiple itemmappings to Yvalue. The line was commented. Indeed i did add itemmappings on xcategory and xvalue.
The xcategory remained there , from when i used groupdescriptors to group by that category (it seems that i misunderstood, and xcategory has nothing to do with grouping) ... After i deleted that line now, the zoom works when we have one single series on the chart.
The mappings i use are:

 temperatureMapping.ItemMappings.Add(new ItemMapping("value" + (i + 1), DataPointMember.Tooltip));
 temperatureMapping.ItemMappings.Add(new ItemMapping("timestamp", DataPointMember.XValue));
 ItemMapping yItemMapping = new ItemMapping("value" + (i+1 ), DataPointMember.YValue);

But we have one more problem. If we have 2 ore more series on the chart, on a level of zoom the data shown is not good.
I attached a picture as an example.
Have you any idea, why it could do that?

Thank you again,

Dan

[Dan]

Hello again,

After other tests, we discovered that the problem has nothing to do with zoom or sampling. There are random time intervals that have the same behavior. Our datasource for ItemSource is a list of:

public class ObjRaport
    {
        public DateTime timestamp;
        public string point_id;
        public double? value1;
        public double? value2;
        public double? value3;
        public double? value4;
        public double? value5;
        public double? value6;
        public double? value7;
        public double? value8;
        public double? value9;
        public double? value10;
    }

We used this structure because we would like our reports to have a selectable number of dataseries.
If we have only one data series all goes well.
If we have at least two the bind does not work correctly. 
For example for 2 dataseries we will have a list of ObjReport. For the first series the list will contain objects with only value1 not null.
For the second series the list will have objects with only value2 not null.

for (int i = 0; i < semnale.Count; i++)
           {
               temperatureMapping = new SeriesMapping();
               lineDefinition = new LineSeriesDefinition();// LineSeriesDefinition();
               lineDefinition.ShowItemLabels = false;
               lineDefinition.ShowItemToolTips = true;
               lineDefinition.ShowPointMarks = false;
               lineDefinition.InteractivitySettings.HoverScope = InteractivityScope.Series;
               lineDefinition.InteractivitySettings.SelectionScope = InteractivityScope.Series;
               lineDefinition.AxisName = "Axis"+i;



               temperatureMapping.SeriesDefinition = lineDefinition;

               temperatureMapping.ItemMappings.Add(new ItemMapping("value" + (i + 1), DataPointMember.Tooltip));

               temperatureMapping.ItemMappings.Add(new ItemMapping("timestamp", DataPointMember.XValue));


                yItemMapping = new ItemMapping("value" + (i + 1), DataPointMember.YValue);

               yItemMapping.SamplingFunction = ChartSamplingFunction.KeepExtremes;
               temperatureMapping.ItemMappings.Add(yItemMapping);
               radChart1.SeriesMappings.Add(temperatureMapping);

               radChart1.DefaultView.ChartArea.LegendName = "LegendaComuna";

               Brush brush = new SolidColorBrush(Culori[i]);

               radChart1.DefaultView.ChartArea.PaletteBrushes.Add(brush);

               ChartLegendItem item1 = new ChartLegendItem();
               item1.Label = semnale[i].Signal_Name + " - " + semnale[i].Description;

               item1.MarkerFill = new SolidColorBrush(Culori[i]);
               radChart1.DefaultView.ChartArea.Legend.Items.Add(item1);
           }

After we debugged, we have seen that the items  of series1 from radchart1, indeed have null YValues. The ObjReport attached to the item had value1 not null . So the databind did not work well.

Could this be a bug, or did i misunderstood something?
Edit1:

I have the feeling that the binding is not working as it should. Even-though i specifically say "value"+(i+1) the item does not look for value1 from ObjReport (when i =0), and it takes somethin random from the other values that are null. I replaced null with -10 for the values that should not contain data, and the spikes go now to -10 instead of 0.

Edit2:

I did another test.
I replaced the nulls with -50000 then after the itemsource i removed those values from the DataSeries:

radChart1.ItemsSource = e.Result;
           for (int i = 0; i < radChart1.DefaultView.ChartArea.DataSeries.Count; i++)
           {
               foreach (DataPoint dt in radChart1.DefaultView.ChartArea.DataSeries[i].ToArray())
               {
                   if (dt.YValue == -50000 )
                   {
                       radChart1.DefaultView.ChartArea.DataSeries[i].Remove(dt);
                   }
               }
           }

Now, i can have more than one dataseries. I can make reports on short intervals (or any other interval in contrast to before) and all goes well.

The problem is that on the first zoom the chart goes nuts again like in the picture attached.



Thank you again,

Dan  

[Dan]

Hello,

I think i resolved the problem with a list of ObservableCollection.
I don`t know why i did not try this before.
Anyhow, the problem still persists on the approach i described above. It may be a bug of radchart.

I will test it further.

Thank you,

Dan

[Mark Woodlief]

I can almost gaurantee your issue lies with the ? nullable types.  I had the exact same issue, my data t ype was DateTime?  Telerik doesnt play well with nullable types in my experience.