kendo.guid memory leak with big data

[Bridge24]

We are using kendo with very large dataset.

We had issue with memory overflow with some data, and chrome crashed.

After investigation, we found that the "kendo.guid" function is the cause.

That function is called when creating observable data I think.

The memory leak is caused by the "+=" technique used to create the guid, as each += create a copy of the previous value. 

It performs very well, but each loop from 1 to 32 is causing that trouble.  And we sometimes raise the chrome RAM to 2.5 GB !!!

We changed the string to an array, and uses the "push" instead of +=.  Then, return that array.join(""). And it works perfectly, no more memory issue, with the same cpu performance and speed.

The situation where we got 2.5GB of ram used is now limited to 500MB, and is quickly going down to 100MB after the execution.

You can see the edited version there.

http://dojo.telerik.com/@foxontherock/EGEziVon/13

Just remove-add the underscore prefix to test it, hit Ctrl-Enter multiple times to repeat it, and use the "chrome task manager" to see the "javascript memory" used (you need to add that column).  You can also use the "trashcan" of the memory tab in chrome dev tools to force a garbage collection.

The edited version (guid1) is based on the "minified" version of kendo.web.js.

 

[Bridge24]

I see that there's no "admin" reply on this, you may think that I post the same bug 2 times related to "big data"... but these are 2 different bugs.

1. CPU and wait-time issue in the .toJSON (the other post that you replied)

2. high memory usage in kendo.guid function. (this post)

I invite you to look at both.

[Veselin Tsvetanov]

Hello Daniel,

Thank you for the sample prepared and for the detailed explanation of the case.

You are right that in scenarios dealing with large mount of data the string concatenation (+=) would result in considerable amount of memory allocated for the operation. In such scenario, with each loop a completely new memory block will be allocated for the new string, while the previously allocated block will still remain in use. In this respect, the array.push() + array.join() approach would consume much less memory for the same operation. The concatenation, however, was chosen mainly due to its slightly faster performance.

Having that said, we will discuss the case and the issue faced within the development team and we will consider the options for improving the kendo.guid() function performance. As a small token of gratitude for suggesting this alternative approach, I have updated your Telerik points.

Here you could find a GitHub issue, where you could track the status of the discussed enhancement.

Regards,
Veselin Tsvetanov
Progress Telerik

[Bridge24]

You are right, when combining "big" strings, += performs better than array.push, as your sample shows.

But in our case, we loop small string of 32 characters, several times.  The case is not exactly the same.

So, I updated your jsperf test, but exactly with the code I propose to replace in kendo.

When repeating 50000 times (like if I have 50000 observableObject to manage), the performance is the same.

So, because the memory is way better, I still suggest to replace += with push in the guid function.

You can look at this sample, that is "exactly" our case.

https://jsperf.com/yet-another-array-vs-concat/7

 

 

[Veselin Tsvetanov]

Hello Daniel,

Thank you for the sample prepared, where the implementation of the kendo.guid() function could be properly tested.

We have also performed performance tests in the other major browsers as well as the memory test already discussed. As the Array.push() implementations demonstrates comparable, or even slightly better performance, it is highly probable that we will proceed with the suggested improvement. Keep an eye on the GitHub item, which will be updated accordingly and many thanks for your help!

Regards,
Veselin Tsvetanov
Progress Telerik