Cannot transfer large files with BufferSize set to 4 MB

[Helder Pinto]

Hi!

I am doing performance tests with RadUpload and I modified the BufferSize attribute of the RadUpload control to 4 MB. If I upload a file with a size up to around 2 MB, it's OK. However, if I try to upload a larger file (4, 10, 80, 600 MB, ...) the upload fails immediately and I see a "Handler not found or execution of the handler failed. The remote server returned an error: NotFound". 

I then repeated the test with BufferSize set to 3 MB: same problems. However, after changing BufferSize to 2 MB I could upload files of any size again.

Is there any chunk size limit at the RadUpload handler-side? If so, how can I change it? Thank you.

Setup: Silverlight 3 + Rad Controls for Silverlight 3 Trial + Windows XP/Vista + IE6/7/Chrome.

Regards,

Helder

[Miroslav]

Hi Helder,

There is no limit on our upload handler, but there possibly is a limit imposed by the web server. There could be a maxRequestLength set in its configuration that limits this.

This article describes a similar problem for the AJAX Upload:

http://www.telerik.com/support/kb/aspnet-ajax/upload/page-not-found-error-when-uploading-large-files-on-win2003.aspx

Can you check the configuration of your server?

All the best,
Miroslav
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.

[Helder Pinto]

Hi, Miroslav.

Thank you for your swift answer. It guided me to the resolution. However, it was not an IIS metabase problem, as described in the link you provided.

I discovered that the maxRequestLength attribute of the system.web/httpRuntime element of web.config is, by default, 4096 KB, i.e., 4194304 bytes (http://msdn.microsoft.com/en-us/library/e1f13641(VS.71).aspx). This should be enough for processing my 3100000 bytes chunk size, that I set at the BufferSize property of the RadUpload control. However, I don't know what RadUpload does behind the scenes, but what is actually sent is a much larger chunk. I ran Fiddler to analyze the client/server HTTP traffic and I discovered that, instead of something around 3100000 bytes, the first request carried 4281689 bytes, which is larger than the maximum request... This provoked an Internal Server Error (error 500) at the server-side. 

After more tests, I discovered that each request to the server carries always around 1 MB more than the buffer size. Is it by design or is it a bug?

Regards,

Helder

[Miroslav]

Hi Helder,

Thank you for sharing your findings.

This happens because of two reasons:

1. There is some extra information (headers) that are passed to the server that inflate the junk size.
2. The data is encoded in Base64 format and the chunk splitting happens before the transformation. Base64 encoding adds to the size of the transferred data.

I imagine that it will be hard to split the file in chunks so that with all its headers the final request will be within a certain size limit, so this behavior is by design.

Kind regards,
Miroslav
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.

[Helder Pinto]

Hi again, Miroslav.

Thank you for the explanation. All the behaviour I observed makes sense now.

I tested the RadUpload server-sidee handler performance with 40 parallel file uploads (size ranging between 30 and 600 MB, from 4 different client hosts), with three chunk size scenarios:

~97 KB (default): very stable regarding memory consumption and a bit processor hungry (avg 13%).

512 KB: same stability regarding memory consumption (though eating a bit more in avg) but hungrier regarding processor (avg. 19%)

4096 KB: the IIS application pool went crazy, with high memory consumption and huge processor time (28%); I don't recommend anyone to set such an upload chunk size. Furthermore, as the Silverlight client is encoding 4 MB before uploading data, the client host performance is also affected...

I suppose the base64 encoding/decoding has much to do with increasing processor time and my suspected lack of scalability. Can we switch off base64? Is it really needed in all scenarios? For example, if I only upload binary data (images, videos, etc.) is it really critical to base64 encode?

Thank you for your patience.

Regards,

Helder

[Valentin.Stoychev]

Hi Helder Pinto,

You are correct in your observations. We will try to address this for our Q3 release by providing an option the files to be sent directly with a POST request, not in chunks.

As for the base64 encoding - it is always turned "on" for the moment and it is not configurable. The reason for this is to enable the chunk file upload.

All the best,
Valentin.Stoychev
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.

[Helder Pinto]

Hi, again.

My team is about to start the integration of a file upload feature and I have two options on the table: Telerik's Upload RadControl and Codeplex's Silverlight File Upload project (http://silverlightfileupld.codeplex.com/).

As both solutions meet our client-side requirements, the decision is mainly influenced by server-side performance. I made comparative load tests with 40 simultaneous uploads of files ranging between 30 and 600 MB. As I stated in my previous post, the Upload RadControl server-side handler does an intensive usage of processor, although apparently scalable regarding memory consumption.

In contrast, Codeplex's project, which uses the same chunk-based approach, seems very scalable regarding processor usage (never more than 10%, while RadControl peaks often at 40/50%) and, though not as good as RadControl regarding memory consumption, the difference does not seem critical. I am about to choose the Codeplex's project for our upload solution, even if it never left beta version since January. I admit, however, that I may be forgetting something in the analysis I made. That's why I wrote this post, to hear your opinion/advice, in defense of Telerik's RadControl.

Thank you.

Helder Pinto

Ubisign

[Valentin.Stoychev]

Hello Helder Pinto,

Basically the upload handler can be customized and optimized for speed. you can even create your own server side handler and to optimize it for your needs.

You can check this help article:
http://www.telerik.com/help/silverlight/creating_custom_server-side_handler.html

As for the highets CPU usage when the  buffer size is 4Mb - we need to research where the problem is.

We have planned speed optimizations on the RadUpload for Q3 and will see how it can be improved.

Thank you for the feedback!

Kind regards,
Valentin.Stoychev
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.