This is a migrated thread and some comments may be shown as answers.

Package.AddStream() : OutOfMemory Exception

8 Answers 137 Views
ZipLibrary
This is a migrated thread and some comments may be shown as answers.
favand
Top achievements
Rank 1
favand asked on 29 May 2013, 01:34 PM
Hi.

I have recently started using the ZipLibrary and I am facing an Issue when adding streams to a ZipPackage.

Here is my code below, aswell as the stack trace. Is there anything wrong with it or is it an issue from the ZipLibrary ?

MemoryStream memStream = new MemoryStream();
ZipPackage Package = ZipPackage.Create(memStream);
foreach (MyItem item in myCollection) {
    Stream stream = default(Stream);
    byte[] bData = GetPdfBytes(item);
    //Average size is {Length=3150000}
    stream = new MemoryStream(bData);
    Package.AddStream(stream, string.Format("MyFileName_{0}.pdf", item.UniqueID), Telerik.Web.Zip.CompressionType.Default, item.Date);
    //Out of memory exception here
//offset: 65855158, Headers: 23, ZipPackageEntries: 23             
}
 
//Code from Telerik Samples
SendZipToClient(memStream, Package);

Stack Trace: 

at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at Telerik.Web.Zip.ZipOutputStream.StreamCopy(Stream dstStream, Stream srcStream, UInt32 len)
at Telerik.Web.Zip.ZipOutputStream.WriteCompressedData(ZipPackageEntry item)
at Telerik.Web.Zip.ZipOutputStream.AddCompressed()
at Telerik.Web.Zip.ZipOutputStream.Close(Boolean shouldCloseStream)
at Telerik.Web.Zip.ZipOutputStream.Close()
at Telerik.Web.Zip.ZipPackage.AddEntry(ZipCompression method, Stream recordStream, String fileNameInZip, DateTime dateTime, CompressionType compressionType)
at Telerik.Web.Zip.ZipPackage.AddStream(Stream stream, String fileNameInZip, ZipCompression method, DateTime dateTime)
at MyCode.DownloadZip_Click() 

Additionnal Infos:
.NET version: 4.5
Telerik version for ASP.NET AJAX: 2013.1.417.45

8 Answers, 1 is accepted

Sort by
0
Radoslav
Telerik team
answered on 03 Jun 2013, 07:17 AM
Hi Favand,

Thank you for contacting us.

Please note that there is currently a 2GB limit for all CLR objects, no matter what OS you are running on 32 / 64 bit. There is also a practical limit, dependent on the current system conditions, (memory fragmentation) that limits the growth to the largest continuous block of memory available. Could you please verify that you do not create larger that 2BG CLR object? You can check the memory of the web server process and see if the consumed memory is bigger than 2GB.
Additionally please note that in order to send a large file to the ASPNET Response stream you can not use the default implementation of SendZipToClient method because it sends the whole stream. Instead you can try using the following code snippet:
private void SendZipToClient(MemoryStream memStream, ZipPackage Package)
{
            Package.Close(false);
            memStream.Position = 0;
            if (memStream != null && memStream.Length > 0)
            {
                Response.Clear();
                Response.AddHeader("content-disposition", "attachment; filename=photos.zip");
                Response.ContentType = "application/zip";
                Response.BufferOutput = false;   // to prevent buffering
                byte[] buffer = new byte[1024];
                int bytesRead = 0;
                while ((bytesRead = memStream.Read(buffer, 0, buffer.Length)) > 0)
                {
                    Response.OutputStream.Write(buffer, 0, bytesRead);
                }
 
                Response.End();
            }
 }

Looking forward for your reply.

Regards,
Radoslav
Telerik
If you want to get updates on new releases, tips and tricks and sneak peeks at our product labs directly from the developers working on the RadControls for ASP.NET AJAX, subscribe to their blog feed now.
0
favand
Top achievements
Rank 1
answered on 11 Jun 2013, 02:45 PM
Hi Radoslav,

Thank you for your response.

I have checked the memory of the worker process it is way under 2GB (about 230 Mb, see attachment). There are also no CLR objects larger than 2GB.
For additionnal information, the objects we create and add to the package stream are about 2 Mb - 3 Mb each, and the error occurs between 20-30 files (pretty inconsistent, even though our test uses the exact same files everytime) added to the package (which again, is lower than 2Gb).

I have also attached a capture of the threads (sorry about the French environment). The error seem to occur within a thread, on a InternalWaitOne().

Regards.

0
Radoslav
Telerik team
answered on 14 Jun 2013, 06:49 AM
Hi Favand,

It is hard to say what is causing the described issue without having a runnable example where the code can be debugged. Could you please send us a small runnable example? You could open a formal support ticket from your Telerik account and attach a ZIP file there. Thus we will be able to debug the project and provide you with more to-the-point answer.
At a meantime could you please try adding different pdf files and let me know if the issue still appears or the problem is caused by a certain pdf file into the stream.

Looking forward for your reply.

Regards,
Radoslav
Telerik
If you want to get updates on new releases, tips and tricks and sneak peeks at our product labs directly from the developers working on the RadControls for ASP.NET AJAX, subscribe to the blog feed now.
0
Phil
Top achievements
Rank 2
answered on 14 Apr 2015, 10:08 PM
That updated SendToZipClient doesn;t convert to valid vb using your own converter. Can you post the vb equivalent please?
0
Radoslav
Telerik team
answered on 15 Apr 2015, 07:26 AM
Hello Phil,

Here is the code VB:
Private  Sub SendZipToClient(ByVal memStream As MemoryStream, ByVal Package As ZipPackage)
            Package.Close(False)
            memStream.Position = 0
            If memStream <> Nothing And memStream.Length > 0 Then
                Response.Clear()
                Response.AddHeader("content-disposition", "attachment; filename=photos.zip")
                Response.ContentType = "application/zip"
                Response.BufferOutput = False   ' to prevent buffering
                Dim buffer() As Byte New Byte(1024) {}
                Dim bytesRead As Integer =  0
                While (bytesRead = memStream.Read(buffer,0,buffer.Length)) > 0
                    Response.OutputStream.Write(buffer, 0, bytesRead)
                End While
  
                Response.End()
            End If
End Sub

I hope this helps.

Regards,
Radoslav
Telerik
 

See What's Next in App Development. Register for TelerikNEXT.

 
0
Phil
Top achievements
Rank 2
answered on 16 Apr 2015, 11:44 AM
Thank you!
0
Phil
Top achievements
Rank 2
answered on 16 Apr 2015, 12:10 PM
Sadly though, that code does not work. The while loop exits immediately, leaving buffer with 1025 bytes of data, bytesread = 0, and an empty pdf document.
0
Phil
Top achievements
Rank 2
answered on 16 Apr 2015, 12:17 PM

here's what I ended up with that does work - note I am not passign zippackage as a parameter as in your exampel though.

Private Sub SendZipToClient(memStream As MemoryStream, filename As String)

memStream.Position = 0

If memStream IsNot Nothing AndAlso memStream.Length > 0 Then

Response.Clear()
Response.AddHeader("content-disposition", "attachment; filename=" & filename)
Response.ContentType = "application/zip"

Response.BufferOutput = False

'Response.BinaryWrite(memStream.ToArray())

Dim buffer() As Byte = New Byte(1024) {}
Dim bytesRead As Integer = 0

bytesRead = memStream.Read(buffer, 0, buffer.Length)

While bytesRead > 0
Response.OutputStream.Write(buffer, 0, bytesRead)
bytesRead = memStream.Read(buffer, 0, buffer.Length)
End While

memStream.Close()
Response.End()

End If

End Sub

Tags
ZipLibrary
Asked by
favand
Top achievements
Rank 1
Answers by
Radoslav
Telerik team
favand
Top achievements
Rank 1
Phil
Top achievements
Rank 2
Share this question
or