Package.AddStream() : OutOfMemory Exception

9 posts, 0 answers
  1. favand
    favand avatar
    8 posts
    Member since:
    Aug 2011

    Posted 29 May 2013 Link to this post

    Hi.

    I have recently started using the ZipLibrary and I am facing an Issue when adding streams to a ZipPackage.

    Here is my code below, aswell as the stack trace. Is there anything wrong with it or is it an issue from the ZipLibrary ?

    MemoryStream memStream = new MemoryStream();
    ZipPackage Package = ZipPackage.Create(memStream);
    foreach (MyItem item in myCollection) {
        Stream stream = default(Stream);
        byte[] bData = GetPdfBytes(item);
        //Average size is {Length=3150000}
        stream = new MemoryStream(bData);
        Package.AddStream(stream, string.Format("MyFileName_{0}.pdf", item.UniqueID), Telerik.Web.Zip.CompressionType.Default, item.Date);
        //Out of memory exception here
    //offset: 65855158, Headers: 23, ZipPackageEntries: 23             
    }
     
    //Code from Telerik Samples
    SendZipToClient(memStream, Package);

    Stack Trace: 

    at System.IO.MemoryStream.set_Capacity(Int32 value)
    at System.IO.MemoryStream.EnsureCapacity(Int32 value)
    at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
    at Telerik.Web.Zip.ZipOutputStream.StreamCopy(Stream dstStream, Stream srcStream, UInt32 len)
    at Telerik.Web.Zip.ZipOutputStream.WriteCompressedData(ZipPackageEntry item)
    at Telerik.Web.Zip.ZipOutputStream.AddCompressed()
    at Telerik.Web.Zip.ZipOutputStream.Close(Boolean shouldCloseStream)
    at Telerik.Web.Zip.ZipOutputStream.Close()
    at Telerik.Web.Zip.ZipPackage.AddEntry(ZipCompression method, Stream recordStream, String fileNameInZip, DateTime dateTime, CompressionType compressionType)
    at Telerik.Web.Zip.ZipPackage.AddStream(Stream stream, String fileNameInZip, ZipCompression method, DateTime dateTime)
    at MyCode.DownloadZip_Click() 

    Additionnal Infos:
    .NET version: 4.5
    Telerik version for ASP.NET AJAX: 2013.1.417.45
  2. Radoslav
    Admin
    Radoslav avatar
    1564 posts

    Posted 03 Jun 2013 Link to this post

    Hi Favand,

    Thank you for contacting us.

    Please note that there is currently a 2GB limit for all CLR objects, no matter what OS you are running on 32 / 64 bit. There is also a practical limit, dependent on the current system conditions, (memory fragmentation) that limits the growth to the largest continuous block of memory available. Could you please verify that you do not create larger that 2BG CLR object? You can check the memory of the web server process and see if the consumed memory is bigger than 2GB.
    Additionally please note that in order to send a large file to the ASPNET Response stream you can not use the default implementation of SendZipToClient method because it sends the whole stream. Instead you can try using the following code snippet:
    private void SendZipToClient(MemoryStream memStream, ZipPackage Package)
    {
                Package.Close(false);
                memStream.Position = 0;
                if (memStream != null && memStream.Length > 0)
                {
                    Response.Clear();
                    Response.AddHeader("content-disposition", "attachment; filename=photos.zip");
                    Response.ContentType = "application/zip";
                    Response.BufferOutput = false;   // to prevent buffering
                    byte[] buffer = new byte[1024];
                    int bytesRead = 0;
                    while ((bytesRead = memStream.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        Response.OutputStream.Write(buffer, 0, bytesRead);
                    }
     
                    Response.End();
                }
     }

    Looking forward for your reply.

    Regards,
    Radoslav
    Telerik
    If you want to get updates on new releases, tips and tricks and sneak peeks at our product labs directly from the developers working on the RadControls for ASP.NET AJAX, subscribe to their blog feed now.
  3. favand
    favand avatar
    8 posts
    Member since:
    Aug 2011

    Posted 11 Jun 2013 Link to this post

    Hi Radoslav,

    Thank you for your response.

    I have checked the memory of the worker process it is way under 2GB (about 230 Mb, see attachment). There are also no CLR objects larger than 2GB.
    For additionnal information, the objects we create and add to the package stream are about 2 Mb - 3 Mb each, and the error occurs between 20-30 files (pretty inconsistent, even though our test uses the exact same files everytime) added to the package (which again, is lower than 2Gb).

    I have also attached a capture of the threads (sorry about the French environment). The error seem to occur within a thread, on a InternalWaitOne().

    Regards.

  4. Radoslav
    Admin
    Radoslav avatar
    1564 posts

    Posted 14 Jun 2013 Link to this post

    Hi Favand,

    It is hard to say what is causing the described issue without having a runnable example where the code can be debugged. Could you please send us a small runnable example? You could open a formal support ticket from your Telerik account and attach a ZIP file there. Thus we will be able to debug the project and provide you with more to-the-point answer.
    At a meantime could you please try adding different pdf files and let me know if the issue still appears or the problem is caused by a certain pdf file into the stream.

    Looking forward for your reply.

    Regards,
    Radoslav
    Telerik
    If you want to get updates on new releases, tips and tricks and sneak peeks at our product labs directly from the developers working on the RadControls for ASP.NET AJAX, subscribe to the blog feed now.
  5. Phil
    Phil avatar
    84 posts
    Member since:
    Apr 2005

    Posted 14 Apr 2015 in reply to Radoslav Link to this post

    That updated SendToZipClient doesn;t convert to valid vb using your own converter. Can you post the vb equivalent please?
  6. Radoslav
    Admin
    Radoslav avatar
    1564 posts

    Posted 15 Apr 2015 Link to this post

    Hello Phil,

    Here is the code VB:
    Private  Sub SendZipToClient(ByVal memStream As MemoryStream, ByVal Package As ZipPackage)
                Package.Close(False)
                memStream.Position = 0
                If memStream <> Nothing And memStream.Length > 0 Then
                    Response.Clear()
                    Response.AddHeader("content-disposition", "attachment; filename=photos.zip")
                    Response.ContentType = "application/zip"
                    Response.BufferOutput = False   ' to prevent buffering
                    Dim buffer() As Byte New Byte(1024) {}
                    Dim bytesRead As Integer =  0
                    While (bytesRead = memStream.Read(buffer,0,buffer.Length)) > 0
                        Response.OutputStream.Write(buffer, 0, bytesRead)
                    End While
      
                    Response.End()
                End If
    End Sub

    I hope this helps.

    Regards,
    Radoslav
    Telerik
     

    See What's Next in App Development. Register for TelerikNEXT.

     
  7. Phil
    Phil avatar
    84 posts
    Member since:
    Apr 2005

    Posted 16 Apr 2015 in reply to Radoslav Link to this post

    Thank you!
  8. Phil
    Phil avatar
    84 posts
    Member since:
    Apr 2005

    Posted 16 Apr 2015 Link to this post

    Sadly though, that code does not work. The while loop exits immediately, leaving buffer with 1025 bytes of data, bytesread = 0, and an empty pdf document.
  9. Phil
    Phil avatar
    84 posts
    Member since:
    Apr 2005

    Posted 16 Apr 2015 Link to this post

    here's what I ended up with that does work - note I am not passign zippackage as a parameter as in your exampel though.

    Private Sub SendZipToClient(memStream As MemoryStream, filename As String)

    memStream.Position = 0

    If memStream IsNot Nothing AndAlso memStream.Length > 0 Then

    Response.Clear()
    Response.AddHeader("content-disposition", "attachment; filename=" & filename)
    Response.ContentType = "application/zip"

    Response.BufferOutput = False

    'Response.BinaryWrite(memStream.ToArray())

    Dim buffer() As Byte = New Byte(1024) {}
    Dim bytesRead As Integer = 0

    bytesRead = memStream.Read(buffer, 0, buffer.Length)

    While bytesRead > 0
    Response.OutputStream.Write(buffer, 0, bytesRead)
    bytesRead = memStream.Read(buffer, 0, buffer.Length)
    End While

    memStream.Close()
    Response.End()

    End If

    End Sub

Back to Top