ZipInputStream

19 posts, 0 answers
  1. Jhonathan
    Jhonathan avatar
    2 posts
    Member since:
    Jul 2011

    Posted 26 Jul 2011 Link to this post

    Hi,

    I am trying to replace my current zip library with Telerik's one.

    My goal is to improve the transfer of huge objects between SL and WCF.
    I can sucessfully zip my serialized object as in (note that I am using IsolatedStorage instead of memory stream as the objects some times are so big that using memory stream would throw an Out of Memory Exception):
    public static byte[] SerializeAndCompress(T obj)
    {
        var filename = DateTime.Now.ToString("ddMMYYYYhhmmss") + ".xml";
      
        var appStore = IsolatedStorageFile.GetUserStoreForApplication();
      
        if (appStore.Quota < TamanhoStorage) appStore.IncreaseQuotaTo(TamanhoStorage);
      
        byte[] bobj;
      
        using (var fileStream = appStore.OpenFile(filename, FileMode.Create))
        {
            var serializer = new DataContractSerializer(typeof(T));
            serializer.WriteObject(fileStream, obj);
      
            fileStream.Position = 0;
      
            bobj = new byte[fileStream.Length];
            fileStream.Read(bobj, 0, (int)fileStream.Length);
        }
      
        using (var fileStream = appStore.OpenFile(filename, FileMode.Create))
        {
            using (var zip = new ZipOutputStream(fileStream))
            {
                zip.Write(bobj, 0, bobj.Length);
            }
        }
      
        byte[] cobj = null;
      
        using (var fileStream = appStore.OpenFile(filename, FileMode.Open))
        {
            cobj = new byte[fileStream.Length];
            fileStream.Read(cobj, 0, cobj.Length);
        }
      
        appStore.DeleteFile(filename);
      
        return cobj;
    }

    This will compress my object as intended. My problem is reverting this process.
    The only way I was able to achieve this was using a stream reader's ReadToEnd Method, but that won't work, as it will return a string, and I need a byte array (I know I can convert the string back to a byte array, but I'd like to make it in a better way, as in filestream.Read). Code sample:

    using (var fileStream = appStore.OpenFile(filename, FileMode.Open))
        {
            using (var zip = new ZipInputStream(fileStream))
            {
                using (var sr = new StreamReader(zip))
                {
                    var str = sr.ReadToEnd();
                }
            }
        }


    After I call the sr.ReadToEnd method I can see the zip properties filled correctly with the compressed and uncompressed size.
    What I need is the byte array of the decompressed object. Is there a way to achieve this? 

    I tryed the following approach but it didn't work, as I can't foresee the size of the uncompressed object and the decompressed byte array won't be correct:

    var bobj = new byte[5000000];
     
    using (var fileStream = appStore.OpenFile(filename, FileMode.Open))
    {
        using (var zip = new ZipInputStream(fileStream))
        {
            zip.Read(bobj, 0, (int)fileStream.Length);
     
        }
    }

    That will not read the decompressed object correctly, and will waste memory if the object is smaller than the byte array "bobj".

    Thanks in advance.

    Regards

    Take Care

    Jhon
  2. Viktor Tsvetkov
    Admin
    Viktor Tsvetkov avatar
    382 posts

    Posted 27 Jul 2011 Link to this post

    Hello Jhonathan,

    You can have a property in which you will store the length of the file stream before it is compressed via the ZipOutputStream and then you will simply pass it as a size of the byte array. If you have further questions feel free to ask.

    All the best,
    Viktor Tsvetkov
    the Telerik team

    Register for the Q2 2011 What's New Webinar Week. Mark your calendar for the week starting July 18th and book your seat for a walk through of all the exciting stuff we will ship with the new release!

  3. Jhonathan
    Jhonathan avatar
    2 posts
    Member since:
    Jul 2011

    Posted 27 Jul 2011 Link to this post

    Thanks for your response, but unfortunately it did not help me at all, as I will compress in one environment and decompress in another environment, in this case I'd have to send the uncompressed size, and in many cases that won't be possible.

    But, I've found a way to decompress my objects correctly, as follows:

    public static T DecompressAndDeserialize(byte[] cobj)
            {
                var filename = DateTime.Now.ToString("ddMMYYYYhhmmss") + ".xml";
     
                var appStore = IsolatedStorageFile.GetUserStoreForApplication();
     
                using (var fileStream = appStore.OpenFile(filename, FileMode.Create))
                {
                    fileStream.Write(cobj, 0, cobj.Length);
                }
     
                var bobjt = new List<byte>();
                var bufferSize = 1000000;
                byte[] bufObj = null;
     
                using (var fileStream = appStore.OpenFile(filename, FileMode.Open))
                {
                    using (var zip = new ZipInputStream(fileStream))
                    {
                        while (zip.CompressedSize < cobj.Length)
                        {
                            bufObj = new byte[bufferSize];
                            zip.Read(bufObj, 0, bufferSize);
     
                            bobjt.AddRange(bufObj);
                        }
                    }
                }
     
                var index = 0;
     
    //Finding the index of the last valid character starting backwards
                for (var i = bobjt.Count - 1; true; i--)
                {
                    if (bobjt[i] == 0x00) continue;
                     
                    index = i + 1;
                    break;
                }
     
                var bobj = new byte[index];
                Array.Copy(bobjt.ToArray(), bobj, index);
     
                T obj;
     
                using (var fileStream = appStore.OpenFile(filename, FileMode.Create))
                {
                    fileStream.Write(bobj, 0, bobj.Length);
     
                    fileStream.Position = 0;
     
                    var serializer = new DataContractSerializer(typeof(T));
                    obj = (T)serializer.ReadObject(fileStream);
                }
     
                appStore.DeleteFile(filename);
                return obj;
            }

    This will correctly decompresse the object.

    But I've found another issue in the code, that I am not sure if it is related with the ZipLibrary itself or due to the approach I'm using. Consider the following code:

    using (var fileStream = appStore.OpenFile(filename, FileMode.Create))
                {
                    using (var zip = new ZipOutputStream(fileStream))
                    {
                        zip.Write(bobj, 0, bobj.Length);
                    }
                }

    I succefully zipped many different objects  using this approach, zipping objects of 5MB into 365KB arrays. But when I try to compres objects of bigger size, I get an error: "IndexOutOfRangeException" - Index was outside the bounds of the array. The object I am trying to compress is 75MB large. The error is thrown in the zip.write method.

    Stack:
       em Telerik.Windows.Zip.Tree.Code(Int32 dist)
       em Telerik.Windows.Zip.DeflateManager.CompressBlock(Int16[] ltree, Int16[] dtree)
       em Telerik.Windows.Zip.DeflateManager.FlushBlock(Int32 buf, Int32 stored_len, Boolean eof)
       em Telerik.Windows.Zip.DeflateManager.FlushBlockOnly(Boolean eof)
       em Telerik.Windows.Zip.DeflateManager.ProcessSlowDeflate(Int32 flush, Int32& num)
       em Telerik.Windows.Zip.DeflateManager.SlowDeflate(Int32 flush)
       em Telerik.Windows.Zip.DeflateManager.Deflate(ZipBaseStream strm, Int32 flush)
       em Telerik.Windows.Zip.ZipBaseStream.Deflate(Int32 flush)
       em Telerik.Windows.Zip.ZipOutputStream.WriteInternal(Byte[] buf, Int32 offset, Int32 count)
       em Telerik.Windows.Zip.ZipOutputStream.Write(Byte[] buffer, Int32 offset, Int32 count)
       em CCM7.Web.Silverlight.Commom.WFM.Utils.SerializadorDCS`1.SerializarEComprimir(T obj)
       em CCM7.Web.Silverlight.Planejamento.WFM.Paginas.PrevisaoIntraAno.BtnAtualizarClick(Object sender, RoutedEventArgs e)
       em System.Windows.Controls.Primitives.ButtonBase.OnClick()
       em System.Windows.Controls.Button.OnClick()
       em Telerik.Windows.Controls.RadButton.OnClick()
       em System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(MouseButtonEventArgs e)
       em System.Windows.Controls.Control.OnMouseLeftButtonUp(Control ctrl, EventArgs e)
       em MS.Internal.JoltHelper.FireEvent(IntPtr unmanagedObj, IntPtr unmanagedObjArgs, Int32 argsTypeIndex, Int32 actualArgsTypeIndex, String eventName)

    Any ideas how to correct this error? is there another approach to zip the stream?

    Thanks in advance.

    Regards

    Take Care

    Jhon
  4. Viktor Tsvetkov
    Admin
    Viktor Tsvetkov avatar
    382 posts

    Posted 27 Jul 2011 Link to this post

    Hello Jhonathan,

    Actually in your scenario it is better and easier if you take advantage of the ZipPackage class, so could you please send me a sample project reproducing your issue, so I will be able to rewrite it using the ZipPackage?

    Greetings,
    Viktor Tsvetkov
    the Telerik team

    Register for the Q2 2011 What's New Webinar Week. Mark your calendar for the week starting July 18th and book your seat for a walk through of all the exciting stuff we will ship with the new release!

  5. Zlatko
    Zlatko avatar
    22 posts
    Member since:
    Sep 2013

    Posted 12 Dec 2016 Link to this post

    Please review your Constructor:

    public ZipInputStream(Stream baseStream)       : base(baseStream, StreamOperationMode.Read, (CompressionSettings) new DeflateSettings(), false, (EncryptionSettings) null)     {     }

     

    My code is:

    ----------------------------------------------------------

    using (var memoryStream = new MemoryStream(content))

    {
    try
    {
    var zipInputStream = new ZipInputStream(memoryStream);
    using (var streamReader = new StreamReader(zipInputStream, new UTF8Encoding()))
    {
    result = streamReader.ReadToEnd();
    }
    }
    catch (Exception ex)
    {
    errorMessage = ex.Message;
    }
    }

    --------------------------------------------------

     

    Content is byte array from UNCOMPRESSED data.

    I get Exception on new ZipInputStream(memoryStream);

    try - catch catches the Exception with message: Unknown compression method (0x3C). My code is continuing to work fine, but after 2-3 sec. the program crash with message:

    "Value cannot be null.
    Parameter name: buffer

    System.ArgumentNullException was unhandled
      HResult=-2147467261
      Message=Value cannot be null.
    Parameter name: buffer
      Source=Telerik.Windows.Zip
      ParamName=buffer
      StackTrace:
           at Telerik.Windows.Zip.OperationStream.ValidateBufferParameters(Byte[] buffer, Int32 offset, Int32 count, Boolean allowZeroCount)
           at Telerik.Windows.Zip.BlockTransformBase.ValidateInputBufferParameters(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount, Boolean validateBlockSize, Boolean allowZeroCount)
           at Telerik.Windows.Zip.CompressionTransformBase.TransformFinalBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount)
           at Telerik.Windows.Zip.OperationStream.FlushFinalBlock()
           at Telerik.Windows.Zip.CompressedStream.Dispose(Boolean disposing)
           at Telerik.Windows.Zip.OperationStream.Finalize()
      InnerException: 

    Or please, tell me the right way to check if my content is uncompressed

  6. Martin Ivanov
    Admin
    Martin Ivanov avatar
    1442 posts

    Posted 15 Dec 2016 Link to this post

    Hi Zlatko,

    There is a feature request about exposing an API that allows checking if the content is a valid zip. You can find it logged in our feedback portal where you can track its status and vote for an implementation. I also updated your Telerik points.

    You can check the following StackOverflow question to see possible approaches for checking if the file is valid zip.

    Regards,
    Martin
    Telerik by Progress
    Try our brand new, jQuery-free Angular 2 components built from ground-up which deliver the business app essential building blocks - a grid component, data visualization (charts) and form elements.
  7. Zlatko
    Zlatko avatar
    22 posts
    Member since:
    Sep 2013

    Posted 15 Dec 2016 in reply to Martin Ivanov Link to this post

    Telerik ZIP content begins with: 78 DA
    Please fix the issue with exception in constructor ZipInputStream

  8. Zlatko
    Zlatko avatar
    22 posts
    Member since:
    Sep 2013

    Posted 19 Dec 2016 Link to this post

    I still waiting for fix...
  9. Boby
    Admin
    Boby avatar
    636 posts

    Posted 20 Dec 2016 Link to this post

    Hi Zlatko,

    I've just created zip file with the sample code from this help article, and the file actually starts with 50-4B-03-04, which means that you can use the sample code from the mentioned SO thread, for example from this answer, to detect the file by its first four bytes. 

    By the way, ZipInputStream class in now obsolete, and the suggested class to use instead is CompressedStream. I've just tried the following sample code and it's working as expected:
    using (var memoryStream = new MemoryStream())
    {
        using (var writeCompressedStream = new CompressedStream(memoryStream, StreamOperationMode.Write, new DeflateSettings()))
        using (var inputFileStream = new FileStream(@"d:\temp\test.txt", FileMode.Open))
        {
            inputFileStream.CopyTo(writeCompressedStream);
            writeCompressedStream.Flush();
        }
     
        memoryStream.Position = 0L;
     
        using (var readCompressedStream = new CompressedStream(memoryStream, StreamOperationMode.Read, new DeflateSettings()))
        using (var outputFileStream = new FileStream(@"d:\temp\test1.txt", FileMode.Create))
        {
            readCompressedStream.CopyTo(outputFileStream);
        }
    }

    The text file is successfully compressed in a stream, and subsequently decompressed. 

    That said, there is difference between a compressed stream and a zip file. The code from SO will check for full zip files, while the compressed streams are only streams compressed with the particular decoded - they do not start with a particular byte sequence (or at least it's different).

    Regards,
    Boby
    Telerik by Progress
    Try our brand new, jQuery-free Angular 2 components built from ground-up which deliver the business app essential building blocks - a grid component, data visualization (charts) and form elements.
  10. Zlatko
    Zlatko avatar
    22 posts
    Member since:
    Sep 2013

    Posted 21 Dec 2016 in reply to Boby Link to this post

    You are not understand me.

    You have problem inside the constructor ZipInputStream but not throw exception. 

    Application crash when GC try to clear your byte[] buffer.

    How I can talk with some of Telerik about that via phone?

  11. Zlatko
    Zlatko avatar
    22 posts
    Member since:
    Sep 2013

    Posted 21 Dec 2016 in reply to Zlatko Link to this post

    There is my code:

    ----------------

    public static string ZipDecompress(byte[] content)
            {
                string result = null;
                using (var memoryStream = new MemoryStream(content))
                {
                    try
                    {
                        Stream zipInputStream = new ZipInputStream(memoryStream); // <-- Here is the problem
                        using (var streamReader = new StreamReader(zipInputStream, new UTF8Encoding()))
                            { result = streamReader.ReadToEnd(); }
                    }
                    catch (Exception ex){}
                }
                return result;
            }

    ----------------------

    There byte[] content is array created from string "Hello World" (Notice, this is uncompressed byte array, if it is compressed all is fine)

    Exception is raised, and I catch him.

    But after 2-3 sec. I've got the application crash with Error like: "buffer" is already closed. Can't close twice.

    Please test this method!

  12. Petar Mladenov
    Admin
    Petar Mladenov avatar
    2991 posts

    Posted 22 Dec 2016 Link to this post

    Hi Zlatko,

    We used your code and tried to replicate the "buffer already closed" issue. We receive our exception when processing the header but we cannot reproduce any other exceptions. We tried forcing the GC if it is related. Please check out our test project. We would be grateful if you can edit it so that the exception is reproducible  or send us other isolation that we can debug locally.

    As for the phone assistance, it is available only with DevCraft Ultimate license, you could check our Support Plans for more details: http://www.telerik.com/purchase/support-plans/devtools


    Regards,
    Petar Mladenov
    Telerik by Progress
    Try our brand new, jQuery-free Angular 2 components built from ground-up which deliver the business app essential building blocks - a grid component, data visualization (charts) and form elements.
  13. Zlatko
    Zlatko avatar
    22 posts
    Member since:
    Sep 2013

    Posted 23 Dec 2016 in reply to Petar Mladenov Link to this post

    Hi Petar,

    Thank you for your Sample project. You used Zip_WPF as reference, but I not have this name and I replaced with Telerik.Windows.Zip. Only this is changes in your Sample project.

    Here is two video results from my computer: 

    https://www.youtube.com/watch?v=s1D47igxwYA

    https://www.youtube.com/watch?v=U4viNedIQRk

    I told you, the problem is in your Telerik.Windows.Zip library.

    I still waiting for fix...

  14. Petar Mladenov
    Admin
    Petar Mladenov avatar
    2991 posts

    Posted 23 Dec 2016 Link to this post

    Hi Zlatko,

    We would be glad to log an issue and plan it for future releases. However,  we are still unable to reproduce it on our side. When we close the application (like in your first) or when we build and run the exe and press the button (like in your second video) we have no such exception like yours. We are testing with R3 2016 version of our controls. Could you please let us know your:

    - version of Telerik UI Controls for WPF
            - VS version (with the exact upgrade)
            - .Net installed version on your machine

    This will better help us in reproducing the exception. Thank you in advance for your cooperation.

    Regards,
    Petar Mladenov
    Telerik by Progress
    Try our brand new, jQuery-free Angular 2 components built from ground-up which deliver the business app essential building blocks - a grid component, data visualization (charts) and form elements.
  15. Zlatko
    Zlatko avatar
    22 posts
    Member since:
    Sep 2013

    Posted 23 Dec 2016 in reply to Petar Mladenov Link to this post

    Telerik UI for WPF Q2 2014 - Telerik.Windows.Zip.dll ver.2014.2.729.40

    MS VS Professional 2012 - ver. 11.0.61219.00 Update 5

    Telerik UI for WPF VSExtensions - 2016.5.1111.0

    .Net 4.0

  16. Petar Mladenov
    Admin
    Petar Mladenov avatar
    2991 posts

    Posted 28 Dec 2016 Link to this post

    Hello Zlatko,

    We tested the project with your version (Q2 2014 SP 7.29) and we managed to reproduce the exception.
    The good news is that with the next version onward (Q3 2014 ->) the exception is fixed. We also found the original bug we have fixed whose name is:

    "Using compressed stream to open corrupted zip archive causes exception after the browser is closed."

    The code for the fix is located in private and internal methods so the only option we can suggest for you is upgrade to our latest release R3 2016 or wait for R1 2017 in mid January.


    Regards,
    Petar Mladenov
    Telerik by Progress
    Try our brand new, jQuery-free Angular 2 components built from ground-up which deliver the business app essential building blocks - a grid component, data visualization (charts) and form elements.
  17. Zlatko
    Zlatko avatar
    22 posts
    Member since:
    Sep 2013

    Posted 28 Dec 2016 in reply to Petar Mladenov Link to this post

    - "Thank you, Zlatko, for the bug report!"

    - Welcome... 

     

    Unfortunately, we have license only for Q2 2014. All next version will work for us as trial.

    I think, is good these bug-fixes to be available for free for all previously licensed customers. Because this is bug-fix, not is "new feature" or improvement.

    I'll be happy if you make update for Q2 2014

    Thanks,

    I still waiting for fix.

    Merry Christmas! 

  18. Stefan
    Admin
    Stefan avatar
    2906 posts

    Posted 29 Dec 2016 Link to this post

    Hello Zlatko,

    We do not have a practice to reissue releases with new functionalities of fixes. All fixes and new functionalities are being issued with new releases, so the only way for you to benefit from such is to upgrade to new version.

    Thank you for the understanding.

    Regards,
    Stefan
    Telerik by Progress
    Try our brand new, jQuery-free Angular 2 components built from ground-up which deliver the business app essential building blocks - a grid component, data visualization (charts) and form elements.
  19. Zlatko
    Zlatko avatar
    22 posts
    Member since:
    Sep 2013

    Posted 29 Dec 2016 in reply to Stefan Link to this post

    Now I really sorry for our choice - Telerik platform.

    DevExpress always make fixes for previous releases for free.

    Anyway, thanks

Back to Top