This is a migrated thread and some comments may be shown as answers.

ZipInputStream

18 Answers 395 Views

ZipLibrary

This is a migrated thread and some comments may be shown as answers.

Jhonathan asked on 26 Jul 2011, 08:37 PM

Hi,

I am trying to replace my current zip library with Telerik's one.

My goal is to improve the transfer of huge objects between SL and WCF.
I can sucessfully zip my serialized object as in (note that I am using IsolatedStorage instead of memory stream as the objects some times are so big that using memory stream would throw an Out of Memory Exception):

public static byte[] SerializeAndCompress(T obj)

{

var filename = DateTime.Now.ToString("ddMMYYYYhhmmss") + ".xml";

var appStore = IsolatedStorageFile.GetUserStoreForApplication();

if (appStore.Quota < TamanhoStorage) appStore.IncreaseQuotaTo(TamanhoStorage);

byte[] bobj;

using (var fileStream = appStore.OpenFile(filename, FileMode.Create))

{

var serializer = new DataContractSerializer(typeof(T));

serializer.WriteObject(fileStream, obj);

fileStream.Position = 0;

bobj = new byte[fileStream.Length];

fileStream.Read(bobj, 0, (int)fileStream.Length);

}

using (var fileStream = appStore.OpenFile(filename, FileMode.Create))

{

using (var zip = new ZipOutputStream(fileStream))

{

zip.Write(bobj, 0, bobj.Length);

}

byte[] cobj = null;

using (var fileStream = appStore.OpenFile(filename, FileMode.Open))

{

cobj = new byte[fileStream.Length];

fileStream.Read(cobj, 0, cobj.Length);

}

appStore.DeleteFile(filename);

return cobj;

}

This will compress my object as intended. My problem is reverting this process.
The only way I was able to achieve this was using a stream reader's ReadToEnd Method, but that won't work, as it will return a string, and I need a byte array (I know I can convert the string back to a byte array, but I'd like to make it in a better way, as in filestream.Read). Code sample:

using (var fileStream = appStore.OpenFile(filename, FileMode.Open))

{

using (var zip = new ZipInputStream(fileStream))

{

using (var sr = new StreamReader(zip))

{

var str = sr.ReadToEnd();

}

After I call the sr.ReadToEnd method I can see the zip properties filled correctly with the compressed and uncompressed size.
What I need is the byte array of the decompressed object. Is there a way to achieve this?

I tryed the following approach but it didn't work, as I can't foresee the size of the uncompressed object and the decompressed byte array won't be correct:

var bobj = new byte[5000000];

using (var fileStream = appStore.OpenFile(filename, FileMode.Open))

{

using (var zip = new ZipInputStream(fileStream))

{

zip.Read(bobj, 0, (int)fileStream.Length);

}

That will not read the decompressed object correctly, and will waste memory if the object is smaller than the byte array "bobj".

Thanks in advance.

Regards

Take Care

Jhon

18 Answers, 1 is accepted

answered on 27 Jul 2011, 01:58 PM

Thanks for your response, but unfortunately it did not help me at all, as I will compress in one environment and decompress in another environment, in this case I'd have to send the uncompressed size, and in many cases that won't be possible.

But, I've found a way to decompress my objects correctly, as follows:

public static T DecompressAndDeserialize(byte[] cobj)

{

var filename = DateTime.Now.ToString("ddMMYYYYhhmmss") + ".xml";

var appStore = IsolatedStorageFile.GetUserStoreForApplication();

using (var fileStream = appStore.OpenFile(filename, FileMode.Create))

{

fileStream.Write(cobj, 0, cobj.Length);

}

var bobjt = new List<byte>();

var bufferSize = 1000000;

byte[] bufObj = null;

using (var fileStream = appStore.OpenFile(filename, FileMode.Open))

{

using (var zip = new ZipInputStream(fileStream))

{

while (zip.CompressedSize < cobj.Length)

{

bufObj = new byte[bufferSize];

zip.Read(bufObj, 0, bufferSize);

bobjt.AddRange(bufObj);

}

var index = 0;

//Finding the index of the last valid character starting backwards

for (var i = bobjt.Count - 1; true; i--)

{

if (bobjt[i] == 0x00) continue;

index = i + 1;

break;

}

var bobj = new byte[index];

Array.Copy(bobjt.ToArray(), bobj, index);

T obj;

using (var fileStream = appStore.OpenFile(filename, FileMode.Create))

{

fileStream.Write(bobj, 0, bobj.Length);

fileStream.Position = 0;

var serializer = new DataContractSerializer(typeof(T));

obj = (T)serializer.ReadObject(fileStream);

}

appStore.DeleteFile(filename);

return obj;

}

This will correctly decompresse the object.

But I've found another issue in the code, that I am not sure if it is related with the ZipLibrary itself or due to the approach I'm using. Consider the following code:

using (var fileStream = appStore.OpenFile(filename, FileMode.Create))

{

using (var zip = new ZipOutputStream(fileStream))

{

zip.Write(bobj, 0, bobj.Length);

}

I succefully zipped many different objects using this approach, zipping objects of 5MB into 365KB arrays. But when I try to compres objects of bigger size, I get an error: "IndexOutOfRangeException" - Index was outside the bounds of the array. The object I am trying to compress is 75MB large. The error is thrown in the zip.write method.

Stack:
em Telerik.Windows.Zip.Tree.Code(Int32 dist)
em Telerik.Windows.Zip.DeflateManager.CompressBlock(Int16[] ltree, Int16[] dtree)
em Telerik.Windows.Zip.DeflateManager.FlushBlock(Int32 buf, Int32 stored_len, Boolean eof)
em Telerik.Windows.Zip.DeflateManager.FlushBlockOnly(Boolean eof)
em Telerik.Windows.Zip.DeflateManager.ProcessSlowDeflate(Int32 flush, Int32& num)
em Telerik.Windows.Zip.DeflateManager.SlowDeflate(Int32 flush)
em Telerik.Windows.Zip.DeflateManager.Deflate(ZipBaseStream strm, Int32 flush)
em Telerik.Windows.Zip.ZipBaseStream.Deflate(Int32 flush)
em Telerik.Windows.Zip.ZipOutputStream.WriteInternal(Byte[] buf, Int32 offset, Int32 count)
em Telerik.Windows.Zip.ZipOutputStream.Write(Byte[] buffer, Int32 offset, Int32 count)
em CCM7.Web.Silverlight.Commom.WFM.Utils.SerializadorDCS`1.SerializarEComprimir(T obj)
em CCM7.Web.Silverlight.Planejamento.WFM.Paginas.PrevisaoIntraAno.BtnAtualizarClick(Object sender, RoutedEventArgs e)
em System.Windows.Controls.Primitives.ButtonBase.OnClick()
em System.Windows.Controls.Button.OnClick()
em Telerik.Windows.Controls.RadButton.OnClick()
em System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(MouseButtonEventArgs e)
em System.Windows.Controls.Control.OnMouseLeftButtonUp(Control ctrl, EventArgs e)
em MS.Internal.JoltHelper.FireEvent(IntPtr unmanagedObj, IntPtr unmanagedObjArgs, Int32 argsTypeIndex, Int32 actualArgsTypeIndex, String eventName)

Any ideas how to correct this error? is there another approach to zip the stream?

Thanks in advance.

Regards

Take Care

Jhon

answered on 12 Dec 2016, 10:09 AM

telerik-unzip.png

Please review your Constructor:

public ZipInputStream(Stream baseStream) : base(baseStream, StreamOperationMode.Read, (CompressionSettings) new DeflateSettings(), false, (EncryptionSettings) null) { }

My code is:

----------------------------------------------------------

using (var memoryStream = new MemoryStream(content))

{
try
{
var zipInputStream = new ZipInputStream(memoryStream);
using (var streamReader = new StreamReader(zipInputStream, new UTF8Encoding()))
{
result = streamReader.ReadToEnd();
}
}
catch (Exception ex)
{
errorMessage = ex.Message;
}
}

--------------------------------------------------

Content is byte array from UNCOMPRESSED data.

I get Exception on new ZipInputStream(memoryStream);

try - catch catches the Exception with message: Unknown compression method (0x3C). My code is continuing to work fine, but after 2-3 sec. the program crash with message:

"Value cannot be null.
Parameter name: buffer

System.ArgumentNullException was unhandled
HResult=-2147467261
Message=Value cannot be null.
Parameter name: buffer
Source=Telerik.Windows.Zip
ParamName=buffer
StackTrace:
at Telerik.Windows.Zip.OperationStream.ValidateBufferParameters(Byte[] buffer, Int32 offset, Int32 count, Boolean allowZeroCount)
at Telerik.Windows.Zip.BlockTransformBase.ValidateInputBufferParameters(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount, Boolean validateBlockSize, Boolean allowZeroCount)
at Telerik.Windows.Zip.CompressionTransformBase.TransformFinalBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount)
at Telerik.Windows.Zip.OperationStream.FlushFinalBlock()
at Telerik.Windows.Zip.CompressedStream.Dispose(Boolean disposing)
at Telerik.Windows.Zip.OperationStream.Finalize()
InnerException:

Or please, tell me the right way to check if my content is uncompressed

answered on 20 Dec 2016, 12:38 PM

Hi Zlatko,

I've just created zip file with the sample code from this help article, and the file actually starts with 50-4B-03-04, which means that you can use the sample code from the mentioned SO thread, for example from this answer, to detect the file by its first four bytes.

By the way, ZipInputStream class in now obsolete, and the suggested class to use instead is CompressedStream. I've just tried the following sample code and it's working as expected:

using (var memoryStream = new MemoryStream())

{

using (var writeCompressedStream = new CompressedStream(memoryStream, StreamOperationMode.Write, new DeflateSettings()))

using (var inputFileStream = new FileStream(@"d:\temp\test.txt", FileMode.Open))

{

inputFileStream.CopyTo(writeCompressedStream);

writeCompressedStream.Flush();

}

memoryStream.Position = 0L;

using (var readCompressedStream = new CompressedStream(memoryStream, StreamOperationMode.Read, new DeflateSettings()))

using (var outputFileStream = new FileStream(@"d:\temp\test1.txt", FileMode.Create))

{

readCompressedStream.CopyTo(outputFileStream);

}

The text file is successfully compressed in a stream, and subsequently decompressed.

That said, there is difference between a compressed stream and a zip file. The code from SO will check for full zip files, while the compressed streams are only streams compressed with the particular decoded - they do not start with a particular byte sequence (or at least it's different).

Regards,
Boby
Telerik by Progress

Try our brand new, jQuery-free Angular 2 components built from ground-up which deliver the business app essential building blocks - a grid component, data visualization (charts) and form elements.

answered on 21 Dec 2016, 09:15 AM

There is my code:

----------------

public static string ZipDecompress(byte[] content)
{
string result = null;
using (var memoryStream = new MemoryStream(content))
{
try
{
Stream zipInputStream = new ZipInputStream(memoryStream); // <-- Here is the problem
using (var streamReader = new StreamReader(zipInputStream, new UTF8Encoding()))
{ result = streamReader.ReadToEnd(); }
}
catch (Exception ex){}
}
return result;
}

----------------------

There byte[] content is array created from string "Hello World" (Notice, this is uncompressed byte array, if it is compressed all is fine)

Exception is raised, and I catch him.

But after 2-3 sec. I've got the application crash with Error like: "buffer" is already closed. Can't close twice.

Please test this method!

answered on 22 Dec 2016, 12:14 PM

ExpanderSample.rar

Hi Zlatko,

We used your code and tried to replicate the "buffer already closed" issue. We receive our exception when processing the header but we cannot reproduce any other exceptions. We tried forcing the GC if it is related. Please check out our test project. We would be grateful if you can edit it so that the exception is reproducible or send us other isolation that we can debug locally.

As for the phone assistance, it is available only with DevCraft Ultimate license, you could check our Support Plans for more details: http://www.telerik.com/purchase/support-plans/devtools

Regards,
Petar Mladenov
Telerik by Progress

Try our brand new, jQuery-free Angular 2 components built from ground-up which deliver the business app essential building blocks - a grid component, data visualization (charts) and form elements.

answered on 23 Dec 2016, 11:57 AM

Hi Zlatko,

We would be glad to log an issue and plan it for future releases. However, we are still unable to reproduce it on our side. When we close the application (like in your first) or when we build and run the exe and press the button (like in your second video) we have no such exception like yours. We are testing with R3 2016 version of our controls. Could you please let us know your:

- version of Telerik UI Controls for WPF
- VS version (with the exact upgrade)
- .Net installed version on your machine

This will better help us in reproducing the exception. Thank you in advance for your cooperation.

Regards,
Petar Mladenov
Telerik by Progress

Try our brand new, jQuery-free Angular 2 components built from ground-up which deliver the business app essential building blocks - a grid component, data visualization (charts) and form elements.

answered on 28 Dec 2016, 12:47 PM

Hello Zlatko,

We tested the project with your version (Q2 2014 SP 7.29) and we managed to reproduce the exception.
The good news is that with the next version onward (Q3 2014 ->) the exception is fixed. We also found the original bug we have fixed whose name is:

"Using compressed stream to open corrupted zip archive causes exception after the browser is closed."

The code for the fix is located in private and internal methods so the only option we can suggest for you is upgrade to our latest release R3 2016 or wait for R1 2017 in mid January.

Regards,
Petar Mladenov
Telerik by Progress

Try our brand new, jQuery-free Angular 2 components built from ground-up which deliver the business app essential building blocks - a grid component, data visualization (charts) and form elements.

Product Bundles

DevCraft

Web

Mobile

Document Management

Desktop

Reporting & Mocking

Automated Testing

CMS

UI/UX Tools

Debugging

Free Tools

ZipInputStream

18 Answers, 1 is accepted