如何将 GZipStream 与 System.IO.MemoryStream 一起使用?

Posted

技术标签:

【中文标题】如何将 GZipStream 与 System.IO.MemoryStream 一起使用?【英文标题】:How do I use GZipStream with System.IO.MemoryStream? 【发布时间】:2011-04-12 22:33:07 【问题描述】:

我遇到了这个测试函数的问题,我在内存中获取一个字符串,压缩它,然后解压缩它。压缩效果很好,但我似乎无法让解压工作。

//Compress
System.IO.MemoryStream outStream = new System.IO.MemoryStream();                
GZipStream tinyStream = new GZipStream(outStream, CompressionMode.Compress);
mStream.Position = 0;
mStream.CopyTo(tinyStream);

//Decompress    
outStream.Position = 0;
GZipStream bigStream = new GZipStream(outStream, CompressionMode.Decompress);
System.IO.MemoryStream bigStreamOut = new System.IO.MemoryStream();
bigStream.CopyTo(bigStreamOut);

//Results:
//bigStreamOut.Length == 0
//outStream.Position == the end of the stream.

我相信 bigStream out 至少应该包含数据,尤其是在读取我的源流 (outStream) 时。这是 MSFT 的错误还是我的?

【问题讨论】:

【参考方案1】:

在您的代码中发生的情况是您不断打开流,但从未关闭它们。

在第 2 行中,您创建了一个 GZipStream。该流不会向底层流写入任何内容,直到它认为是正确的时间。你可以通过关闭它来告诉它。

但是,如果你关闭它,它也会关闭底层流 (outStream)。因此你不能在上面使用mStream.Position = 0

您应该始终使用using 来确保您的所有信息流都已关闭。这是您的代码的一个变体。

var inputString = "“ ... ”";
byte[] compressed;
string output;

using (var outStream = new MemoryStream())

    using (var tinyStream = new GZipStream(outStream, CompressionMode.Compress))
    using (var mStream = new MemoryStream(Encoding.UTF8.GetBytes(inputString)))
        mStream.CopyTo(tinyStream);

    compressed = outStream.ToArray();


// “compressed” now contains the compressed string.
// Also, all the streams are closed and the above is a self-contained operation.

using (var inStream = new MemoryStream(compressed))
using (var bigStream = new GZipStream(inStream, CompressionMode.Decompress))
using (var bigStreamOut = new MemoryStream())

    bigStream.CopyTo(bigStreamOut);
    output = Encoding.UTF8.GetString(bigStreamOut.ToArray());


// “output” now contains the uncompressed string.
Console.WriteLine(output);

【讨论】:

+1 好答案 Timwi。除此之外,GZip 有一些内部数据缓冲,它需要进行压缩才能进行压缩。在您关闭它之前,它无法知道它已完成接收数据,因此它不会吐出最后几个字节并且部分流的解压缩失败。 我认为我们使用的是 .NET 3.5(使用 Unity),所以 .CopyTo 还不存在。在 SO 的其他地方寻找如何从一个流复制到另一个流:***.com/questions/230128/… 谢谢你,我一直无法弄清楚如何安排流以在两个方向上获得正确的输出【参考方案2】:

这是一个已知问题:http://blogs.msdn.com/b/bclteam/archive/2006/05/10/592551.aspx

我已经稍微修改了你的代码,所以这个可以工作:

var mStream = new MemoryStream(new byte[100]);
var outStream = new System.IO.MemoryStream();

using (var tinyStream = new GZipStream(outStream, CompressionMode.Compress))

    mStream.CopyTo(tinyStream);           


byte[] bb = outStream.ToArray();

//Decompress                
var bigStream = new GZipStream(new MemoryStream(bb), CompressionMode.Decompress);
var bigStreamOut = new System.IO.MemoryStream();
bigStream.CopyTo(bigStreamOut);

【讨论】:

不应该使用new GZipStream(outStream, CompressionMode.Compress, true) 来打开流,以便使用语句可以按照@briantyler 的答案中的建议关闭它?【参考方案3】:

MemoryStream 进行压缩和解压缩的方法是:

public static Stream Compress(
    Stream decompressed, 
    CompressionLevel compressionLevel = CompressionLevel.Fastest)

    var compressed = new MemoryStream();
    using (var zip = new GZipStream(compressed, compressionLevel, true))
    
        decompressed.CopyTo(zip);
    

    compressed.Seek(0, SeekOrigin.Begin);
    return compressed;


public static Stream Decompress(Stream compressed)

    var decompressed = new MemoryStream();
    using (var zip = new GZipStream(compressed, CompressionMode.Decompress, true))
    
        zip.CopyTo(decompressed);
    

    decompressed.Seek(0, SeekOrigin.Begin);
    return decompressed;

这会使压缩/解压缩的流保持打开状态,并在创建后可用。

【讨论】:

【参考方案4】:

另一种实现,在 VB.NET 中:

Imports System.Runtime.CompilerServices
Imports System.IO
Imports System.IO.Compression

Public Module Compressor

    <Extension()> _
    Function CompressASCII(str As String) As Byte()

        Dim bytes As Byte() = Encoding.ASCII.GetBytes(str)

        Using ms As New MemoryStream

            Using gzStream As New GZipStream(ms, CompressionMode.Compress)

                gzStream.Write(bytes, 0, bytes.Length)

            End Using

            Return ms.ToArray

        End Using

    End Function

    <Extension()> _
    Function DecompressASCII(compressedString As Byte()) As String

        Using ms As New MemoryStream(compressedString)

            Using gzStream As New GZipStream(ms, CompressionMode.Decompress)

                Using sr As New StreamReader(gzStream, Encoding.ASCII)

                    Return sr.ReadToEnd

                End Using

            End Using

        End Using

    End Function

    Sub TestCompression()

        Dim input As String = "fh3o047gh"

        Dim compressed As Byte() = input.CompressASCII()

        Dim decompressed As String = compressed.DecompressASCII()

        If input <> decompressed Then
            Throw New ApplicationException("failure!")
        End If

    End Sub

End Module

【讨论】:

即使您的帖子已有 6 年历史并且我不是提问者 - 它今天对我有所帮助。谢谢!【参考方案5】:
    public static byte[] compress(byte[] data)
    
        using (MemoryStream outStream = new MemoryStream())
        
            using (GZipStream gzipStream = new GZipStream(outStream, CompressionMode.Compress))
            using (MemoryStream srcStream = new MemoryStream(data))
                srcStream.CopyTo(gzipStream);
            return outStream.ToArray();
        
    

    public static byte[] decompress(byte[] compressed)
    
        using (MemoryStream inStream = new MemoryStream(compressed))
        using (GZipStream gzipStream = new GZipStream(inStream, CompressionMode.Decompress))
        using (MemoryStream outStream = new MemoryStream())
        
            gzipStream.CopyTo(outStream);
            return outStream.ToArray();
        
    

【讨论】:

【参考方案6】:

如果您尝试使用 MemoryStream(例如,将其传递给另一个函数)但收到异常“无法访问已关闭的流”。那么你可以使用另一个 GZipStream 构造函数来帮助你。

通过将 true 传递给 leaveOpen 参数,您可以指示 GZipStream 在处理完自身后使流保持打开状态,默认情况下它会关闭目标流(这是我没想到的)。 https://msdn.microsoft.com/en-us/library/27ck2z1y(v=vs.110).aspx

using (FileStream fs = File.OpenRead(f))
using (var compressed = new MemoryStream())

    //Instruct GZipStream to leave the stream open after performing the compression.
    using (var gzipstream = new GZipStream(compressed, CompressionLevel.Optimal, true))
        fs.CopyTo(gzipstream);

    //Do something with the memorystream
    compressed.Seek(0, SeekOrigin.Begin);
    MyFunction(compressed);

【讨论】:

【参考方案7】:

如果你仍然需要它,你可以使用带有布尔参数的 GZipStream 构造函数(有两个这样的构造函数)并在那里传递真值:

tinyStream = new GZipStream(outStream, CompressionMode.Compress, true);

在这种情况下,当您关闭 tynyStream 时,您的输出流仍将打开。不要忘记复制数据:

mStream.CopyTo(tinyStream);
tinyStream.Close();

现在你已经有了带有压缩数据的内存流 outStream

给你的虫子和亲吻

祝你好运

【讨论】:

【参考方案8】:

请参考以下链接,避免使用双内存流。 https://***.com/a/53644256/1979406

【讨论】:

【参考方案9】:

我遇到了一个问题,*.CopyTo(stream)* 最终会得到 byte[0] 结果。 解决方案是在调用.CopyTo(stream) 之前添加.Position=0 Answered here

我还使用BinaryFormatter,如果在反序列化之前位置未设置为 0,则会引发“解析完成之前遇到的流结束”异常。 Answered here

这是对我有用的代码。

 public static byte[] SerializeAndCompressStateInformation(this IPluginWithStateInfo plugin, Dictionary<string, object> stateInfo)
    
        byte[] retArr = new byte[]  byte.MinValue ;
        try
        
            using (MemoryStream msCompressed = new MemoryStream())//what gzip writes to
            
                using (GZipStream gZipStream = new GZipStream(msCompressed, CompressionMode.Compress))//setting up gzip
                using (MemoryStream msToCompress = new MemoryStream())//what the settings will serialize to
                
                    BinaryFormatter formatter = new BinaryFormatter();
                    //serialize the info into bytes
                    formatter.Serialize(msToCompress, stateInfo);
                    //reset to 0 to read from beginning byte[0] fix.
                    msToCompress.Position = 0;
                    //this then does the compression
                    msToCompress.CopyTo(gZipStream);
                
                //the compressed data as an array of bytes
                retArr = msCompressed.ToArray();
            
        
        catch (Exception ex)
        
            Logger.Error(ex.Message, ex);
            throw ex;
        
        return retArr;
    


    public static Dictionary<string, object> DeserializeAndDecompressStateInformation(this IPluginWithStateInfo plugin, byte[] stateInfo)
    
        Dictionary<string, object> settings = new Dictionary<string, object>();
        try
        

            using (MemoryStream msDecompressed = new MemoryStream()) //the stream that will hold the decompressed data
            
                using (MemoryStream msCompressed = new MemoryStream(stateInfo))//the compressed data
                using (GZipStream gzDecomp = new GZipStream(msCompressed, CompressionMode.Decompress))//the gzip that will decompress
                
                    msCompressed.Position = 0;//fix for byte[0]
                    gzDecomp.CopyTo(msDecompressed);//decompress the data
                
                BinaryFormatter formatter = new BinaryFormatter();
                //prevents 'End of stream encountered' error
                msDecompressed.Position = 0;
                //change the decompressed data to the object
                settings = formatter.Deserialize(msDecompressed) as Dictionary<string, object>;
            
        
        catch (Exception ex)
        
            Logger.Error(ex.Message, ex);
            throw ex;
        
        return settings;
    

【讨论】:

以上是关于如何将 GZipStream 与 System.IO.MemoryStream 一起使用?的主要内容,如果未能解决你的问题,请参考以下文章

如何在 C# 中使用 GZipStream 解压缩?

大数据上的 GZipStream

GZipStream - 块长度与其补码不匹配

gzipstream 解压缩 UNIX 文件中止且没有错误

Java压缩流GZIPStream导致的内存泄露

Java压缩流GZIPStream导致的内存泄露