Class ParallelBZip2OutputStream

Namespace

Ionic.BZip2

Assembly

SunamoDotNetZip.dll

A write-only decorator stream that compresses data as it is written using the BZip2 algorithm. This stream compresses by block using multiple threads.

public class ParallelBZip2OutputStream : Stream, IAsyncDisposable, IDisposable

Inheritance

object

MarshalByRefObject

Stream

ParallelBZip2OutputStream

Implements

IAsyncDisposable

IDisposable

Inherited Members

Stream.Null

Stream.BeginRead(byte[], int, int, AsyncCallback, object)

Stream.BeginWrite(byte[], int, int, AsyncCallback, object)

Stream.CopyTo(Stream)

Stream.CopyTo(Stream, int)

Stream.CopyToAsync(Stream)

Stream.CopyToAsync(Stream, int)

Stream.CopyToAsync(Stream, int, CancellationToken)

Stream.CopyToAsync(Stream, CancellationToken)

Stream.CreateWaitHandle()

Stream.Dispose()

Stream.Dispose(bool)

Stream.DisposeAsync()

Stream.EndRead(IAsyncResult)

Stream.EndWrite(IAsyncResult)

Stream.FlushAsync()

Stream.FlushAsync(CancellationToken)

Stream.ObjectInvariant()

Stream.Read(Span<byte>)

Stream.ReadAsync(byte[], int, int)

Stream.ReadAsync(byte[], int, int, CancellationToken)

Stream.ReadAsync(Memory<byte>, CancellationToken)

Stream.ReadAtLeast(Span<byte>, int, bool)

Stream.ReadAtLeastAsync(Memory<byte>, int, bool, CancellationToken)

Stream.ReadByte()

Stream.ReadExactly(byte[], int, int)

Stream.ReadExactly(Span<byte>)

Stream.ReadExactlyAsync(byte[], int, int, CancellationToken)

Stream.ReadExactlyAsync(Memory<byte>, CancellationToken)

Stream.Synchronized(Stream)

Stream.ValidateBufferArguments(byte[], int, int)

Stream.ValidateCopyToArguments(Stream, int)

Stream.Write(ReadOnlySpan<byte>)

Stream.WriteAsync(byte[], int, int)

Stream.WriteAsync(byte[], int, int, CancellationToken)

Stream.WriteAsync(ReadOnlyMemory<byte>, CancellationToken)

Stream.WriteByte(byte)

Stream.CanTimeout

Stream.ReadTimeout

Stream.WriteTimeout

MarshalByRefObject.GetLifetimeService()

MarshalByRefObject.InitializeLifetimeService()

MarshalByRefObject.MemberwiseClone(bool)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

FuncExtensions.AsFuncFast<T>(T)

FuncExtensions.AsFunc<T>(T)

ObjectExtensions.GetStackTrace(object)

ReClasser.FixMeUp(object)

Constructors

ParallelBZip2OutputStream(Stream)

Constructs a new ParallelBZip2OutputStream, that sends its compressed output to the given output stream.

public ParallelBZip2OutputStream(Stream output)

Parameters

output Stream

The destination stream, to which compressed output will be sent.

Examples

This example reads a file, then compresses it with bzip2 file, and writes the compressed data into a newly created file.

var fname = "logfile.log";
using (var fs = File.OpenRead(fname))
{
    var outFname = fname + ".bz2";
    using (var output = File.Create(outFname))
    {
        using (var compressor = new ParallelBZip2OutputStream(output))
        {
            byte[] buffer = new byte[2048];
            int n;
            while ((n = fs.Read(buffer, 0, buffer.Length)) > 0)
            {
                compressor.Write(buffer, 0, n);
            }
        }
    }
}

ParallelBZip2OutputStream(Stream, bool)

Constructs a new ParallelBZip2OutputStream.

public ParallelBZip2OutputStream(Stream output, bool leaveOpen)

Parameters

output Stream

the destination stream.

leaveOpen bool

whether to leave the captive stream open upon closing this stream.

ParallelBZip2OutputStream(Stream, int)

Constructs a new ParallelBZip2OutputStream with specified blocksize.

public ParallelBZip2OutputStream(Stream output, int blockSize)

Parameters

output Stream

the destination stream.

blockSize int

The blockSize in units of 100000 bytes. The valid range is 1..9.

ParallelBZip2OutputStream(Stream, int, bool)

Constructs a new ParallelBZip2OutputStream with specified blocksize, and explicitly specifies whether to leave the wrapped stream open.

public ParallelBZip2OutputStream(Stream output, int blockSize, bool leaveOpen)

Parameters

output Stream

the destination stream.

blockSize int

The blockSize in units of 100000 bytes. The valid range is 1..9.

leaveOpen bool

whether to leave the captive stream open upon closing this stream.

Properties

BlockSize

The blocksize parameter specified at construction time.

public int BlockSize { get; }

Property Value

int

BytesWrittenOut

The total number of bytes written out by the stream.

public long BytesWrittenOut { get; }

Property Value

long

Remarks

This value is meaningful only after a call to Close().

CanRead

Indicates whether the stream can be read.

public override bool CanRead { get; }

Property Value

bool

Remarks

The return value is always false.

CanSeek

Indicates whether the stream supports Seek operations.

public override bool CanSeek { get; }

Property Value

bool

Remarks

Always returns false.

CanWrite

Indicates whether the stream can be written.

public override bool CanWrite { get; }

Property Value

bool

Remarks

The return value depends on whether the captive stream supports writing.

Length

Reading this property always throws a NotImplementedException.

public override long Length { get; }

Property Value

long

MaxWorkers

The maximum number of concurrent compression worker threads to use.

public int MaxWorkers { get; set; }

Property Value

int

Remarks

This property sets an upper limit on the number of concurrent worker threads to employ for compression. The implementation of this stream employs multiple threads from the .NET thread pool, via ThreadPool.QueueUserWorkItem(), to compress the incoming data by block. As each block of data is compressed, this stream re-orders the compressed blocks and writes them to the output stream.

A higher number of workers enables a higher degree of parallelism, which tends to increase the speed of compression on multi-cpu computers. On the other hand, a higher number of buffer pairs also implies a larger memory consumption, more active worker threads, and a higher cpu utilization for any compression. This property enables the application to limit its memory consumption and CPU utilization behavior depending on requirements.

By default, DotNetZip allocates 4 workers per CPU core, subject to the upper limit specified in this property. For example, suppose the application sets this property to 16. Then, on a machine with 2 cores, DotNetZip will use 8 workers; that number does not exceed the upper limit specified by this property, so the actual number of workers used will be 4 * 2 = 8. On a machine with 4 cores, DotNetZip will use 16 workers; again, the limit does not apply. On a machine with 8 cores, DotNetZip will use 16 workers, because of the limit.

For each compression "worker thread" that occurs in parallel, there is up to 2mb of memory allocated, for buffering and processing. The actual number depends on the BlockSize property.

CPU utilization will also go up with additional workers, because a larger number of buffer pairs allows a larger number of background threads to compress in parallel. If you find that parallel compression is consuming too much memory or CPU, you can adjust this value downward.

The default value is 16. Different values may deliver better or worse results, depending on your priorities and the dynamic performance characteristics of your storage and compute resources.

The application can set this value at any time, but it is effective only before the first call to Write(), which is when the buffers are allocated.

Position

The position of the stream pointer.

public override long Position { get; set; }

Property Value

long

Remarks

Setting this property always throws a NotImplementedException. Reading will return the total number of uncompressed bytes written through.

Methods

Close()

Close the stream.

public override void Close()

Remarks

This may or may not close the underlying stream. Check the constructors that accept a bool value.

Flush()

Flush the stream.

public override void Flush()

Read(byte[], int, int)

Calling this method always throws a NotImplementedException.

public override int Read(byte[] buffer, int offset, int count)

Parameters

buffer byte[]

this parameter is never used

offset int

this parameter is never used

count int

this parameter is never used

Returns

int

never returns anything; always throws

Seek(long, SeekOrigin)

Calling this method always throws a NotImplementedException.

public override long Seek(long offset, SeekOrigin origin)

Parameters

offset long

this is irrelevant, since it will always throw!

origin SeekOrigin

this is irrelevant, since it will always throw!

Returns

long

irrelevant!

SetLength(long)

Calling this method always throws a NotImplementedException.

public override void SetLength(long value)

Parameters

value long

this is irrelevant, since it will always throw!

Write(byte[], int, int)

Write data to the stream.

public override void Write(byte[] buffer, int offset, int count)

Parameters

buffer byte[]

The buffer holding data to write to the stream.

offset int

the offset within that data array to find the first byte to write.

count int

the number of bytes to write.

Remarks

Use the ParallelBZip2OutputStream to compress data while writing: create a ParallelBZip2OutputStream with a writable output stream. Then call Write() on that ParallelBZip2OutputStream, providing uncompressed data as input. The data sent to the output stream will be the compressed form of the input data.

A ParallelBZip2OutputStream can be used only for Write() not for Read().

See Also

BZip2OutputStream