SWCompression

A framework which contains implementations of (de)compression algorithms and functions which parse various archives and containers.

Developed with Swift.

Motivation

There are a couple of reasons for the project's development.

The main reason is that it is very educational.

Secondly, if you are a Swift developer and you want to compress/decompress something in your project you have to use either wrapper around system libraries (which is probably written in Objective-C) or you have to use built-in Compression framework. You might think that last option is what you need, but, frankly that framework has a bit complicated API and somewhat questionable choice of supported compression algorithms. And yes, it is also in Objective-C.

And here comes SWCompression: no Objective-C, pure Swift.

Features

• Containers:
  • ZIP
  • TAR
  • 7-Zip
• Decompression algorithms:
  • LZMA/LZMA2
  • Deflate
  • BZip2
• Compression algorithms:
  • Deflate
• Archives:
  • XZ
  • GZip
  • Zlib
• Platform independent.
• Written with Swift only.

By the way, it seems like GZip, Deflate and Zlib implementations are specification compliant.

Installation

SWCompression can be integrated into your project either using CocoaPods, Carthage or Swift Package Manager.

CocoaPods

Add to your Podfile pod 'SWCompression'.

There are several sub-podspecs in case you need only parts of framework's functionality. Available subspecs:

• SWCompression/LZMA
• SWCompression/XZ
• SWCompression/Deflate
• SWCompression/Gzip
• SWCompression/Zlib
• SWCompression/BZip2
• SWCompression/ZIP
• SWCompression/TAR
• SWCompression/SevenZip

You can add some or all of them instead of pod 'SWCompression'

Also, do not forget to include use_frameworks! line in your Podfile.

To complete installation, run pod install.

Carthage

Add to your Cartfile github "tsolomko/SWCompression".

Then run carthage update.

Finally, drag and drop SWCompression.framework from Carthage/Build folder into the "Embedded Binaries" section on your targets' "General" tab.

Swift Package Manager

Add to you package dependecies .Package(url: "https://github.com/tsolomko/SWCompression.git"), for example like this:

import PackageDescription

let package = Package(
    name: "PackageName",
    dependencies: [
        .Package(url: "https://github.com/tsolomko/SWCompression.git", majorVersion: 3)
    ]
)

More info about SPM you can find at Swift Package Manager's Documentation.

Options for CocoaPods users

Both ZIP and 7-Zip containers have compression method which is most likely to be used when compressing files into them. This is Deflate for ZIP and LZMA/LZMA2 for 7-Zip. Thus, SWCompression/ZIP subspec have SWCompression/Deflate subspec as a dependency and SWCompression/LZMA subspec as a dependency for SWCompression/SevenZip.

But both these containers support other compression methods, some of them are implemented in SWCompression. For CocoaPods configurations some sort of 'optional dependecies' are provided for such compression methods.

'Optional dependency' in this context means that SWCompression/ZIP or SWCompression/7-Zip will support particular compression methods only if a corresponding subspec is expicitly specified in your Podfile and installed.

List of 'optional dependecies'.

For SWCompression/ZIP:

• SWCompression/BZip2
• SWCompression/LZMA

For SWCompression/SevenZip:

• SWCompression/BZip2
• SWCompression/Deflate

Note: If you use Carthage or Swift Package Manager you always have the full package, and ZIP will be built with both additional BZip2 and LZMA support as well as 7-Zip will be build with both additional Deflate and BZip2 support.

Usage

Basics

If you'd like to decompress "deflated" data just use:

let data = try! Data(contentsOf: URL(fileURLWithPath: "path/to/file"),
                     options: .mappedIfSafe)
let decompressedData = try? Deflate.decompress(data: data)

Note: It is highly recommended to specify Data.ReadingOptions.mappedIfSafe, especially if you are working with large files, so you don't run out of system memory.

However, it is unlikely that you will encounter deflated data outside of any archive. So, in case of GZip archive you should use:

let decompressedData = try? GzipArchive.unarchive(archiveData: data)

One final note: every SWCompression function can throw an error and you are responsible for handling them.

Documentation

Every function or class of public API of SWCompression is documented. This documentation can be found at its own website.

Handling Errors

If you look at list of available error types and their cases, you may be frightened by their number. However, most of these cases (such as XZError.wrongMagic) exist for diagnostic purposes.

Thus, you only need to handle the most common type of error for your archive/algorithm. For example:

do {
  let data = try Data(contentsOf: URL(fileURLWithPath: "path/to/file"),
                      options: .mappedIfSafe)
  let decompressedData = XZArchive.unarchive(archive: data)
} catch let error as XZError {
  <handle XZ related error here>
} catch let error {
  <handle all other errors here>
}

Sophisticated example

There is a small program, swcomp, which uses SWCompression for unarchiving several types of archives.

Performace

TL;DR Constantly trying to improve performance; use whole module optimizations, which are enabled by default for Release configurations.

Further thoughts, details and notes about performance you can read in a separate document.

Tests Results document contains results of performance testing of various algorithms.

Running tests locally

If you want to run tests locally you need to clone this repository and do some additional steps:

git submodule update --init --recursive
cd Tests/Test\ Files
git lfs pull

These commands fetch example archives and other files which are used for testing. These files are stored in a separate repository. Git LFS is also used for storing them which basically is the reason for having them in other repository. Otherwise, using Swift Package Manager to install SWCompression is a bit challenging (requires installing git-lfs locally with --skip-smudge option to solve the problem).

Known issues

• wrongCRC and wrongCheck errors for XZ and GZip multi-member archives contain only last member's data as their associated value instead of all successfully processed members.

Comment: Philosophy for such errors is that by the time these errors are thrown, decompression was already performed, so we can still provide the result of decompression to the caller. It is intended to fix this problem, but solution requires backwards-incompatible API changes so it is delayed until 4.0 release.

Future plans

• BZip2 compression.
• Container API rework.
• Better Deflate compression.
• Something else...

References

• pyflate
• Deflate specification
• GZip specification
• Zlib specfication
• LZMA SDK and specification
• XZ specification
• Wikipedia article about LZMA
• .ZIP Application Note
• ISO/IEC 21320-1
• List of defined ZIP extra fields
• Wikipedia article about TAR
• Pax specification
• Basic TAR specification
• Apache Commons Compress
• A walk through the SA-IS Suffix Array Construction Algorithm