LibCompress

This project is abandoned.

Whether this project is out of date or its author marked it as abandoned, this project is no longer maintained.

If you wish to take this project over, please report it and state your intentions.

LibCompress is a compression and decompression library implemented entirely in WoW-friendly Lua. It supports the LZW and Huffman algorithms, and can automatically choose the most efficient algorithm for your data. One popular usage for this library is to send a compressed table to another player or add-on. Doing this requires additional encoding to remove the \000 characters from the data stream.

Take a look at the forum post for more info and a development discussion:

http://forums.wowace.com/showthread.php?t=12660...

Usage:

Compression

Load the library with:

libc = LibStub:GetLibrary("LibCompress")

Compress data (must be in string form):

compressed_data = libc:Compress(data)

This will try all compression algorithms and return the best compressed result. It is possible to specify a specific compression algorithm like this:

compressed_data = libc:CompressHuffman(data)

or

compressed_data = libc:CompressLZW(data) 

Data will either be compressed with the Huffman compression algorithm or not at all. Data returned with a prefix byte identifying that the data is decompressed.

To decompress the data, simply use this:

decompressed_data = libc:Decompress(compressed_data)

Compress and Decompress can return an error and this is signaled by the first returned argument being nil and the second the error message. So checking for that would be appropriate.

Encoding

LibCompress also has the possibility to encode and decode data, preparing it for transmission over the addon channel or chat channel (or a custom encoding). Two forms of encoding is provided:

Prefix encoding

The first form is prefix-encoding. Basically, reserved characters are replaced with a prefix/escape character followed by the suffix character, i.e. reserved bytes are replaced by a double-byte combination. This is how it is done:

table, msg = libc:GetEncodeTable(reservedChars, escapeChars,  mapChars)

reservedChars: The characters in this string will not appear in the encoded data. escapeChars: A string of characters used as escape-characters (don't supply more than needed). #escapeChars >= 1 mapChars: First characters in reservedChars maps to first characters in mapChars. (#mapChars <= #reservedChars)

If table is nil, then msg holds an error message. Otherwise the usage is simple:

encoded_message = table:Encode(message)
message = table:Decode(encoded_message)

Two predefined setups have been included:

GetAddonEncodeTable: Sets up encoding for the addon channel (\000 is encoded)

GetChatEncodeTable: Sets up encoding for the chat channel (many bytes encoded, see the function for details)

7-bit encoding

This encoding packs bits, not bytes. It puts 7 bits into every byte, enlarging the data by approx 14%. Values from 0 to 127 (both inclusive) are present in the encoded data and therefor has to be prefix-encoded as well. This encoding generates a bit of string trash and should be used with consideration.

Encode data like this:

encoded_data = libc:Encode7bit(data)

Decode data like this:

decoded_data = libc:Decode7bit(encoded_data)

Checksum/hash algorithms

LibCompress also provides 2 reasonable fast hash algorithms. They are converted from a C-implementation to lua and are quite fast. The hash value is either 16 bit or 32 bit.

Use like this (data1, data2, data... = string):

code = libc:fcs16init()
code = libc:fcs16update(code, data1)
code = libc:fcs16update(code, data2)
code = libc:fcs16update(code, data...)
code = libc:fcs16final(code)

data = string fcs16 provides a 16 bit checksum, fcs32 provides a 32 bit checksum.

You must login to post a comment. Don't have an account? Register to get one!

  • 1 comment
  • Avatar of crazedfred crazedfred Jan 06, 2011 at 23:15 UTC - 0 likes

    Hello!

    Just a quick note for those interested, you can use LibCompress, AceSerializer, and AceComms as an efficient way to send arbitrary data to other clients.

    This code will prepare all the libraries needed:

    --Load the libraries
    local libS = LibStub:GetLibrary("AceSerializer-3.0")
    local libC = LibStub:GetLibrary("LibCompress")
    local libCE = libC:GetAddonEncodeTable()
    --AceComm doesn't need loading in this manner if using LibStub
    

    Once you have the library objects set up (see above documentation, as well as the two relevant Ace pages), the following code takes an object and sends it:

    --Serialize and compress the data
    local data = "Example string"
    local one = libS:Serialize(data)
    local two = LibSyncC:CompressHuffman(one)
    local final = LibSyncCE:Encode(two)
    
    --Send it via an addon message
    YourAddon:SendCommMessage("YourAddon", final, channel, target, "BULK")
    

    On the other end, re-constructing the result applies the exact same procedure but in reverse. Since this could fail in various creative ways, this code is more involved:

    -- Decode the compressed data
    local one = libCE:Decode(data)
    	
    --Decompress the decoded data
    local two, message = libC:Decompress(one)
    if(not two) then
    	print("YourAddon: error decompressing: " .. message)
    	return
    end
    	
    -- Deserialize the decompressed data
    local success, final = libS:Deserialize(two)
    if (not success) then
    	print("YourAddon: error deserializing " .. final)
    	return
    end
    
    print("final data: " .. final)
    

    Now, in this particular example, I only used one variable, and a string at that - so there was no real need to use the AceSerializer step. Thanks to LibCompress, sending a string in this manner is still better (length-wise, for any reasonably-long string) than sending it raw, however an advantage of AceSerializer is that any number of arbitrary data types can be thrown at it (tables, floats, etc) and it will be lossessly transmitted.

    It shouldn't be too hard to modify this example to allow you to send many different data fields at once. The combination of these three libraries gives you a powerful method of transferring data.

    Please note that I used CompressHuffman in this case, as it appears to be more length-efficient and faster for compressing (in my limited testing). LibCompress offers a feature that will try all compression algorithms available and use the smallest one: simply use the Compress function instead. This trades runtime (compressing each data set multiple times) for best-possible compression.

    Last edited Jan 06, 2011 by crazedfred
  • 1 comment

Facts

Date created
Jun 01, 2008
Category
Last update
Jan 04, 2011
Development stage
Abandoned
Language
  • enUS
License
GNU General Public License version 2 (GPLv2)
Reverse relationships
8
Downloads
13,409
Recent files

Authors

Relationships

Embedded library
LibStub