WinApi compression, decompression in python3 Code Answer

Hello Developer, Hope you guys are doing great. Today at Tutorial Guruji Official website, we are sharing the answer of WinApi compression, decompression in python3 without wasting too much if your time.

The question is published on by Tutorial Guruji team.

I have a small client application designed to run on windows machines. It is written in C/C++ (more in C). I want to keep it small and therefore I would prefer not to use external libraries and stick to WinAPI on client side.

On the other hand I have a backend server which is implemented in python3. Here I’m happy to use any existing lib. Now I want to add compression layer to increase transfer speed. The problem I’ve encountered is that it seems that WinAPI provides only:

// source: https://docs.microsoft.com/en-us/windows/win32/cmpapi/using-the-compression-api
XPRESS
XPRESS with Huffman encoding 
MSZIP  
LZMS

which seem to be a unique Microsoft compression algorithm implementation and I cannot find a way to decompress the data on the server side in python3.

Is there anything I am missing? I would love to hear some solutions:)

Thanks in advance

UPDATE

I’ve decided to use zlib https://zlib.net/ as suggested in the comment. The answer suggesting using ctypes was also very interesting, but unfortenately my backend is running on UNIX system.

As I am compiling my client part agains Multi-threaded CRT (and not DLL) I had some problems as zlib is being linked agains the Muli-threaded DLL. If anyone has enountered such issue, I have found a great and super simple solution here: https://yongweiwu.wordpress.com/2017/10/02/a-journey-of-purely-static-linking/ I will copy-paste it here:

zlib
This part requires a small change to the build script (for version 1.2.11).
I need to open win32Makefile.msc and change all occurrences of ‘-MD’ to ‘-MT’. 
Then these commands will work:

nmake -f win32Makefile.msc zlib.lib

Answer

I found this Python script, and reverse-engineered it to produce a quick library to handle WinAPI Compression/Decompression. Basically, you can just use ctypes and call the WinAPI from Python. Please keep in mind that I haven’t extensively tested this, but it should give you a pretty good place to start 🙂

EDIT: As requested, I’ve included the implementations of the compress and decompress functions in case the links ever go down.

def compress(UncompressedBuffer, UncompressedBufferSize, Format, Engine):
    CompressedBuffer = (ctypes.c_ubyte * UncompressedBufferSize)()
    CompressionFormatAndEngine = ctypes.c_uint16(Format.value | Engine.value)

    CompressBufferWorkSpaceSize = ctypes.c_uint32()
    CompressFragmentWorkSpaceSize = ctypes.c_uint32()
    WorkSpace = (CompressFragmentWorkSpaceSize.value * ctypes.c_ubyte)()
    FinalCompressedSize = ctypes.c_uint32()

    WinDLLCall(RtlGetCompressionWorkSpaceSize,
               CompressionFormatAndEngine,
               ctypes.byref(CompressBufferWorkSpaceSize),
               ctypes.byref(CompressFragmentWorkSpaceSize))

    WinDLLCall(RtlCompressBuffer,
              CompressionFormatAndEngine,
              ctypes.byref(UncompressedBuffer),
              ctypes.c_uint32(UncompressedBufferSize),
              ctypes.byref(CompressedBuffer),
              ctypes.c_uint32(UncompressedBufferSize),
              UncompressedChunkSize,
              ctypes.byref(FinalCompressedSize),
              ctypes.byref(WorkSpace))

    return CompressedBuffer, FinalCompressedSize

def decompress(CompressedBuffer, CompressedBufferSize, UncompressedBufferSize, Format):
    UncompressedBuffer = (ctypes.c_ubyte * UncompressedBufferSize)()
    FinalUncompressedSize = ctypes.c_uint32()

    WinDLLCall(RtlDecompressBuffer,
               Format,
               ctypes.byref(UncompressedBuffer),
               ctypes.c_uint32(UncompressedBufferSize),
               ctypes.byref(CompressedBuffer),
               ctypes.c_uint32(CompressedBufferSize),
               ctypes.byref(FinalUncompressedSize))

    return UncompressedBuffer, FinalUncompressedSize

The WinAPI functions are loaded with the following code:

import ctypes

RtlDecompressBuffer = ctypes.windll.ntdll.RtlDecompressBuffer
RtlCompressBuffer = ctypes.windll.ntdll.RtlCompressBuffer
RtlGetCompressionWorkSpaceSize = ctypes.windll.ntdll.RtlGetCompressionWorkSpaceSize

Here are some other useful resources if you end up tinkering around:

We are here to answer your question about WinApi compression, decompression in python3 - If you find the proper solution, please don't forgot to share this with your team members.

Related Posts

Tutorial Guruji