Writing a bit reader in JAVA (32-bit little-endian most-to-least-significant bit packing)

No matter how I toss and turn the bytes & bits, I just cannot get this to work. I understand endianess quite ok, and MSB (most significant bit) on some degree. But I’m not able to put the two together it seems.

Currently I have the data in an ordinary JAVA byte[] array. Symbols within the bit stream are encoded using 32-bit little-endian most-to-least-significant bit packing. And I need to read it in a kind of bit reader N bits a time like:

public int read(int bits, boolean advanceReader)

Of course keeping track of the

private int byteOffset;
private int bitOffset;

The issue is getting the values as integers, this I just can’t comprehend how to achieve correctly 🙁

Edit: I’m trying with this Apache Licensed BitReader class with added little endian (I added on to the readInt()) support (original code: https://raw.githubusercontent.com/jcodec/jcodec/master/src/main/java/org/jcodec/common/io/BitReader.java):

public class BitReader {

protected int deficit;
protected int curInt;
private ByteBuffer bb;
private int initPos;

public BitReader(ByteBuffer bb) {
    this.bb = bb;
    initPos = bb.position();
    curInt = readInt();
    deficit = 0;
}

public final int readInt() {
    if (bb.remaining() >= 4) {
        deficit -= 32;
        final int b1 = bb.get() & 0xff;
        final int b2 = bb.get() & 0xff;
        final int b3 = bb.get() & 0xff;
        final int b4 = bb.get() & 0xff;
        if (bb.order() == ByteOrder.BIG_ENDIAN) {
            return (b1 << 24) | (b2 << 16) | (b3 << 8) | b4;
        } else {
            return (b4 << 24) | (b3 << 16) | (b2 << 8) | b1;
        }
    } else {
        return readIntSafe();
    }
}

public int readNBit(int n) {
    if (n > 32) {
        throw new IllegalArgumentException("Can not read more then 32 bit");
    }

    int nn = n;

    int ret = 0;
    if (n + deficit > 31) {
        ret |= (curInt >>> deficit);
        n -= 32 - deficit;
        ret <<= n;
        deficit = 32;
        curInt = readInt();
    }

    if (n != 0) {
        ret |= curInt >>> (32 - n);
        curInt <<= n;
        deficit += n;
    }

    // for(--nn; nn >=0; nn--)
    // System.out.print((ret >> nn) & 1);
    // System.out.println();

    return ret;
}

public int position() {
    return ((bb.position() - initPos - 4) << 3) + deficit;
}

}

Maybe I’m missing something in the readNBit, it still handles like big endian? The results are near there, but not quite correct.

Some data:

The array:
[ 0] byte 64
[ 1] byte 1
[ 2] byte -32
[ 3] byte 1
[ 4] byte 100
[ 5] byte 20
[ 6] byte 30
[ 7] byte 3
[ 8] byte 47
[ 9] byte -91
[10] byte 52
[11] byte -12
[12] byte 2
[13] byte -6
[14] byte 11
[15] byte -24
[16] byte 41
[17] byte 98
[18] byte -124
[19] byte 52
Deficit = 21
Position (byte array) = 12

Reading 32 bits gets me: -1510145665

Should be apparently: -1510145696

byte[] array = {
  64, 1, -32, 1,
  100, 20, 30, 3,
  47, -91, 52, -12,
  2, -6, 11, -24,
  41, 98, -124, 52
};
ByteBuffer bArray = ByteBuffer.wrap(array);
bArray.order(ByteOrder.LITTLE_ENDIAN);
BitReader bitReader = new BitReader(bArray);
bitReader.readNBit(32);
bitReader.readNBit(32);
bitReader.readNBit(21);
int luku = bitReader.readNBit(32);

Luku == -1510145665, and I think it should be -1510145696.

Answer

47, -91, 52, -12 to little endian int => 11110100001101001010010100101111
2, -6, 11, -24 to little endian int => 11101000000010111111101000000010

After consuming 21 bits, the following 32 bits 10100101111111010000000101111111 which is 0xA5FD017F in hex and 2784821631 in decimal.

System.out.println(0xA5FD017F);
long l = 2784821631L;
System.out.println((int)l);

will both give you -1510145665. So the original code is correct.

Your changes to take the input byte array as little endian may not what you want when you are reading less than 32 bits at a time. You may have to read one byte at a time instead of using the existing readInt()

Leave a Reply

Your email address will not be published. Required fields are marked *