Java NIO/ MappedByteBuffer and map, read part of file in chunks

I want to read parts of a large file in a loop. I had to just read the entire file but that didn’t work, I was getting an exception that the file is too large. I changed my code to the listing below. The code below only reads in the first chunk. What do I need to change to move to the next chunk.

   final FileInputStream fis = new FileInputStream(f);
    final FileChannel fc = fis.getChannel();
    final long sizeRead = fc.size() < defaultReadBufferSize ? fc.size() : defaultReadBufferSize;
    final MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, sizeRead);        
    while (bb.hasRemaining()) {                        
        final CharBuffer cb = decoder.decode(bb);
        this.search(f, cb);   
        System.out.println("============>" + cb.length());
        System.out.println("============>" + bb.hasRemaining());            
    }        
    fc.close();

Answer

The problem you have is that character encoded data cannot be accessed this way. i.e. you need to know where the boundaries between characters are.

The cost of accessing the file and character decoding it is likely to be far more expensive than how you read it, so I would use a BufferedReader which will be much simpler as well.

e.g. say you want to read from the 1000th byte. You can do that but you won’t know if the 1000th byte is part of a multi-byte character or not.

If you can assume all characters are bytes, the whole issue is much simpler and you don’t need a CharBuffer, you can access the ByteBuffer directly which would be much faster.

Leave a Reply

Your email address will not be published. Required fields are marked *