How to write an S3 object to a file?

What’s the fastest way to write an S3 object (of which I have the key) to a file? I’m using Java.

Answer

While IOUtils.copy() and IOUtils.copyLarge() are great, I would prefer the old school way of looping through the inputstream until the inputstream returns -1. Why? I used IOUtils.copy() before but there was a specific use case where if I started downloading a large file from S3 and then for some reason if that thread was interrupted, the download would not stop and it would go on and on until the whole file was downloaded.

Of course, this has nothing to do with S3, just the IOUtils library.

So, I prefer this:

InputStream in = s3Object.getObjectContent();
byte[] buf = new byte[1024];
OutputStream out = new FileOutputStream(file);
while( (count = in.read(buf)) != -1)
{
   if( Thread.interrupted() )
   {
       throw new InterruptedException();
   }
   out.write(buf, 0, count);
}
out.close();
in.close();

Note: This also means you don’t need additional libraries

Leave a Reply

Your email address will not be published. Required fields are marked *