Aws S3 TransferService Upload failing without errors

I’ve implemented the AWS core and S3 SDK to handle file upload. Unfortunately the upload is not working properly sometimes. (it seems to be only under wifi connection but i am no 100% sure about that fact)

For some unknown reason the aws service is receiving a successful response even though the file hasnt finished uploading and then everything hangs.

I can see that the Transfer Service is started, and for example the first file starts uploading properly and finishes but the second one never finishes uploading and the TransferService stops silently without me receiving any error through transfer listeners…i’ve enabled debug logs for aws to see if i can get more info via

   java.util.logging.Logger.getLogger("com.amazonaws").setLevel(java.util.logging.Level.FINEST);

And here is an upload log in which the first file upload succeeded and the second one failed – i just start the next file when the first one notifies its completed. (without package names and timestamps for easier analysis):

D/AwsUploadManager: Uploading file: /storage/emulated/0/folder/profile20160219_123233.jpg
D/AwsUploadManager: Uploading under key: 56cb00d2121bbfcc0e8c68e7-843f1417-3f44-4d76-915d-f3a72c56a7d4
D/AwsUploadManager: AwsUploadManager 56cb00d2121bbfcc0e8c68e7-843f1417-3f44-4d76-915d-f3a72c56a7d4
D/TransferSerivce: Starting Transfer Service
D/TransferSerivce: Loading transfers from database
D/TransferSerivce: 1 transfers are loaded from database
W/TransferSerivce: Transfer has already been added: 6
D/TransferSerivce: Network connected: true
D/AwsUploadManager: onStateChanged IN_PROGRESS
D/AwsUploadManager: onProgressChanged id= 6 ::bytesCurrent0 bytesTotal16917
D/com.amazonaws.request: Sending Request: PUT.........
D/libc: [NET] android_getaddrinfofornet+,hn 41(0x6369616f2d696d),sn(),hints(known),family 0,flags 4
D/libc: [NET] android_getaddrinfofornet-, err=8
D/libc: [NET] android_getaddrinfofornet+,hn 41(0x6369616f2d696d),sn(),hints(known),family 0,flags 1024
D/libc: [NET] android_getaddrinfofornet-, pass to proxy
D/libc: [NET] android_getaddrinfo_proxy+
D/libc: [NET] android_getaddrinfo_proxy get netid:0
D/libc: [NET] android_getaddrinfo_proxy-, success
D/AwsUploadManager: onProgressChanged id= 6 ::bytesCurrent16917 bytesTotal16917
D/com.amazonaws.request: Received successful response: 200, AWS Request ID: 85952ED0E0FCF444
D/AwsUploadManager: onProgressChanged id= 6 ::bytesCurrent16917 bytesTotal16917
D/AwsUploadManager: onStateChanged COMPLETED
D/AwsUploadManager: Upload finished: 56cb00d2121bbfcc0e8c68e7-843f1417-3f44-4d76-915d-f3a72c56a7d4
D/AwsUploadManager: Uploaded url: https://server-name.s3.eu-central-1.amazonaws.com/56cb00d2121bbfcc0e8c68e7-843f1417-3f44-4d76-915d-f3a72c56a7d4
D/AwsUploadManager: Uploading file: /storage/emulated/0/folder/profile20160219_122512.jpg
D/AwsUploadManager: Uploading under key: 56cb00d2121bbfcc0e8c68e7-7ea27f9e-0e48-4fde-81a1-7d72383ada2d
D/AwsUploadManager: AwsUploadManager 56cb00d2121bbfcc0e8c68e7-7ea27f9e-0e48-4fde-81a1-7d72383ada2d
D/AwsUploadManager: onStateChanged IN_PROGRESS
D/AwsUploadManager: onProgressChanged id= 7 ::bytesCurrent0 bytesTotal1930575
D/com.amazonaws.request: Sending Request: PUT https://server-name.s3.eu-central-1.amazonaws.com /56cb00d2121bbfcc0e8c68e7-7ea27f9e-0e48-4fde-81a1-7d72383ada2d Headers: (x-amz-decoded-content-length: 1930575, Content-MD5....
D/AwsUploadManager: onProgressChanged id= 7 ::bytesCurrent655360 bytesTotal1930575
D/AwsUploadManager: onProgressChanged id= 7 ::bytesCurrent1048576 bytesTotal1930575
D/AwsUploadManager: onProgressChanged id= 7 ::bytesCurrent1310720 bytesTotal1930575
D/AwsUploadManager: onProgressChanged id= 7 ::bytesCurrent1703936 bytesTotal1930575
D/com.amazonaws.request: Received successful response: 200, AWS Request ID: DE9670A8450D4BAB
**D/TransferSerivce: Stop self**  --And here amazon stopped self without notifying registered listeners about transfer..

The code used to invoke the upload:

TransferObserver transferObserver = mTransferUtility.upload(BUCKET_NAME, fileKey, file);
transferObserver.setTransferListener(new TransferListener() {...}

Everything is standard from samples, except that i am using a standard auth method with pub/sec key credential instead of Cognito pool.

  mAwsCredentials = new BasicAWSCredentials(mContext.getString(R.string.aws_accress_key), context.getString(R.string.aws_secret_key));
  mAmazonS3Client = new AmazonS3Client(mAwsCredentials);
  mAmazonS3Client.setRegion(com.amazonaws.regions.Region.getRegion(REGION));
  mTransferUtility = new TransferUtility(mAmazonS3Client, mContext);

Btw this issue sometimes happens on first file uploading sometimes on 3rd, its not related to the number of files i try to upload. Also sometimes the onProgressChanged is fired just once before the upload hangs and i can see Received successful response and Stop Self afterwards.

The Question: Has anyone experienced this issue before and what could it be related to? I am experiencing this relatively often (every second time) and thus am not sure if its network related. Even if it is, i was expecting amazon to properly handle the network errors or at least notify registered listeners. Plus i’ve went through some source code and got a feeling by their docs that they are handling network issues properly so i am hopping i am doing something terribly wrong on my side.

Answer

A recent change in v2.2.12 keeps only a weak reference of registered transfer listeners. User is responsible to hold the transfer listener till the transfer finishes. The intention is to prevent memory leak. However its usage isn’t clearly documented. We plan to tweak it in future releases. Stay tuned.