Put records in batches to kinesis firehose using aws lambda in java

I need to put data in batches to firehose using AWS Lambda written in Java.
The data is a list of strings: [“john”, “saul”, “paul”, “jonas”].
I know how to do this in python but I am struggling in Java.

Following are the steps I need to perform in Java:

1). Build a firehose client.

2). Build a batch of data.(say the batch size is 2)

3). Put data in firehose.

4). get the response in firehose and extract which records failed to be delivered.

How to write code for all these steps in Java?

Answer

This use case can be done by using the Lambda Java runtime API and the Amazon Kinesis Data Firehose Java API. First, you need to create a Lambda function using the Java runtime API. If you are not aware how to perform this task, may I suggest you read this AWS doc:

Using AWS Step Functions and the AWS SDK for Java to build workflows that sends notifications over multiple channels

This doc teaches you how to write a Lambda function by using the Java runtime API and how to handle IAM roles required to invoke AWS Services from a Lambda function. For your use case, you would need to create an IAM role that has permission to use Amazon Kinesis Data Firehose.

In addition, this Lambda function can invoke other AWS Services using various AWS SDK for Java APIs. In this example, Pinpoint, SES, and SNS services are invoked to perform the specific functionality. For your use case, you would need to use the Amazon Kinesis Data Firehose API.

To achieve want you want to do, make sure you look at the examples for the Amazon Kinesis Data Firehose API V2.

So to sum up, to achieve your use case, you need to write a Lambda function using the Java runtime API and use the Amazon Kinesis Data Firehose API from within the Lambda function, similar to how the above tutorial uses the other AWS Service Java APIs.

Leave a Reply

Your email address will not be published. Required fields are marked *