AWS Lambda With Dynamo DB Stream

Table of Contents

In this article, we will learn how we can use AWS Lambda with Dynamo DB stream. Dynamo DB Streams is a time-ordered sequence of events that reflects near-real-time operations in DynamoDB tables. Dynamo DB Streams can store the data for up to 24 hours. Any operations to be performed should be done within this time frame else the data will be lost. Dynamo DB streams are composed of different shards. Each Shard is made up of Records, where each record represents a single change in the stream’s table. Dynamo DB provides different options on how we want to get stream data as. The options are:

OLD_IMAGE:- Records before it was modified
NEW_IMAGE:- Records after it was modified
NEW_AND_OLD_IMAGES:- Records before and after it was modified
KEYS_ONLY:- Only the key attributes of the modified item.

AWS Lambda is an Amazon Web Service compute service that lets you run code without managing or provisioning servers. In Lambda, your code is executed on a high-availability compute infrastructure, and all of the compute resources are managed, including server and operating system maintenance, capacity provisioning, automatic scaling, and logging.

aws lambda with dynamo db stream — AWS Lambda with Dynamo DB Stream

Prerequisites

AWS account
Basic AWS Lambda and Dynamo DB concepts

How to use AWS Lambda with Dynamo DB streams?

Step 1: Create an IAM role

Go to AWS Console and Log In
In the search bar, type IAM
Select the IAM options displayed under Services section

After that AWS IAM page will appear. Then click the Create role button on the right side of the page so that we can create our custom role so that our lambda can access the dynamo DB and streams.

Now, let’s create our custom role. First, select a trusted entity for the role.
Here, let’s choose AWS Service under Trusted entity type and Lambda under User case as shown in the image.

Now, let’s add permissions to the role. We will be adding AmazonDynamoDBFullAccess policy and CloudWatchFullAccess policy so that our lambda has full access to the dynamo DB operations and can log the response in the cloud watch.

After adding permission, let’s provide a name to our role and finally create a role.

Step 2: Create a Lambda Function

Let’s create a lambda function using which we will log the Dynamo DB data on which the write operation is performed.
In the search bar, type lambda
Select the Lambda options displayed under Services section

After that AWS Lambda page will appear. Then click the Create function button on the right side of the page.

Now, let’s create a function named dynamoDBStreamTestFunction and select the above-created customDynamoDBStreamRole role under the Execution role section. Then click Create Function.

Now, let’s go to the newly created lambda detail page and modify the index.js file which will log the dynamo DB data on which a write operation is performed.

# index.js

exports.handler = async (event) => {
    // List all the records provided by Dynamo DB streams according to write operations
    console.log("Event:", event.Records);
};

Step 3: Create a Dynamo DB

Now, let’s create a dynamo DB with stream enabled so that we can perform a task using lambda when any write operation is performed in our DB.
In the search bar, type dynamodb
Select the DynamoDB options displayed under Services section.

After that Dynamo DB service page will appear. Then click the Create table button on the right side of the page.

Enter the desired table name, partition key, and sort key for the Dynamo DB.

After completing the above step, a new table named dynamo-stream-test will be created.
Now, let’s enable a stream on the table.
Click the dynamo-stream-test table and go to Exports and streams tab.
Then click enable on the bottom right of the page.

Now, let’s configure our stream properties.

After that stream will be enabled in our Dynamo DB table.
Now let’s create a trigger that will trigger lambda when a write operation is performed on any data of the table.

Now, let’s select a previously created lambda function so that the Lamba will be associated with our Dynamo DB stream.

Now, on every write operation on our dynamo-stream-test DB, the lambda dynamoDBStreamTestFunction will be triggered.

Step 4: Add DynamoDB record

Now, to create a new item, click the Explore items under the Tables section and choose our table name then click the Create item button on the right side of the page.

Now, add the following code in the Attribute section.

{
 "userId": "123456",
 "name": "Test User",
 "email": "test_user@yopmail.com",
 "age": "25",
 "address": "New York"
}

Now, let’s verify whether our Dynamo DB stream is working or not by checking the log added by our lambda above.
For that, let’s go to our lambda detail page and choose the Monitor tab.
Then click View logs in CloudWatch button.

Now, we will be redirected to the cloud watch page.

Under the Log streams tab, different log streams will be listed according to the event time.
Click a record to view the detail of the log stream.

Here, we have logged all the responses provided by the Dynamo DB stream according to the write operations.

Conclusion

In this article, we have learned how to use AWS Lambda with Dynamo DB streams. Dynamo DB also provides features like transactions and indexes for performing different DB operations. You can use this link to learn more about other AWS-related features.