How To Easily Read Files From S3 Using Lambda?

Table of Contents

In this article, we will learn how to read files from S3 using lambda and node js. AWS Lambda is an Amazon Web Service compute service that lets you run code without managing or provisioning servers. In Lambda, your code is executed on a high-availability compute infrastructure, and all of the compute resources are managed, including server and operating system maintenance, capacity provisioning, automatic scaling, and logging.

Amazon S3 is a web-based cloud storage service provided by AWS (Amazon Web Services) that stores files in buckets. Amazon S3 provides durability and security for the files stored within the buckets. There are no limitations on the number of files stored in Amazon S3. The Amazon S3 charges only for the resources you used, there are no hidden charges.

Prerequisites

AWS account
Basic AWS Lambda and AWS S3 concepts

How to read files from s3 using lambda?

Step 1: Create S3 Bucket

Go to AWS Console and Log In
In the search bar, type S3
Select the S3 options displayed under Services section.

read files from s3 using lambda — s3 service

After that AWS S3 page will appear. Then click the Create bucket button on the right side of the page so that we can create a new bucket where we will be uploading our test file whose content will be read using AWS lambda.

Now, let’s create an S3 bucket.
Here, enter your desired name and region of the bucket, we will be using scanskill-file-read-test and ap-south-1 respectively.

Now, upload any text file in the above bucket. We will be using the filename demo.txt with the following content:

Amazon S3 is a web-based cloud storage service provided by AWS (Amazon Web Services) that stores files in buckets. Amazon S3 provides durability and security for the files stored within the buckets. There are no limitations on the number of files stored in Amazon S3.

Step 2: Upload the file to S3

Now, let’s upload the above-prepared data to the S3.
Let’s go to the S3 dashboard and click the bucket created. In our case, the bucket will be named scanskill-file-read-test.
Then, click the Upload button on the right side of the page. You can also drag and drop your file inside the bucket.
Then, click the Add Files option and select the file you want to upload to the S3 bucket.

Step 3: Create a Lambda Function

Go to AWS Console and Log In
In the search bar, type lambda
Select the Lambda options displayed under Services section

After that AWS Lambda page will appear. Then click the Create function button on the right side of the page.
Then, create a function named readFileContentFunction . Then click Create Function.

It will take some time to create a lambda function and you will be redirected to the following page.
Then, let’s add a new role to our lambda so that it can access the s3 features.
To do that click the Configuration tab and click the role provided under the Execution Role section.

Then, you will be redirected to the IAM roles page.
Here, click Add permissions the button and then choose Attach policies the option.

Now, let’s search for the s3 bucket policies to be attached to the lambda.

Now, let’s modify the index.js a file of our lambda. So, that it will read the content of the file present in the s3 bucket and log its content when the Lamda is executed.

var AWS = require('aws-sdk');
const S3Config = {
  accessKeyId: //your aws access key
  secretAccessKey: //your aws secret key,
  region://your bucket region,
  signatureVersion: "v4",
};
var s3 = new AWS.S3(S3Config);

exports.handler = (event) => {
  var bucketName = "scanskill-file-read-test; //your bucket name
  var filename = "demo.txt"; //your sample file name
  var params = { Bucket: bucketName, Key: filename };
  s3.getObject(params, function (err, data) {
    if (!err)
      console.log("Content", data.Body.toString())
    else
      console.log(err);
  });
};

Now after executing the lambda created above, the file content will be logged in the console.

Conclusion

In this article, we have learned how to read files from S3 using lambda and node js