With the rise of serverless architecture Lambda along with API gateway is becoming automatic choice over EC2 or ECS to developers for computing layer of their application. For database layer, DynamoDB is a good option due to a managed service but I feel it is not fit for all applications due to its limited data query features. AWS RDS service is another good option but it is a relational database service and modern applications are preferring NoSQL database or a mix of RDBMS and NoSQL for the database layer. MongoDB is a good option to use with Lambda. However, it is a little bit tricky to make MongoDB on EC2 accessible to Lambda functions. Following is a step by step guide to setup MongoDB with Lambda function.
Step 1: Create VPC
To enable access to MongoDB EC2 instance from Lambda function we need to configure lambda with VPC.
Step 1.1: Go to VPC Console
Step 1.2: Create new VPC.
Give a suitable name like “lambda-to-mongo” and specify IPv4 CIRD range. For development purpose range specified in the image is good enough but for production, you need to consider concurrent usages, use large CIDR block.
Step 1.3: Create a public subnet
This subnet will contain our MongoDB instance hence name it carefully so that you can identify while launching the instance. CIDR range for this subnet can be small as it will contain only MongoDB instances.
Step 1.4: Create a private subnet
This subnet will be linked to lambda functions hence name it carefully so that you can identify while creating lambda functions. CIDR range for this subnet needs to be defined based on our usage.
Step 1.5: Create internet gateway
Internet gateway is required to allow internet access to and from VPC.
Step 1.5: Attach internet gateway to VPC
Step 1.6: Private route table
Main route table create with VPC is by default private as it only allows network traffic within the VPC only. We just rename it to identify clearly.
Step 1.7: Create public route table
Create a new route table and make it public by attaching internet gateway route.
Step 1.8: Associate public route table to public subnet
To allow internet access to and from public subnet associate public route table with the public subnet.
Step 1.9: Create security group for lambda
This step is necessary to group all lambda containers which can have any private IP from our private subnet IP range. By grouping, we can allow inbound traffic from lambda containers to our MongoDB instance on MongoDB port.
Step 1.10: Create security group for mongo instance
This group allows access to mongo instance from ssh and MongoDB port.
Step 1.10: Modify inbound rules of mongo security group
We will add 3 inbound rules to the group.
1. Allow MongoDB access from lambda security group on 27017 port
2. Allow ssh access from your machine on 22 port. for this, you need to get your machine public IP address (recommended) or can allow for all 0.0.0.0/0
3. Allow MongoDB access from your machine on 27017 port. for this, you need to get your machine public IP address. This rule is optional.
Step 2: Launch EC2 instance
Launch an instance for MongoDB
Step 2.1: Launch instance
Step 2.2: Select AMI
Select AMI of your choice. I prefer Amazon Linux AMI.
Step 2.3: Select instance type
I am selecting free tire T2.micro instance. You can select any instance type depending on your use case. Storage optimizes instance types are more suitable for MongoDB workload.
Step 2.4: Configure instance details
Do this step very carefully. Select the VPC created in step 1.2, Public subnet created in step 1.3 and enable auto assign public IP.
Step 2.5: Add storage
Change storage if you want to use this instance for production for testing purpose default storage will be sufficient.
Step 2.6: Add Tags
And a Name tag to your instance to identify it.
Step 2.7: Configure security group
Assign the security group created for mongo instance in step 1.10
Step 2.8: Review instance details
Review instance details like VPC, Subnet, security group etc. once before clicking Launch. If anything is not appropriate please go back and correct it.
Step 2.9: Download key pair
Download key pair to ssh into the instance. This key is required for shell access of the instance, if lost we need to relaunch the instance.
Step 2.10: Launch status
Click view instance
Step 2.11: Instance details
After 1-2 min instance will be ready for use. Note down the Public and Private IP address as marked in the screenshot.
Step 3: Install MongoDB
Step 3.1: Use the following commands to ssh into the instance and install MongoDB
Step 3.2: Create a sample data
Step 4: Create Lambda
I am creating a lambda function using nodejs you can use any other language supported by lambda. All steps remain same except 4.1 and 4.2 which will be replaced by your code structure.
Step 4.1: Create code structure on local system
Execute the following commands to create file structure. I am assuming node is already setup on your system
Step 4.2: Lambda code
Copy the folloing code into index.js. Replace PRIVATEIP with the private ip captured in step 2.8
Step 4.3: Create zip file for upload
while inside the code directory run following commond
Step 4.4: Create lambda function from AWS Console
Step 4.5: Provide basic details
Provide a name, runtime and select Create a custom role for Role. It will open a new tab to create a custom role.
Step 4.6: Create custom role
Edit policy document and past the follwing policy
Step 4.7: Select created role and save
Step 4.8: Upload code
Step 4.9: Configure VPC
Step 4.10: Test lambda
You can see the successful connection and data in the screenshot.