šŸŒŸĀ Help others discover us,Ā star our GitHub repo!Ā šŸŒŸ

Mobile Menu

Connecting to AWS DocumentDB from a Lambda function

Adrian SmijuljTwitter
January 24, 2019

Few weeks ago, AWS announced their own NoSQL, MongoDB-compatible database called DocumentDB.

I must say, as a fan of MongoDB, I was immediately pumped! šŸ¤—

Okay, maybe that statement was a bit exaggerated šŸ¤Ŗ. But yeah, Iā€™ve always wanted to have a ā€œnatively supportedā€ MongoDB in the AWS cloud. I was actually secretly hoping this would happen when I saw that MongoDB was a platinum sponsor of the AWS re:Invent 2018, but at the time that did not happen.

Well, now itā€™s here and my team and I immediately started playing with it, primarily to see how it performs in conjunction with Lambda functions and to determine if it would be a good fit for us here at Webiny.

After Iā€™d created a basic DocumentDB cluster, the natural step was to try to connect to it. But, in order to do that, I soon realized there are a few things you need to know and take care of first.

So, in this post, I wanted to show you how to connect to a DocumentDB inside of your Lambda functions (using Node.js runtime) and also, as a small bonus (who doesnā€™t love bonuses šŸ˜„), how to do the same from your local machine.

Letā€™s dig in!

Before We Startā€¦

There are a couple of things you should consider before connecting to your DocumentDB cluster.

First of all, it is always deployed inside of a VPC. This means there isnā€™t a public endpoint via which you can connect from the outside world. For example, you cannot just connect to your cluster directly from your machine.

Amazon DocumentDB is virtual private cloud (VPC)-only and does not currently support public endpoints. Thus, you canā€™t connect directly to your Amazon DocumentDB cluster from outside of your VPC.

Amazon DocumentDB is virtual private cloud (VPC)-only and does not currently support public endpoints. Thus, you canā€™t connect directly to your Amazon DocumentDB cluster from outside of your VPC.

Also, upon connecting, except the usual username and password, you must also use a public key they provide in the form of a .pem file.

By default, a newly created Amazon DocumentDB cluster only accepts secure connections using Transport Layer Security (TLS). Before you can connect using TLS, you first need to download the public key for Amazon DocumentDB.

By default, a newly created Amazon DocumentDB cluster only accepts secure connections using Transport Layer Security (TLS). Before you can connect using TLS, you first need to download the public key for Amazon DocumentDB.

So make sure to download it before continuing. For more information on this topic, you can visit the official AWS and MongoDB docs.

Alright, now we have everything we need to know. Letā€™s see how to connect to DocumentDB from a Lambda function.

Establishing a Connection From a Lambda Function

Code-wise, the cool thing about DocumentDB is that you donā€™t have to install any additional JS libraries or SDKs. You use it like you would use a regular MongoDB, via the official NodeJS mongodb package.

So, setting up a connection shouldnā€™t be much of a problem if youā€™ve already done it before. You only have to make sure to pass the connection URI and the public key correctly into the MongoClient.connect method.

You can find the connection URI in the AWS DocumentDB Console:

max 1304 14d9nOiaP8kPklY5jKg7IBQ

Note that for our case, you only need ā€œthe baseā€ of the connection URI in the ā€œConnect to this cluster with an applicationā€ section. Basically, you just leave out the extra query parameters (since the Node.js mongodb driver will take care of the rest), like so:

mongodb://some_user:some_password@webiny-cloud.cluster-some-id.us-east-2.docdb.amazonaws.com:27017

And of course, donā€™t forget to replace <insertYourPassword> with your actual password.

Alright, now, letā€™s see how to use the connection URI and downloaded certificate in our code:

import { MongoClient } from "mongodb"; import caBundle from "raw-loader!./path/to/rds-combined-ca-bundle.pem"; let cachedDb = null; function connectToDatabase() { if (cachedDb) { return Promise.resolve(cachedDb); } return MongoClient.connect( process.env.MONGODB_URI, { ssl: true, sslCA: caBundle } ).then(db => { cachedDb = db; return cachedDb; }); } export const handler = async (event, context) => { const db = await connectToDatabase(); // Do something with it... };

Please note that, in my development flow, my code is compiled with Babel so there are things like async, await, and import. Iā€™m also using Webpack, which enables me to use raw-loader. With it, I can conveniently import the .pem file the same way I would any other .js file. But, if you donā€™t use Webpack in your development flow, you can just copy/paste the whole content of the downloaded .pem file into a single .js file, like so:

export default `-----BEGIN CERTIFICATE-----MIID9DCCAtygAwIBAgIBQjANBgkqhkiG9w0BAQUFADCBijELMAkGA1UEBhMCVVMxEzARBgNVBAgMCldhc2hpbmd0b24xEDAOBgNVBAcMB1NlYXR0bGUxIjAgBgNVB...`;

Also, make sure to use dotenv if youā€™re not already, since using the environment variables is the most reliable and secure method for storing sensitive configuration values.

Alright, we are ready to test our Lambda function inside the AWS!

max 922 1xKwXQm svYdSxOo2gitx1w

I assume you already got the ā€œdeploymentā€ part figured out. If not, serverless framework is certainly a great way to do it.

If we try to run our deployed function via the Lambda Management Console, you will notice that it times-out!

Well, the DocumentDB connection cannot be established because, as already mentioned, our Lambda function and DocumentDB cluster must reside in the same VPC. To do that, just scroll down to the Network section and select the correct VPC, its subnets, and an adequate security group.

In my case, I had everything set up in my default VPC:

max 1374 1vCwlyd7lByGmFojW9r3pJw

Now, if you try to run the same function again, the connection should be established successfully!

max 800 01G9mrCEkdWaKJTHX

Connection Management

In a serverless world, connection management is unfortunately still something that might give you headaches.

If youā€™ve noticed, in our Lambda function, we actually never ended the established DocumentDB connection, because we wanted to re-use it on every following call. But, after some time, this might cause problems because a number of active connections will rise, while the old connections are still alive (and not doing anything), and eventually you hit the limit of active connections. At that point, new connection attempts will be rejected, which is certainly undesirable.

You can check the current state of connections by executing the following command inside of your DocumentDB cluster:

db.serverStatus().connections // Outputs the following: { "current" : 16, "available" : 1143, "totalCreated" : 511 }

The same problem can also be found in the RDS world, where there are some solutions to mitigate this, like e.g., the serverless-mysql lib that actually has a built-in connection manager which handles this for you. Additionally, Serverless Aurora also received the Data API, which abstracts connection management from us and let us interact with the database over an HTTP API.

At Webiny, we will closely monitor this for now. If required, I might even try to tackle this problem in my spare time, in the form of a separate NPM package.

Weā€™ll see what time brings, but who knows; maybe weā€™ll see a serverless ā€” Data API ā€” powered DocumentDB in the near future? šŸ˜‰

To finish this post, letā€™s see how we can establish a DocumentDB connection from a local machine.

Establish a Connection From Your Local Machine

If you need to directly connect to the DocumentDB cluster from a local machine, you will need to do it via a jumpbox / SSH tunneling.

Itā€™s actually pretty simple to setup. You only need to create a simple EC2 instance (t2.micro will suffice) in the same VPC. Once itā€™s up and running, fire up your favorite MongoDB client; I like to use Studio 3t.

Make sure to correctly enter all of the necessary information and to attach the downloaded certificate file.

max 1100 1ze99Gqhq0R6yVzkHSwRRuw

max 1106 1HBAbplqdh4q1H5FGOCbmjw

max 1106 1Jr20JptKtisCs 3nCI LbA

max 1110 10 nmkIUe81f0V06vNCzNrA

With everything entered correctly, you should be able to connect successfully to your DocumentDB cluster.

Final Thoughts

Except the mentioned connection management issue, everything else seems to be working pretty well. I could write about our initial experiences, but I think I might save that for another post.

Here at Webiny, we are certainly excited about the newest AWS addition and weā€™ll be using it to power our serverless CMS, which you will be able to check out here in a couple of weeks from now. In the meantime, you can follow us via Twitter if you want to know more.

I hope this article has helped you to successfully overcome these initial steps in your Lambda / DocumentDB journey. If you have any questions or you feel Iā€™ve maybe forgotten to mention something, feel free to let me know!


Thanks for reading! My name is Adrian and I work as a full stack developer at Webiny. In my spare time, I like to write about my experiences with some of the modern frontend and backend web development tools, hoping it might help other developers. If you have any questions, comments or just wanna say hi, feel free to reach out to me via Twitter.

Find more articles on the topic of:AWSServerlessDocumentDBMongoDBNodeJS

About Webiny

Webiny is an open source serverless CMS that offers you all the enterprise-grade functionalities, while keeping your data within the security perimeter of your own infrastructure.

Learn More

Newsletter

Want to get more great articles like this one in your inbox. We only send one newsletter a week, don't spam, nor share your data with 3rd parties.

Webiny Inc Ā© 2024
Email
  • We send one newsletter a week.
  • Contains only Webiny relevant content.
  • Your email is not shared with any 3rd parties.
Webiny Inc Ā© 2024
By using this website you agree to our privacy policy
Webiny Chat

Find us on Slack

Webiny Community Slack