Understanding Lambda Execution Models
Know the Services you are using; really know them. Understanding how they work and fail is essential for constructing a resilient serverless architecture.
This is the number one piece of advice I share with new Serverless Engineers. Knowing the AWS Services you plan to build with is essential. A vital aspect of this is how your Lambda is invoked and what this means. Lambda execution models can be broken down into three groups, Synchronous, Asynchronous and Event Source Mappings.
Synchronous Invocations
This is the simplest execution model — this is where all parties in the lambda execution chain are involved end to end in the transaction and wait for a response. In this model, error handling falls back to your consumers since failures are communicated directly via the lambda or service response. This means you consumers will be responsible for any retry behaviour on failure.
A common pitfall with synchronous invocations is your lambda timeout configuration which can be up to 15 minutes. You need to be careful setting this limit since there is no guarantee your clients will wait this long for the lambda execution to complete. The most common timeout error for synchronous invocations is when lambda is triggered by the API gateway which has a 29-second hard timeout limit. You need to make sure your lambda timeout configuration makes sense within your overall solution.
Since error handling falls back to consumers, the synchronous lambda invoke model is the simplest to code for. Being invoked synchronously means you must carefully consider rate limiting and throttling within your overall solution so that your lambda does not get over-run and cause problems for downstream services.
Asynchronous Invocations
Asynchronous invocations are where the caller of your Lambda does not wait for the outcome. They trigger the Lambda and then go on with whatever else they need to do; all they care about is whether the triggering of the Lambda was successful and nothing else about what your Lambda code is doing. Any errors from your Lambda invocation go nowhere since nobody is waiting for the result. Asynchronous invokes are great for enabling greater scale within your solutions at the expense of adding additional complexity when it comes to handling errors. With this invocation model, it is the responsibility of your Lambda code or configuration to take care of all error handling.
A common mistake serverless developers make with AWS Lambda and the asynchronous invoke model is expecting when their Lambda fails that the service performing the invocation will actually know something about the failure. This will never be the case — the service performing the invoke will only know about the success or failure of the actual invocation action, which returns immediately the Lambda function is triggered successfully. This is a common misunderstanding of how the Eventbridge Dead Letter Queues work — these queues will only capture the Eventbridge events where an asynchronous invoke of the target from the Eventbridge rule failed (after 24 hours of retries) not when the lambda code invoked returns an error response — subtle difference but an essential lesson in knowing how the AWS services you are using work!
AWS Lambda has introduced several nice features around the asynchronous event model to assist with lambda execution failures. By default, an asynchronous Lambda invocation allows for up to two retries on Lambda execution failure with some backoff and delay mechanisms you can configure. This gives you a very simple retry mechanism out of the box. You also have the option to configure a Dead Letter Queue for asynchronous invocation failures when the two retries ultimately fail to execute your code successfully. If this small out of the box retry and dead letter queuing is not enough, the Lambda service also introduced Lambda destinations for the asynchronous invocation model only.
Lambda Destinations
Lambda Destinations is a great feature that allows you to configure several destination types on either success or failure of the asynchronous invocation. You can forward success or failure events to multiple destinations: SNS, SQS, Lambda or Eventbridge Eventbus. This provides you with many options for dealing with failures for your asynchronously invoked lambdas. With Lambda Destinations the event content provided to the destination contains details about the request and response in JSON format, this differs from Dead Letter Queues, where Lambda only sends the content of the event, without details of the response.
Event Source Mappings
The Event Source Mapping is an AWS provided polling mechanism for triggering Lambda from a stream or queue which are not capable of invoking Lambda directly. This mechanism causes Lambda functions to be invoked synchronously with batches of records which can be configurable in size depending on the event source. As we have learned, the caller in the synchronous invoke model is responsible for handling any error response, and the Event Source mapping is no exception. The Event Source Mapping will handle errors in specific ways depending on the AWS Managed service being used — I recommend you read up on how each service behaves with Event Source Mapping failures — another example of knowing your services!
So the funny thing with Event Source mappings is that the lambda execution mode is a synchronous call, even though from a serverless solution perspective the interaction between the AWS service and callers is asynchronous in nature, i.e. my Lambda execution response returns to the AWS managed service and not to the caller or consumer. This to me behaves in an identical fashion to the asynchronous invocation model and I really wish that Lambda destinations were an option for this “asynchronous solution execution model”, I get the invocation is synchronous, but from an overall serverless solution perspective, it isn’t. I really think that Lambda Destinations for this invocation model being available would open up more flexibility for handling events.
AWS recently introduced filtering to Event Source mappings enabling messages from the source to be specifically filtered for your Lambda function — this is an amazing addition and I encourage you to check it out!
In this article, I have summarised the Lambda invocation models that every Lambda developer must know and understand to be able to successfully develop and deploy to the cloud.
