Concurrency

You can configure Reserved and Provisioned Concurrency for Jets Lambda Functions.

Controller Concurrency

To configure Concurrency for controller requests.

config/jets/deploy.rb

Jets.deploy.configure do
  config.lambda.controller.provisioned_concurrency = 1
  # config.lambda.controller.reserved_concurrency = 25
end

Avoiding the Cold Starts

You can use Provisioned Concurrency to avoid cold starts. This is because Lambdas with Provisioned concurrency are always running. In a sense, they are like regular servers. However, there’s a cost associated with Provisioned Concurrency. The cost depends on the Function size. The AWS Lambda Console provides a useful cost estimate under the Provisioned Concurrency setting.

For example, a 1.5GB Lambda function has a baseline cost of $16.74/mo. In addition to that baseline, you are charged for provisioned requests. Provisioned requests offer a lower rate than on-demand requests. Depending on how frequently your Lambdas are requested, provisioned concurrency costs may be more or less than on-demand concurrency costs. For many, on-demand is the more cost-effective option.

Note: Even with Provisioned Concurrency, there is still a smaller cold start when new functions are deployed. The cold start is eliminated when AWS Lambda cycles the function.

Jets also has a Prewarming feature that will regularly ping your Jets Controller Lambda function to reduce the likelihood of a cold start. This can be a more cost-effective way to minimize the effect of cold starts.

Reserved vs Provisioned Concurrency

Summary:

• Reserved concurrency: Guaranteed and maximum number of function “instances” to run from the AWS Account default concurrency pool, usually 1,000. There is no extra cost for this. See: Reserved Concurrency Concept
• Provisioned concurrency: Pre-initialized always running function “instances”. There are extra costs to use this.

Interestingly, Reserved concurrency can be used to limit requests and control costs since AWS Lambda bills for both request and duration. AWS Lambda bills for the processed requests regardless of whether or not it returns its TooManyRequestsException, but the duration is shorter. This is similar to traditional EC2 instances. Even if an EC2 instance returns an error, you still get charged for the request. Generally, users still stop requesting after getting error responses.

Provisioned Concurrency does not affect limiting Lambda scaling. If more scaling is required, Lambda uses additional on-demand Lambdas.

Related

• AWS Docs: Configuring provisioned Concurrency
• AWS Docs: Understanding and visualizing Concurrency
• AWS Blog: Set Concurrency Limits on Individual AWS Lambda Functions

Reserved Concurrency Defaults

The Jets Reserved Concurrency defaults are

  config.lambda.function.reserved_concurrency = 5
  config.lambda.controller.reserved_concurrency = 25

The defaults provides a safeguard to mitigate runaway costs in the event of a DDOS. Since AWS Lambda can essentially scale limitlessly. Setting reserved concurrency limits scaling.

The controller limit is configured at 25 to handle scenarios where a page initiates multiple simultaneous Lambda requests, like in photo galleries. This limit of 25 matches the typical pagination settings found in libraries such as Kaminari. This default avoids surpassing Lambda’s reserved concurrency capacity, which could otherwise result in a 429 “Too Many Requests” error.

If your application doesn’t generate parallel requests, you can likely reduce the controller.reserved_concurrency setting to a much lower value. A setting of controller.reserved_concurrency = 5 is fine for most cases. It just depends on how your app works. AWS Lambda efficiently reuses functions after their initial cold start. It will only scale up if multiple requests arrive simultaneously in parallel.

AWS Account-Level Lambda Concurrency Limit

It is important to know that AWS accounts usually have default account Reserved Concurrency of 1,000. Using Reserved Concurrency takes away from your overall AWS Account limit “pool”. 100 is unreservable.

AWS Docs: Concurrency quotas

Lambda always reserves 100 units of concurrency for your functions that don’t explicitly reserve concurrency.

Hence, the most you can reserve is 900 of 1,000. This means if you can have 36 app deployments with the Jets defaults (36 * 25 = 900). To help simplify the ballpark calculation and help explain, we’re not including Jets Jobs or Events Functions Reserved Concurrency.

You can check your Lambda Concurrency Limit.

• With the AWS Lambda Console. This nice because it’s readily available. It makes it easier to spot when you are nearing your account limit.
• The Service Quotas Console. You can request for an increase here. You can also open up an AWS support ticket to request an increase.
• Here are more ways from the AWS Docs to check: AWS service quotas

Here’s also an AWS CLI command to check.

❯ aws lambda get-account-settings | jq
{
  "AccountLimit": {
    "TotalCodeSize": 80530636800,
    "CodeSizeUnzipped": 262144000,
    "CodeSizeZipped": 52428800,
    "ConcurrentExecutions": 1000,
    "UnreservedConcurrentExecutions": 577
  },
  "AccountUsage": {
    "TotalCodeSize": 2576140188,
    "FunctionCount": 77
  }
}

• Related: Stack Overflow: Can I limit concurrent invocations of an AWS Lambda?

Unlimited Reserved Concurrency

If you need to remove the limits, you can set both configs to nil.

config/jets/deploy.rb

Jets.deploy.configure do
  config.lambda.controller.reserved_concurrency = nil  # remove limit for controller requests
  config.lambda.function.reserved_concurrency = nil    # remove limit for functions and events
end

This removes the limit for both Controller requests and Events.

When no Reserved Concurrency is specified, the default AWS Lambda behavior is to scale up to the limit of the AWS account. AWS Accounts have a Concurrent execution limit of 1,000. Note that brand-new AWS accounts have reduced concurrency and memory quotas. See: Lambda Quotas.

This means that when reserved_concurrency = nil, AWS Lambda can scale up to 1,000 concurrent requests. This is a soft limit. You can request a limit increase with an AWS support ticket.

Database Connections Limit

It is recommended to not configure unlimited Reserved Concurrency and rely on your AWS account limit. Your app will likely hit another bottleneck, a common one being the database connection limit.

With traditional EC2 servers, you would normally configure the web server, IE: puma threads, to be less than the available database connections. Otherwise, you would run out of DB connections. Similarly, with AWS Lambda, you should limit the Reserved Concurrency to the number of available DB connections. Otherwise, Lambda will scale beyond the number of available DB connections and cause errors.

Monitoring Concurrency

It is important to monitor Thottles and Total concurrent executions. You can see these metrics in the Lambda Console Monitor tab. If throttling is occurring, users will see this error:

429 Too Many Requests

This means the default Reserved Concurrency is too low for your app. You want the metrics to be:

• Thottles to always be 0.
• Total concurrent executions to always be less than the reserved_concurrency config.

If your app is hitting the limit, you can increase Reserved Concurrency to resolve the issue. You can also try to speed up your app. If your app logic is faster, AWS Lambda can reuse the same Lambda to serve requests instead of provisioning new concurrent ones. You might also be able to change the way your app works to make fewer requests in parallel. For example, if you have a photo page with 1,000 images and no pagination, then the page triggers all 1,000 images to load simultaneously. The photos could be paginated or lazy loaded.

Important: The “429 Too Many Requests” error will not show up in your AWS Lambda Function Logs. This is because AWS Lambda throttles and cuts off the request before it reaches your Lambda function. That’s the point of throttling.

Reserved > Provisioned Concurrency

An important note from the AWS Provisioned concurrency Docs

If the amount of provisioned Concurrency on a function’s versions and aliases adds up to the function’s reserved Concurrency, then all invocations run on provisioned Concurrency. This configuration also has the effect of throttling the unpublished version of the function ($LATEST), which prevents it from executing. You can’t allocate more provisioned Concurrency than reserved Concurrency for a function.

This means you should never configure reserved_concurrency to be equal or less than provisioned_concurrency, otherwise you’ll get a Rate Exceeded ReservedFunctionConcurrentInvocationLimitExceeded error.

Events Concurrency

You can also configure Concurrency for Lambda Functions from Jets Events. There are multiple ways to configure it. You can configure it app-wide with a config.

config/jets/deploy.rb

Jets.deploy.configure do
  config.lambda.function.reserved_concurrency = 2
  # config.lambda.function.provisioned_concurrency = 1
end

You can also configure it on a per-method basis in the code itself.

app/events/cool_event.rb

class CoolEvent < ApplicationEvent
  reserved_concurrency 2
  # provisioned_concurrency = 1

  rate "10 hours"
  def handle
    puts "Do something with event #{JSON.dump(event)}"
  end
end

Note, you can also configure Provisioned Concurrency for events, but there’s little point since Event handling is typically async. A cold-start is not the end of the world. Using only on-demand Concurrency is more cost-effective.

Reference

The table below covers each setting. Each option is configured with config.OPTION. The config. portion is not shown for conciseness. IE: logger.level vs config.logger.level.

See Full Config Reference