Jets Job Multiple Queues
Deploy Multiple Queues
To have Jets deploy and create multiple Queue
config/jets/deploy.rb
Jets.deploy.configure do
config.job.enable = true
config.job.default_queue.lambda.reserved_concurrency = 5
# Add additional queues. Note: default queue is always created
config.job.additional_queues = %w[urgent low_priority]
# Adjust settings for each queue
config.job.queues.low_priority.lambda.reserved_concurrency = 2
config.job.queues.urgent.lambda.memory_size = 2048
config.job.queues.urgent.lambda.reserved_concurrency = 10
end
This creates 2 additional queues. The default queue is always created. In the example, there’ll be 3 queues.
default urgent low_priority
Each queue has its own Lambda Function for processing. This allows you to individually control settings like reserved_concurrency and memory_size. You can manually use jets concurrency:set to tune the concurrency and then codify the settings.
Example Job with Queue
Here’s an example job
app/jobs/mailman_job.rb
class MailmanJob < ApplicationJob
queue_as :urgent
def perform(*args)
puts "Delivering: #{args}"
end
end
When you call the job, it will send it to the specific SQS queue.
$ rails console
> MailmanJob.perform_later("mail") # => urgent queue
> MailmanJob.set(queue: "low_priority").perform_later("candy") # => low_priority queue
Reference
The table below covers each setting. Each option is configured with config.OPTION. The config. portion is not shown for conciseness. IE: logger.level vs config.logger.level.
| Name | Default | Description |
|---|---|---|
| job.additional_queues | [] | Example: %w[urgent low_priority] |
| job.default_queue.lambda.reserved_concurrency | 5 | Reserved concurrency to use for Lambda function associated with default queue. |
| job.default_queue.properties | {} | Override the SQS Queue properties. See Generated Function SQS Queue and AWS::SQS::Queue. |
| job.enable | false | Enable Jets to create Job resources to handle ActiveJob work, like SQS Queue. |
| job.queue_defaults.lambda.memory_size | 1536 | Default reserved concurrency for memory_size lambda functions. Can be overwritten individually with config.job.queues.NAME.lambda.memory_size |
| job.queue_defaults.lambda.properties | {} | Default properties for the additional_queues lambda functions. Can be overwritten individually with config.job.queues.NAME.lambda.PROPERTY |
| job.queue_defaults.lambda.reserved_concurrency | 5 | Default reserved concurrency for additional_queues lambda functions. Can be overwritten individually with config.job.queues.NAME.lambda.reserved_concurrency |
| job.queue_defaults.lambda.timeout | 900 | Default timeout for additional_queues lambda functions. Can be overwritten individually with config.job.queues.NAME.lambda.timeout |
| job.queue_defaults.properties | {} | Default properties for the additional_queues. Can be overwritten individually with config.job.queues.NAME.PROPERTY. See AWS::SQS::Queue |
| job.queues.NAME.memory_size | 1536 | An example of how you can set individual additional queue the memory_size setting. |
| job.queues.NAME.reserved_concurrency | 5 | An example of how you can set individual additional queue the reserved_concurrency setting. |