In our daily work we often have the need to plan periodic or recurring activities. In the past this activity has always been managed using cron.
This obliges us, always, to have a server on which to schedule activities.
In recent years we have been using Serverless services with increasing frequency. We will do even more in the future. So we asked ourselves the question of how to transform our crontab into crontabless!
We immediately looked for the services offered by Amazon AWS, and we found what we needed.
Combining Cloudwatch Events Rules and the Lambda functions, we can schedule events at any time.
Cloudwatch Events Rules permits to create rules, which are very similar to Unix-like crontab. With appropriate permissions, when an event rule is executed, we can invoke a Lambda function.
The Lambda function performs the operations which we indicate, or can call other AWS services.
By our choice we decided to create the various events rules with Ansible, but it is possible to do the same operations as GUI. Let’s see a simple example, using the same procedure explained in this article.
The first file that has to be created is:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
vars_manage_cron.yml install_libs: - datetime include_local_files: - "{{ function_name }}" - libs/amqp_utility - libs/nuvola_cron arn_role: arn:aws:iam::0123456789:role/MyLambdaFunction cron_cloudwatch_rules: "no" #MB lambda_memory: 128 #seconds lambda_timeout: 120 description: "{{ function_name }} function management" input_parameters_lambda: env: prod # Enables execution in vpc vpc_subnet: - "nuvola_{{ env }}_local_lambda_0" - "nuvola_{{ env }}_local_lambda_1" vpc_security_group: - "nuvola_{{ env }}_local_sg" region: "{{ default_region }}" cron_cloudwatch_rules_nuvola: - { name_cron: 'send-message-broker', schedules: 'cron(00 11 * * ? *)', input_event: '{{ {"cron_name":"send-message-broker","data":"null"} }}' } - { name_cron: 'download-report-server', schedules: 'cron(15 13 * * ? *)', input_event: '{{ {"cron_name":"download-report-server","data":"null"} }}' } - { name_cron: 'invokes-lambda-store-data', schedules: 'cron(0/5 06-13 ? * MON-SAT *)', input_event: '{{ {"cron_name":"invokes-lambda-store-data","data":"null"} }}' } - { name_cron: 'connect-ssh-server', schedules: 'cron(00 04 ? * MON-FRI *)', input_event: '{{ {"cron_name":"connect-ssh-server","data":"null"} }}' } |
With a list of dictionaries we can create a lot of events rules at once.
To understand how it works read the article “Lambda function manager”.
In particular, in the main.yml file the task:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
main.yml - name: LAMBDA FUNCTIONS | Creation rules cloudwatch cloudwatchevent_rule: name: "{{ function_name }}" schedule_expression: "{{ cron_cloudwatch_rules }}" state: present description: "Run {{ function_name }} scheduled task" region: "{{ region }}" targets: - id: "{{ function_name }}" arn: "{{ lambda_function.results.0.configuration.function_arn }}" input: '{{input_parameters_lambda | to_json}}' when: cron_cloudwatch_rules != "no" |
Then, it is a good idea to create the code needed to create the events rules on AWS. As mentioned, we use Ansible to do this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
hook_post_manage_cron.yml - name: LAMBDA FUNCTIONS | Creation rules cloudwatch for nuvola cron cloudwatchevent_rule: name: "cron_{{ item.name_cron }}" schedule_expression: "{{ item.schedules }}" state: present description: "Cron {{ item.name_cron }}" region: "{{ region }}" targets: - id: "{{ function_name }}" arn: "{{ lambda_function.results.0.configuration.function_arn }}" input: '{{item.input_event | to_json}}' with_items: "{{ cron_cloudwatch_rules_nuvola }}" |
Finally, the Lambda code to make our manage_cron.py operations:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
manage_cron.py #!/usr/bin/python import boto3 import os from amqp_utility import Queue from nuvola_cron import NuvolaCron def handler(event, context): channel_name = os.environ['channel'] amqp_host = os.environ['amqp_host'] persistent_queue = os.environ['persistent_queue'] data_creazione=event['data'] nuvola_cron = NuvolaCron(event['cron_name']) message = "my message" try: append_messages = Queue(channel_name, amqp_host) append_messages.declare_channel(persistent_queue) code=append_messages.send(message) append_messages.disconnect() del append_messages return 200 except: return 501 if __name__ == '__main__': handler() |
We can, in this file, create the code that queues messages to a message broker (such as RabbitMQ or SQS).
Or we can connect our Lambda function to a server using the SSH protocol (with paramiko in python).
Or simply by recalling other Lambda functions in a recursive way.
We now have a completely Serverless tool to manage our cron or scheduled tasks.
Good fun.
