When we started developing Lambda functions to automate processes, or simply to delegate various jobs, compared to our infrastructure, we faced the problem of how to handle the deploy of functions and maintain the versions of this code.
There are many frameworks, more or less useful and well done, but none had everything we needed. First of all we wanted to keep a single versioning and profiling system. Generally we use Ansible and Git to manage our infrastructure. This allows us to have a simple and powerful versioning and provisioning.
We have therefore decided, at least for the simplest projects, to manage the deploy of Lambda functions in the same way.
We have created a directory in our github repository:
|
1 |
lambda_functions |
Within lambda_functions we find a structure of this type:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
files/ hello_world.py; other_lambda_function.py libs/ main.yml make_function.sh tasks/ hook_[pre|post]_hello_world.yml vars/ vars_hello_world.yml vars_other_lambda_function.yml |
files
It contains the files with the python code to be inserted in the Lambda function. So we need to create a file containing function name.
Here is the code of an example function hello_world.py:
|
1 2 3 4 5 6 7 8 9 10 11 |
#!/usr/bin/python def handler(event, context): return main(event, context) def main(event, context): print("Hello world!") if __name__ == '__main__': main() |
libs
Contains modules, libraries or classes called by Lambda functions.
tasks
It contains additional tasks to include in the main.yml. By default the file hook_ [pre | post]_function_name.yml is included to insert any customizations or additional tasks that Ansible can execute. Pre and post indicate that the script is executed before or after the creation of the function and the installation of any additional packages with pip.
Below is an example of pre and post task. In the first case variables passed to the Lambda function are encrypted. This operation must be done before generating the function (hook_pre_):
|
1 2 3 4 5 6 |
- name: LAMBDA FUNCTIONS PRE HOOK | Make KMS KEY set_fact: env_var: "{{ item }}" with_items: - "{{ env_var }}" - secret_var: "{{ token_priv | kms_encrypt(kms_key) }}" |
Below is a post-task example for creating a cloudwatch alarm metric, which is created after (hook_post_) that we have defined the Lambda function:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
- name: LAMBDA FUNCTIONS POST HOOK | Configure Metric Alarms ec2_metric_alarm: name: "Nuvola-prod-Metric" state: present metric: "CPUUtilization" namespace: "AWS/EC2" statistic: "Average" comparison: "<=" threshold: "{{ threshold }}" period: "{{ period }}" evaluation_periods: "{{ evaluation_periods }}" unit: "Count" dimensions: AutoScalingGroupName: "Nuvola_metric" alarm_actions: - "{{ arn_scaling_policy }}" - "{{ result_sns_topic }}" |
vars
It contains the variables needed to play the playbook. Specifically, vars_function_name.yml contains the variables to be modified to customize the Lambda function.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
#Required install_libs: - datetime - dnspython include_local_files: - "{{ function_name }}" arn_role: arn:aws:iam::1234567890:role/ManageLambdaDNS cron_cloudwatch_rules: cron(26 04 ? * MON *) #MB lambda_memory: 128 #seconds lambda_timeout: 60 description: "{{ function_name }} function management" env: prod |
main.yml
It contains instructions for downloading any dependencies specified in the vars file (install_libs), creating the Lambda package, uploading it online, deleting the newly created package locally and finally creating the cloudwatch rule (if requested).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
- hosts: localhost connection: local any_errors_fatal: true vars_files: - "vars/vars_{{ function_name }}.yml" - vars/vars_lambda_functions_secure.yml tasks: - name: LAMBDA FUNCTIONS | check if hook_pre exists stat: path=tasks/hook_pre_{{ function_name }}.yml register: hook_pre - include_tasks: "tasks/hook_pre_{{ function_name }}.yml" static: no when: hook_pre.stat.exists - name: LAMBDA FUNCTIONS | Make libs directory file: path: files/{{ function_name }} state: directory mode: 0755 tags: install_libs - name: LAMBDA FUNCTIONS | Install local lib args: chdir: "files/" command: pip install {{ item }} -t {{ function_name }} with_items: "{{ install_libs }}" tags: install_libs - name: LAMBDA FUNCTIONS | Copy local file on directory project copy: src: "files/{{ item }}.py" dest: "files/{{ function_name }}/" mode: 0755 with_items: "{{ include_local_files }}" - name: LAMBDA FUNCTIONS | Make archive zip args: chdir: "files/{{ function_name }}/" shell: zip -9 -r {{ function_name }}.zip {{ function_name }}.py . tags: make_archive - name: LAMBDA FUNCTIONS | Creation functions lambda: name: "{{ item.name }}" state: present zip_file: "{{ item.zip_file }}" runtime: "python2.7" role: "{{ arn_role }}" handler: "{{ function_name }}.handler" memory_size: "{{ lambda_memory }}" timeout: "{{ lambda_timeout }}" description: "{{ description }}" environment_variables: '{{ item.list_variables }}' region: "{{ region }}" with_items: - name: "{{ function_name }}" zip_file: "files/{{ function_name }}/{{ function_name }}.zip" list_variables: "{{ env_var }}" register: lambda_function - name: LAMBDA FUNCTIONS | Delete libs directory file: path: files/{{ function_name }} state: absent tags: delete_libs - name: LAMBDA FUNCTIONS | Creation rules cloudwatch cloudwatchevent_rule: name: "{{ function_name }}" schedule_expression: "{{ cron_cloudwatch_rules }}" state: present description: "Run {{ function_name }} scheduled task" region: "{{ region }}" targets: - id: "{{ function_name }}" arn: "{{ lambda_function.results.0.configuration.function_arn }}" input: '{{input_parameters_lambda | to_json}}' when: cron_cloudwatch_rules != "no" - name: LAMBDA FUNCTIONS | check if hook_post exists stat: path=tasks/hook_post_{{ function_name }}.yml register: hook_post - include_tasks: "tasks/hook_post_{{ function_name }}.yml" static: no when: hook_post.stat.exists |
make_function.sh
It contains the executable that calls the playbook Ansible.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
#!/bin/bash . $PATH_ABS/libs/check_ansible_version.sh . $PATH_ABS/libs/lib_color.sh MESSAGE="Do you want manage LAMBDA functions (y/n) ? " . $PATH_ABS/libs/confirm_action.sh cleanup() { exit 0 } trap cleanup INT TERM if [ -z $FUNCTION_NAME ]; then echo "Missing --lambda_name parameter, please enter lambda name function" exit 0 fi ansible-playbook \ --vault-password-file $PATH_ABS/secrets/vars_lambda_functions_secure.secret \ main.yml $TAGS_OPTION -e "function_name=$FUNCTION_NAME" |
At this point it is possible to launch the creation with:
|
1 |
./make_function.sh --lambda_name name_function |
inside
|
1 |
lambda_functions/ |
The function, assuming that there are no mistakes, should already be operational on AWS Lambda.
With this system we have collected in one single point our lambda functions in a centralized and controlled way with the github versioning. It is very easy, thanks to Ansible, to make the deploy of new versions.
Good fun.
We’re always looking for talented developers… find out all open job positions!
