In this article we will describe how we have automated a workflow thanks to the services managed by Amazon AWS. This has allowed us to have a unique and centralized service that can be managed by anyone.
Let’s come to the problem.
To manage hundreds of DNS zones for our customers automatically, where possible, or to delegate them to people without technical skills.
We thought about the solution, which seemed more reasonable to us. To create a webservice, interact with it through an API. We wanted to do it without installing on our servers anything or almost nothing, not to have another service to manage and maintain.
So we immediately thought about the integration of Amazon AWS services.
To move the management of all our areas on Route53.
To use Gateway API as an API to perform the set of operations that interest us, each with their own call.
To have the ability to perform a code-side check on API calls before they are turned to Route53. Here the choice fell on the Lambda functions.
In this way our API can easily be passed to automation scripts for creating, modifying, deleting zones, records or settings in general.
Or through simple web forms gived to our customers or any person in our company in order to provide the ability to manage DNS in a simple and controlled way. Possibly without creating problems!
Well we have the idea, now we need to move on to the realization, studying the abundant documentation of AWS on the three selected services.
To maintain the “infrastructure as code” approach that we are always trying to carry out, we decided to configure all the services, not through the convenient manual console of AWS, but through Ansible. This allows us in case of problems or updates to completely recreate our API very quickly and automatically.
To create functions we used the procedure described in this previous article.
So according to the scheme described, we created the python file manage_dns.py, controls and logic for DNS management are managed. The following is an excerpt:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
#!/usr/bin/python import boto3, datetime def add_zone(**kwargs): client = boto3.client('route53') now = datetime.datetime.now() zone_id=search_id_zone(domain=kwargs['domain']) if check_zone_exists(zone_id): print("Zone already exist") return zone_id else: response = client.create_hosted_zone( Name=kwargs['domain'], CallerReference=now.strftime("%Y%m%d%-H%M%S") ) print("Zone created") return response['ResponseMetadata']['HTTPStatusCode'] def del_zone(**kwargs): client = boto3.client('route53') zone_id=search_id_zone(domain=kwargs['domain']) if check_zone_exists(zone_id): response = client.delete_hosted_zone( Id=zone_id ) print("Zone deleted") return response['ResponseMetadata']['HTTPStatusCode'] else: print("Check zone failed. Zone does not exist or mismatch") return zone_id |
The vars_manage_dns.yml file where variables are placed:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
#Required install_libs: - datetime - dnspython include_local_files: - "{{ function_name }}" arn_role: arn:aws:iam::1234567890:role/ManageLambdaDNS cron_cloudwatch_rules: "no" #MB lambda_memory: 64 #seconds lambda_timeout: 30 description: "{{ function_name }} function management" env: prod #Not required account_id: 1234567890 uri_arn: arn:aws:apigateway:region:lambda:path/2015-03-31/functions/arn:aws:lambda:region:1234567890:function:my_api/invocations stage_name_api: managedns plan: burstLimit: 100 rateLimit: 10 limit: 1000 period: MONTH |
Finally, a hook_post_manage_dns.yml where, after creating the lambda function, we proceed to the creation and configuration of Gateway API.
In this case, not being the modules for ansible complete with all the parts necessary for us, we decided to call the aws command line from ansible.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 |
- name: LAMBDA FUNCTIONS POST HOOK | Create API Gateway command: aws apigateway create-rest-api \ --name "{{ function_name }}" \ --description "Manage DNS with Route53" \ --output text \ --query 'id' \ --region "{{ region }}" register: api_id - name: LAMBDA FUNCTIONS POST HOOK | Get resource API Gateway command: aws apigateway get-resources \ --rest-api-id "{{ api_id.stdout }}" \ --output text \ --query 'items[?path==`'/'`].[id]' \ --region "{{ region }}" register: resource_id - name: LAMBDA FUNCTIONS POST HOOK | Put method API Gateway command: aws apigateway put-method \ --rest-api-id "{{ api_id.stdout }}" \ --api-key-required \ --resource-id "{{ resource_id.stdout }}" \ --http-method POST \ --region "{{ region }}" - name: LAMBDA FUNCTIONS POST HOOK | Put integration command: aws apigateway put-integration \ --rest-api-id "{{ api_id.stdout }}" \ --resource-id "{{ resource_id.stdout }}" \ --http-method POST \ --type AWS \ --request-templates "{ \"application/json\"{{ ":" }} \"{\n \\\"domain\\\"{{ ":" }} \\\"$input.params('domain')\\\",\n \\\"object\\\"{{ ":" }} \\\"$input.params('object')\\\",\n \\\"value\\\"{{ ":" }} \\\"$input.params('value')\\\",\n \\\"record_type\\\"{{ ":" }} \\\"$input.params('record_type')\\\",\n \\\"ttl\\\"{{ ":" }} \\\"$input.params('ttl')\\\",\n \\\"record\\\"{{ ":" }} \\\"$input.params('record')\\\",\n \\\"action\\\"{{ ":" }} \\\"$input.params('action')\\\"\n }\" }" \ --integration-http-method POST \ --uri "{{ uri_arn }}" \ --region "{{ region }}" - name: LAMBDA FUNCTIONS POST HOOK | Put method response command: aws apigateway put-method-response \ --rest-api-id "{{ api_id.stdout }}" \ --resource-id "{{ resource_id.stdout }}" \ --http-method POST \ --status-code 200 \ --region "{{ region }}" - name: LAMBDA FUNCTIONS POST HOOK | Put integration response command: aws apigateway put-integration-response \ --rest-api-id "{{ api_id.stdout }}" \ --resource-id "{{ resource_id.stdout }}" \ --http-method POST \ --status-code 200 \ --selection-pattern "" \ --response-templates '{"application/json"{{ ":" }} ""}' \ --region "{{ region }}" - name: LAMBDA FUNCTIONS POST HOOK | Lambda add permission command: aws lambda add-permission \ --function-name "{{ function_name }}" \ --statement-id apigateway-{{ function_name }}_{{lookup('pipe','date +%Y%m%d%H%M')}} \ --action "lambda:InvokeFunction" \ --principal apigateway.amazonaws.com \ --source-arn "arn:aws:execute-api:{{ region }}:{{ account_id }}:{{ api_id.stdout }}/*/POST/" \ --region "{{ region }}" - name: LAMBDA FUNCTIONS POST HOOK | Lambda create deployment command: aws apigateway create-deployment \ --rest-api-id "{{ api_id.stdout }}" \ --stage-name "{{ stage_name_api }}" \ --region "{{ region }}" - name: LAMBDA FUNCTIONS POST HOOK | Create usage plan command: aws apigateway create-usage-plan \ --name "{{ function_name }}_plan" \ --description "Usage plan for {{ function_name }} API" \ --throttle burstLimit="{{ plan.burstLimit }}",rateLimit="{{ plan.rateLimit }}" \ --quota limit="{{ plan.limit }}",offset=0,period="{{ plan.period }}" \ --api-stages apiId="{{ api_id.stdout }}",stage="{{ stage_name_api }}" \ --output text \ --query 'id' \ --region "{{ region }}" register: plan_id - name: LAMBDA FUNCTIONS POST HOOK | Create API Key command: aws apigateway create-api-key \ --name '{{ function_name }}_apikey' --description 'Used for manage API DNS' \ --enabled \ --stage-keys restApiId="{{ api_id.stdout }}",stageName="{{ stage_name_api }}" \ --output text \ --query 'id' \ --region "{{ region }}" register: api_key_id - name: LAMBDA FUNCTIONS POST HOOK | Create usage plan key command: aws apigateway create-usage-plan-key \ --usage-plan-id "{{ plan_id.stdout }}" \ --key-type "API_KEY" \ --key-id "{{ api_key_id.stdout }}" \ --region "{{ region }}" |
At this point after running the script to generate the lambda function:
|
1 |
./make_function.sh --lambda_name manage_dns --no-confirm |
we will have activated the manage_dns function and the API Gateway.
We do not go into the details of the customization of the API, all the details on how we configured it can be found in the sample code published above or in general in the AWS documentation. We focus only on some points.
As you can see, in the image below, Gateway API shows us how our API is configured. Making a POST call to the invoke URL calls our Lambda function, which in turn performs the operations specified in the code. The Lambda function at the end of the invocation responds with an HTTP Status code.
Our invoke URL can be found on the following page:
This is the URL that we have to recall with our POST and on which we must authenticate.
Doing a POST on our API:
|
1 |
https://nameapi_code_etc_123.amazonaws.com/myapi/?domain=mydomain.ltd&object=manage_record&value=8.8.8.8&record_type=A&ttl=30&record=blabla&action=CREATE |
With this API call, we call the lambda function via Gateway API, which creates the record on our zone through the SDK of Route53.
Obviously it is possible, through the SDK of Route53 to do any operation on the DNS and thanks to the flexibility of Lambda we can insert any logic or intelligence in our function.
In the Gateway API console there are several interesting features to monitor our API, for example:
Or set usage limits to our Gateway API:
This API can be used with an API development environment like Postman, or integrated into an automated script/procedure.
Through the services and tools offered by AWS, we soon managed to create a complete and easy to use API. Thanks to Ansible we are able to make the deploy of a new version of API in a few tens of seconds.
The goal we set ourselves has been achieved.
Good development of your API!
