AWS Route 53 DNS Logs via S3 Setup (Console)

If you have already configured your AWS account to ingest VPC flow logs to Fusion using an S3 bucket and IAM role, the additional steps required to ingest DNS resolver query logs are:

1. Configure Resolver Query Logging to an S3 bucket in AWS Route 53
2. Add a DNS AWS S3 traffic source in Fusion for each VPC configured to log resolver queries.

You must collect the information in the table below to add the traffic source to Fusion in step 2.

Configuring Resolver Query Logging in AWS to an S3 bucket

📘

Choosing the S3 bucket to log to

You can utilize the same S3 bucket you send VPC flow logs to for resolver query logs. Using the same S3 bucket simplifies configuration, as the existing IAM policy permissions for Fusion to read that bucket have already been configured.

To view the S3 bucket that is configured for an existing VPC configured in Fusion, go to:
SETTINGS > Traffic Sources, click … next to the AWS flow source, and select Edit.

If you wish to create a new S3 bucket, ensure you update the Resource section of the IAM policy associated with the IAM role you created for Netography to include the new S3 bucket ARN.

1. Sign in to the AWS Management Console and open the Route 53 console at
   https://console.aws.amazon.com/route53/

2. Expand the Route 53 console menu. Choose the three horizontal bars in the console's upper left corner.

3. Within the Resolver menu, choose Query logging.

4. In the Region selector, choose the AWS Region where you want to create the query logging configuration. This must be the same region where you created the VPCs for which you want to log DNS queries. If you have VPCs in multiple Regions, you must create one query logging configuration for each Region.

5. Choose Configure query logging and fill in the following:

   1. Query logging configuration name - Enter a descriptive name for your query logging configuration.
   2. Query logs destination - Select S3 Bucket
      1. Choose the S3 bucket to log to.
      2. If you are using the same S3 bucket as you are using for flow logs, you can separate DNS logs from VPC flow logs by adding to the end of the S3 bucket a folder name, such as dnslogs.
         Make a note of the S3 bucket and prefix
   3. VPCs to log queries for - Check the check box for each VPC in the current Region that you want to log then choose Choose.
      Make a note of the the VPC IDs of each VPC you chose.

For additional details on configuring resolver query logging, see AWS documentation:

https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resolver-query-logging-configurations-managing.html

Adding a DNS AWS S3 Traffic Source to Fusion

You must add a new DNS traffic source in Fusion for each VPC configured in the previous step.

1. In Fusion, Navigate to Settings > Traffic Sources
2. Click ADD TRAFFIC SOURCE
3. In the DNS section, click AWS S3 VPC
4. Complete the form with the information you collected in the previous step.

5. In the Authentication section, select Role for AWS Authentication Type and enter the ARN for the IAM role you are using to integrate Fusion to AWS.

📘

Fusion role permissions required to add a new DNS traffic source

To add a new DNS traffic source, the user's Role in Fusion must have theCloud Providers > Manage permission. This setting is found in Settings > Roles > Setup.

Optional AWS Configuration Step: Adding an SNS Topic and SQS Queue

❓

Should I skip this step?

Fusion will poll the configured S3 bucket for new logs every minute. However, if you add a SNS topic and SQS queue for a VPC you configured in the previous step, Fusion will receive a notification from AWS when a new log is available and immediately trigger reading that log. This means that DNS resolver logs will be ingested up to 59 seconds faster (~30 seconds on average) if you complete this step. The cost of this added efficiency is the additional configuration required for this step (AWS usage charges for SQS/SNS for this purpose are less than 1 cent per VPC per month).

You can safely skip this step and come back to it later if you decide it is important.

For each VPC you configured for resolver query logging:

1. Create a SNS topic
2. Create a SQS queue
3. Subscribe the SQS queue to the SNS topic
4. Add event notifications (destined for the SQS queue) in the S3 bucket
5. Add the SQS queue ARN to IAM permission policy attached to the Fusion IAM role.

1. Create a SNS Topic

1. Sign in to the Amazon SNS console at:
   https://console.aws.amazon.com/sns/home
2. On the navigation panel, choose Topics.
3. On the Topics page, choose Create topic.
4. For Type, choose Standard.
5. Enter a Name for the topic (e.g. fusion_dns_vpc-04abc123000de4500).
6. Choose Create topic.
7. Make a note of the topic ARN displayed.

For additional details on creating a SNS topic, see AWS documentation:

https://docs.aws.amazon.com/sns/latest/dg/sns-create-topic.html

2. Create a SQS queue

1. Open the Amazon SQS console at:
   https://console.aws.amazon.com/sqs/
2. Choose Create queue.
3. For Type, use the default Standard queue type.
4. Enter a Name for your queue (e.g. fusion_dns_vpc-04abc123000de4500).
5. In the Configuration section, set the Message retention period to 1 day.
6. In the Access Policy section, set the Method to Advanced, and paste the following JSON, updating the aws:SourceArn value from "arn:aws:s3:::<bucketname>" to the S3 bucket ARN you are using.
   JSON

   {
      "Version": "2012-10-17",
      "Id": "PushMessageToSQSPolicy",
      "Statement": [
         {
            "Sid": "allow-sns-to-send-message-to-sqs",
            "Effect": "Allow",
            "Principal": {
               "AWS": "*"
            },
            "Action": "sqs:SendMessage",
            "Resource": "*",
            "Condition": {
               "StringLike": {
                  "aws:SourceArn": "arn:aws:s3:::<bucketname>"
               }
            }
         }
      ]
   }
   

7. Choose Create queue. Amazon SQS creates the queue and displays the queue's Details page.

For additional details on creating a SQS queue, see AWS documentation:

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/creating-sqs-standard-queues.html

3. Subscribe SQS Queue to SNS Topic

1. After creating the queue, you will see the queue's Details page. Under the SNS Subscriptions Tab, select the Subscribe to Amazon SNS Topic button.
2. Choose Enter Amazon SNS topic ARN and then enter the SNS topic ARN you noted in the previous step.
3. Choose Save.
4. Make a note of your SQS URL and SQS ARN.

4. Add event notifications (destined for the SQS queue) in the S3 bucket

1. In the AWS console, navigate to S3, select your S3 bucket, and click the Properties tab.

2. In the Event Notifications section, click Create event notification.

3. Enter a name for the notification (e.g. fusion_dns_notification_vpc-04abc123000de4500).

4. The prefix field needs to be set to the path to which the resolver logs for the VPC are being written. This is NOT the same prefix as you set when first configuring the DNS resolver query logs for the VPC (but it starts with that if you set it).

   If you set a prefix when configuring the DNS resolver query logs for the VPC to dnslogs, this path will be: dnslogs/AWSLogs/ACCOUNTID/vpcdnsquerylogs/VPCID. For example, if the query log prefix was set to dnslogs, AWS account ID is 1234567, and the VPC ID is vpc-987654321, the prefix field should be set to: dnslogs/AWSLogs/1234567/vpcdnsquerylogs/vpc-987654321.

   If you did not set a prefix, this path will be: AWSLogs/ACCOUNTID/vpcdnsquerylogs/VPCID. For example, if the AWS account ID is 1234567, and the VPC ID is vpc-987654321, the prefix field should be set to: AWSLogs/1234567/vpcdnsquerylogs/vpc-987654321.

5. In the Event Types section, check All objects create events.

6. In the Destination section, select SQS Queue.

7. Enter the SQS Queue ARN

8. Click Save.

5. Add SQS ARN to IAM permission policy attached to the Fusion IAM role

To integrate with AWS, Fusion is configured with an IAM role with a permission policy attached. You must add each SQS ARN you created in the previous steps to the Resource section of this permission policy so that Fusion can receive notifications that new logs are available to ingest.

If you are creating multiple SQS queues, you can collect all the SQS ARN you created in the previous steps and add them to the permission policy in one final step.

📘

Identifying the right IAM permission policy to update

The IAM policy name isNetographyPolicyin Fusion's default CloudFormation, but if it was created with the AWS Console or a non-default name is used, you will need to find the right policy. If you have a small AWS deployment, searching for Netography or Fusion may be a quick way to find it.

If you do not know the policy name, you can identify it by:

1. In Fusion, navigate to Settings > Traffic Sources, click … next to any AWS flow source using this role, and select Edit. Make a note of the AWS ARN in the Authentication section of this page.
2. In the AWS Console, under Services go to IAM, select Roles from the menu, and then search for the last section of the ARN (e.g. if the ARN is arn:aws:iam::1234567890:role/NetographyFusionRole, search for NetographyFusionRole).
3. Select the role for the ARN to bring up the role details page. This page will contain the name of the permissions policy associated with this role. Select the permission policy.

1. Sign in to the AWS Management Console and open the IAM console at:
   https://console.aws.amazon.com/iam/
2. In the navigation pane, choose Policies.
3. In the list of policies, choose the policy name associated with the Netography Fusion IAM role (see the call-out above if you do not know what this is).
4. Choose the Permissions tab, and then choose Edit.
5. Choose the JSON option to modify your policy.
6. In the Resource section of the JSON, add the SQS ARNs that you created in the previous steps.
7. Choose Save changes to save your work, setting it as the new default policy version.

For additional details on updating the permission policy, see AWS documentation:

https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-edit.html#edit-managed-policy-console