AWS VPC via Kinesis Setup
This document provides instructions for configuring the collection of AWS VPC Flow Logs with AWS Kinesis.
Limitations/Notes
- This is for provisioning(create/delete) only.Β Edits must be done manually but thatβs largely limited by AWS
- This provisions everything required within a region.Β
- This must be run in every region that contains target VPCs
- Once the initial provisioning is done, adding additional VPCs, within a region, is trivial through the AWS GUI
CloudFormation Steps
- Setup supporting configuration with Cloudformation template
- Create VPC Flow Logs that publishes to Kinesis
Setup Cloudformation template
Setup supporting configuration with Cloudformation template.
-
In the AWS Console select Services and type cloudformation into the search bar
-
Click Create stack then With new resources(standard)
-
You will see the import overview, click next
-
Make sure Amazon S3 URL is checked and input the following URL then click nexthttps://neto-downloads.s3.amazonaws.com/aws/vpc-flow-logs/Netography-AWS-Cloud-Formation.v2.(kinesis).json
-
Choose a stack name
-
Select the number of Kinesis shards 1 is defaultEach shard ingests upto 1 MiB/second and 1000 records/second and emits up to 2 MiB/second.
-
Select TargetVPC and click next
-
Add tags for the stack (optional) and click next
-
Review and check the "I acknowledge that AWS CloudFormation might create IAM resources with custom names."
-
Now click Create stack
-
Take note of the information on the Outputs tab
Create VPC Flow Logs
Create VPC Flow Logs that publishes to Kinesis
-
In the AWS Console select Services and type vpc into the search bar
-
Click VPC then select your VPC and click the Flow Logs tab
-
Then click create flow Log
Filter | All |
Maximum aggregation interval | 1 minute |
Destination | Send to CloudWatch Logs |
Destination log group | Select the destination log group |
IAM role | Hot |
Access tier | Select the IAM role |
Format | Custom format |
Access tier | Hot |
Log format | ${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status} ${tcp-flags} ${type} ${pkt-dstaddr} ${pkt-srcaddr} ${instance-id} ${vpc-id} ${az-id} ${sublocation-id} ${sublocation-type} ${subnet-id} |
- Click create
Netography Portal Steps
Navigate to Traffic Sources, and click "Add Traffic Source", then select AWS Kinesis.
Configuration
The following fields are specific to the AWS Kinesis configuration.
Field | Required | Description | Examples |
---|---|---|---|
Region | yes | Location of the flow source | us-east-1 |
Stream | yes | Kinesis data stream name |
Authentication
Netography Fusion can access your AWS account using one of two different methods:
- IAM user via an Access Key ID & Secret Access Key
- IAM Roles using a Custom Trust Policy created by Netography.
AWS Access Key
To configure access via Access Key/Secret, select the "Key/Secret" Authentication Type. The values for the ID and Secret are accessible in the AWS IAM console.
AWS IAM Roles
You can use an IAM role in Netography Fusion to access your Cloud Flow Logs for flow ingest or account data for the AWS Context Integration. To enable this, go to the portal and retrieve the AWS Account ID and External ID from your Account Settings. Navigate to the gear button on the top right to view your Account Settings to see the Overview tab as shown below:
In AWS, you will configure permissions using the Account ID grabbed from above to create the IAM Role. When configured, AWS creates the Amazon Resource Number (ARN) for the role. For more information in configuring the permissions to the Account ID, refer to the external ID guide.
The newly created ARN is required in order to configure IAM role access in the Netography Fusion portal.
Once the ARN has been created, the remaining steps are to toggle the Authentication Type to Role in your AWS
S3 configuration settings, input the AWS Account ID grabbed earlier from your Netography account settings, and the supply the ARN configured from AWS as shown below:
Updated 4 months ago