AWS VPC via Kinesis Setup

This document provides instructions for configuring the collection of AWS VPC Flow Logs with AWS Kinesis.

Limitations/Notes

• This is for provisioning(create/delete) only. Edits must be done manually but that’s largely limited by AWS

• This provisions everything required within a region.

• This must be run in every region that contains target VPCs

• Once the initial provisioning is done, adding additional VPCs, within a region, is trivial through the AWS GUI

CloudFormation Steps

1. Setup supporting configuration with Cloudformation template

2. Create VPC Flow Logs that publishes to Kinesis

Setup Cloudformation template

Setup supporting configuration with Cloudformation template.

1. In the AWS Console select Services and type cloudformation into the search bar

2. Click Create stack then With new resources(standard)

3. You will see the import overview, click next

4. Make sure Amazon S3 URL is checked and input the following URL then click nexthttps://neto-downloads.s3.amazonaws.com/aws/vpc-flow-logs/Netography-AWS-Cloud-Formation.v2.(kinesis).json

5. Choose a stack name

6. Select the number of Kinesis shards 1 is defaultEach shard ingests upto 1 MiB/second and 1000 records/second and emits up to 2 MiB/second.

7. Select TargetVPC and click next

8. Add tags for the stack (optional) and click next

9. Review and check the "I acknowledge that AWS CloudFormation might create IAM resources with custom names."

10. Now click Create stack

11. Take note of the information on the Outputs tab

Create VPC Flow Logs

Create VPC Flow Logs that publishes to Kinesis

1. In the AWS Console select Services and type vpc into the search bar

2. Click VPC then select your VPC and click the Flow Logs tab

   
3. Then click create flow Log

Filter	All
Maximum aggregation interval	1 minute
Destination	Send to CloudWatch Logs
Destination log group	Select the destination log group
IAM role	Hot
Access tier	Select the IAM role
Format	Custom format
Access tier	Hot
Log format	${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status} ${tcp-flags} ${type} ${pkt-dstaddr} ${pkt-srcaddr} ${instance-id} ${vpc-id} ${az-id} ${sublocation-id} ${sublocation-type} ${subnet-id}

1. Click create

Netography Portal Steps

1. Navigate to "Traffic Sources"

2. Click "Add Traffic Source".

3. Click the "Show Advanced" button at the top of the page.

4. Click "AWS Kinesis".

Configuration

The following fields are specific to the AWS Kinesis configuration.

Field	Required	Description	Examples
Region	yes	Location of the flow source	us-east-1
Stream	yes	Kinesis data stream name

Authentication

Netography Fusion can access your AWS account using one of two different methods:

1. IAM user via an Access Key ID & Secret Access Key

2. IAM Roles using a Custom Trust Policy created by Netography.

AWS Access Key

To configure access via Access Key/Secret, select the "Key/Secret" Authentication Type. The values for the ID and Secret are accessible in the AWS IAM console.

AWS IAM Roles

You can use an IAM role in Netography Fusion to access your Cloud Flow Logs for flow ingest or account data for the AWS Context Integration. To enable this, go to the portal and retrieve the AWS Account ID and External ID from your Account Settings. Navigate to the gear button on the top right to view your Account Settings to see the Overview tab as shown below:

In AWS, you will configure permissions using the Account ID grabbed from above to create the IAM Role. When configured, AWS creates the Amazon Resource Number (ARN) for the role. For more information in configuring the permissions to the Account ID, refer to the external ID guide.

🚧The newly created ARN is required in order to configure IAM role access in the Netography Fusion portal.

Once the ARN has been created, the remaining steps are to toggle the Authentication Type to Role in your AWS

S3 configuration settings, input the AWS Account ID grabbed earlier from your Netography account settings, and the supply the ARN configured from AWS as shown below: