Netography Fusion ingests VPC flow logs from Google Cloud Platform (GCP) via a GCP Pub/Sub subscription. The steps to integrate with GCP are:

1. Enable VPC flow logs
2. Create a Pub/Sub topic
3. Create a Cloud Logging Sink Pub/Sub for the topic
4. Create a Pub/Sub Pull Subscription to the topic
5. Add Netography's GCP service account as a principal for the Pub/Sub subscription
6. In Fusion, Add GCP as a new flow source.

In addition to ingesting VPC flow logs, you may want to enrich them with context from GCP resources by adding the GCP Context Integration

🤖

Using Terraform to automate onboarding

The instructions linked from this page are suitable for onboarding one or a small number of cloud accounts manually or using as a reference for building automation for larger scale deployment. In addition to these instructions, Netography provides a Terraform project, NetoOnboard, that provides Netography Fusion Cloud Onboarding Automation for AWS Organizations, Azure Tenants, and GCP Organizations.

This automation provides the following:

• Enables and configure VPC flow logging based on a simple policy and tags that defines which VPC are in scope.
• Deploy all the infrastructure required to integrate to Fusion across multiple accounts (AWS), subscriptions (Azure), and projects (GCP) in a single deployment
• Adds VPCs configured for flow logging to Netography Fusion as traffic sources.
• Deploys a single AWS Lambda function, Azure Function, or Google Function that provides context enrichment across all the accounts/subscriptions/projects as an outbound push from your cloud to the Fusion API, eliminating the need to add context integrations from the Fusion portal, to grant Netography permissions to directly enumerate resource properties, or to add individual context integrations in Fusion for each cloud account.
• Monitor for VPC changes and trigger enabling and configuring flow logs, and onboarding to Fusion new VPCs that are in scope, and offboarding VPCs that are removed or no longer in scope.

For access to Netography's terraform onboarding automation contact Netography Support.

Prerequisites

• If you have GCP organization policy constraints in place, you may be unable to perform these steps until you update the organizational policies. If you receive an error referring to an organization policy, update the policy and retry. Updating an organization policy requires the Organization Policy Administrator role (roles/orgpolicy.policyAdmin).
• You need sufficient permissions in GCP to perform each step. The GCP documentation referenced in each step details the roles and permissions associated with that action.

GCP Setup

1. Enable VPC flow logs

You can skip this step if you already have VPC flow logs enabled for the networks to monitor.

Follow these steps using the configuration settings below: GCP: Enable VPC Flow Logs when you create a subnet

Additional instructions for enabling GCP VPC flow logs are available at GCP: Use VPC Flow Logs.

Flow Log Configuration

GCP Console Steps

1. Go to the VPC networks page in the Google Cloud console.
2. Select a VPC to go to the VPC network details page.
3. Select the Subnets tab to see the list of Subnets for the VPC.
4. Select the subnet(s) to enable flow logging for.
5. Click Flow Logs above the subnets list, then select Configure from the drop-down menu.
6. Fill out the form using the above configuration values, then click Save.

2. Create a Cloud Pub/Sub topic

Create a Cloud Pub/Sub topic to publish flow logs to. If you are onboarding an individual GCP project, you can create the topic as part of creating the sink in step 3. If you are onboarding multiple projects at an organization or folder level, you must create the topic in a project you will use for centralized logging resources.

To separately create the topic, follow these steps using the configuration settings below: GCP: Create a Topic

Pub/Sub Topic Configuration

Note: GCP charges for unacknowledged message retention over 1 day. In most circumstances, the messages will be acknowledged and removed from the topic in real-time, but retention will ensure there is no data lost unless the logs are not read in that time period. You can adjust the retention period based on your organization's requirements.

GCP Console Steps

1. Go to the Pub/Sub Topics page in the Google Cloud console.
2. Click Create Topic.
3. Fill out the form using the above configuration values, then click Save.

3. Create a Cloud Logging Sink Pub/Sub

Create a Cloud Logging Sink with a destination of Cloud Pub/Sub topic, using the topic you created in step 2 or creating the topic in the process.

ℹ️

Use an aggregated sink for onboarding multiple projects in a GCP organization

If you are onboarding multiple projects in a GCP organization, you can use an aggregated sink to simplify the deployment. Using an aggregated sink lets you create 1 sink for a GCP organization or folder rather than 1 sink per project. If you are using an aggregated sink, the GCP flow log sampling percentage needs to be the same value across all the VPCs logging to the sink.

To use an aggregated sink, you will need Owner access to the sink's destination, and to perform the following steps when creating the sink:

1. Select the organization or folder to onboard in the GCP project picker.
2. When creating the sink, select Include logs ingested by this folder and all child resourcesin the section Choose logs to include in sink (this option will not appear if you selected a project).
3. Add the sink's writer identity as a principal by using IAM, and then grant it the Pub/Sub Publisher role (roles/pubsub.publisher). See GCP: Route logs to supported destinations > Set destination permissions

For more information on aggregated sink configuration, see GCP: Collate and route organization- and folder-level logs to supported destinations

Follow these steps using the configuration settings below: GCP: Create a sink.

Cloud Logging Sink Configuration

Inclusion Filter

The inclusion filterresource.type="gce_subnetwork" will include all VPC flow logs in the sink. You can add filters using inclusion or exclusion based on your desired configuration. For example, to only publish to the sink VPC flow logs that are ingress/exgress a VPC (excluding internal intra-VPC traffic), the inclusion filter would be:

resource.type="gce_subnetwork"and NOT ( jsonPayload.src_vpc.vpc_name:_ AND jsonPayload.dest_vpc.vpc_name:_ )

Adding this filter at the sink will still generate the VPC flow logs for intra-VPC traffic but will not deliver those logs to Fusion (this may be useful if you are using intra-VPC flow logs for other purposes). To filter which VPC flow logs are generated, set the filter in the VPC flow log configuration instead of at the sink (see GCP: Filtering VPC flow logs

GCP Console Steps

1. Go to the Log Router page in the Google Cloud console.
2. Select the project (or folder or organization if using an aggregated sink) to create the sink in.
3. Click Create sink.
4. Fill out the form using the above configuration values, then click Save.

4. Create a Pub/Sub Pull Subscription to the topic

Follow these steps using the configuration settings below: GCP: Create a pull subscription.

Pub/Sub Subscription Configuration

Default values for all other fields can be used.

GCP Console Steps

1. Go to the Topics page in the Google Cloud console.
2. Click ⋮ next to the topic you created in previous step.
3. From the context menu, select Create Subscription.
4. Fill out the form using the above configuration values, then click Save.

Note: Alternatively, you can create a subscription from the Subscriptions page by entering the Topic ID from the previous step.

5. Add Netography's GCP service account as a principal to the Pub/Sub subscription

To grant Netography access to read logs from the Pub/Sub subscription, add the Netography GCP service account as a new principal in the subscription.

📘

If you have a Domain Restricted Sharing Organizational Policy

If your GCP organization has an Organizational Policy constraint for Domain Restricted Sharing constraints/iam.allowedPolicyMemberDomains, you must add a rule to that pollicy to allow Netography's GCP customer ID C04ddcbu8before adding the principal to the Pub/Sub subscription.

This constraint is the default setting for all GCP organizations created on or after May 3, 2024.

If this policy restriction exists and you do not add the rule, you will receive the following error when you save the Pub/Sub Subscription:
IAM policy update failed - The ‘Domain Restricted Sharing’ organization policy (constraints/iam.allowedPolicyMemberDomains) is enforced.

For detailed instructions and options for configuration, see GCP: Restricting Domains

Domain Restricted Sharing Configuration

Field	Value
Policy Value	[email protected]
Policy Type	Pub/Sub Subscriber
Custom Value	C04ddcbu8

GCP Console Steps

To update your Organizational Policy to allow you to grant Netography's GCP service account access to the Pub/Sub subscription:

1. Go to the Organization Policies page in the Google Cloud console IAM & Admin section.
2. Next to where it says Filter above the list of policies, type Domain restricted sharing.
3. You should see 1 policy with that name in the list, with ID constraints/iam.allowedPolicyMemberDomains. Click ⋮ and Edit Policy.
4. Add a new rule (or add a value to an existing rule) for the policy under Rules using the above configuration values, then click Set Policy.

Follow these steps to add a principal to the subscription: GCP: Access Control for Pub/Sub > Controlling access through the Google Cloud Console

Pub/Sub Subscription Principal Configuration

GCP Console Steps

1. Go to the Subscriptions page in the Google Cloud console in the Pub/Sub section.
2. Select the subscription you created in the previous step to bring up the subscription info panel on right.
3. Select Add Principal in the info panel for the subscription.
4. Fill out the form using the above configuration values, then click Save.

Netography Fusion Setup

6. Add a new GCP flow source to Fusion

In the Fusion portal, click the gear icon to go to Settings, navigate to Flow Sources, click Add Flow Source, select GCP, and fill out the form using the configuration below.

GCP Flow Source Configuration

The following fields are specific to the GCP configuration.