Google BigQuery

How to connect using the BigQuery system integration

Overview

Couchdrop can insert data from files directly into Google BigQuery using file actions and Automations.

This allows data received through cloud storage, SFTP/FTP, APIs, or inbound webhooks to be sent into an existing BigQuery table without requiring a separate transfer process.

How BigQuery works with Couchdrop

BigQuery is configured in Couchdrop as a System Integration.

Unlike storage connections, a BigQuery integration does not appear as a Virtual Folder. Instead, the connection is used as a file action inside an Automation.

When configuring the BigQuery file action, you provide information about your BigQuery table schema and define how the data being ingested by Couchdrop maps to that schema.

How to connect BigQuery

Before you start

To connect Google BigQuery to Couchdrop, you will need:

• A Google BigQuery account
• An existing BigQuery table that you want to upload data into
• Your BigQuery Project ID
• Your Dataset name
• Your Table name

1

Create service account credentials

In Google, navigate to APIs & Services > Credentials.

Create a service account, then create a key for that account and download the associated JSON credentials file.

You will upload this file when configuring the BigQuery System Integration in Couchdrop.

2

Create a BigQuery system integration

Log in to Couchdrop and add a new System Integration.

Select BigQuery from the list of available integrations.

3

Configure the BigQuery connection

Provide the required BigQuery details and upload your service credentials JSON file.

Click Add System Integration to save the connection.

4

Use BigQuery in an Automation

Create an Automation or file action and select BigQuery as the platform type.

Configure the file action with your BigQuery table schema and define how the data being ingested by Couchdrop maps to the table.

BigQuery configuration details

Configuration	Description
Project ID	Identifies the Google Cloud project containing your BigQuery resources.
Dataset name	Identifies the BigQuery dataset containing the destination table.
Table name	Identifies the existing BigQuery table that will receive the data.
Service credentials JSON file	Provides the service account credentials Couchdrop uses to connect to Google BigQuery.
Table schema mapping	Configured within the file action to define how data being ingested by Couchdrop maps to the BigQuery table schema.