BigQuery

How to connect using the BigQuery system integration

Introduction to BigQuery with Couchdrop

Couchdrop’s native support for inserting data into BigQuery through file actions makes it incredibly simple to move data from virtually any source into your data warehouse. You can configure Couchdrop to automatically ingest files and stream their contents directly into BigQuery tables. This enables seamless integration of data coming from cloud storage, SFTP/FTP servers, APIs, or inbound webhooks - all with minimal setup.

With this capability, teams can unlock fast and reliable ingestion of structured or semi-structured data without needing to manage ETL infrastructure. Couchdrop handles the heavy lifting of getting data into BigQuery so it’s immediately available for analysis.

How BigQuery works with Couchdrop

BigQuery is configured in Couchdrop as a platform integration. This type of integration does not appear directly as a virtual folder as Couchdrop needs to understand how to translate the data into rows in BigQuery. BigQuery is configured as a connection and then you can use this connection as a file action inside an Transfer Automations or File Actions.

When you configure the file action - you provide details about the data table schema in BigQuery and how that maps to the data files being ingested by Couchdrop.

Requirements

To connect to BigQuery in Couchdrop you will need:

• A BigQuery account, with a table created that you want to upload data into

• Information about your table in BigQuery, such as your project ID, your dataset name and your table name

• Service credentials JSON file

  • You can create this within the APIs & Services -> Credentials section in Google Admin Console

  • Create a service account, and then create a key for that account, and download the JSON file associated to use within Couchdrop

Configuration Steps

1. Log in to Couchdrop and add a new system integration

2. Select BigQuery from the list of available integrations

3. Provide the details above

4. Click Add System Integration

5. Create an automation or file action and select BigQuery as the platform type.