Snowflake

Learn about streaming data into snowflake using Couchdrop.

Introduction to Snowflake with Couchdrop

Couchdrop’s native support for inserting data into Snowflake through file actions makes it incredibly simple to move data from virtually any source into your data warehouse. You can configure Couchdrop to automatically ingest files and stream their contents directly into Snowflake tables. This enables seamless integration of data coming from cloud storage, SFTP/FTP servers, APIs, or inbound webhooks - all with minimal setup.

With this capability, teams can unlock fast and reliable ingestion of structured or semi-structured data without needing to manage ETL infrastructure. Couchdrop handles the heavy lifting of getting data into Snowflake so it’s immediately available for analysis.

How Snowflake works with Couchdrop

Snowflake is configured in Couchdrop as a platform integration. This type of integration does not appear directly as a virtual folder as Couchdrop needs to understand how to translate the data into rows in Snowflake.

Instead; Snowflake is configured as a connection and then you can use this connection as a file action inside an Transfer Automations or File Actions.

When you configure the file action - you provide details about the data table schema in Snowflake and how that maps to the data files being ingested by Couchdrop.

Requirements

To connect to Snowflake in Couchdrop you will need:

• A snowflake account and access to the table you are working with

• Access credentials; account ID, access key

• A location or inbound method in Couchdrop that you are ingesting data via

Configuration Steps

1. Log in to Couchdrop and add a new system integration

2. Select Snowflake from the list of available integrations

3. Provide the details above

4. Click Add System Integration and follow the wizard

5. Create an automation or file action and select Snowflake as the platform type.

Common issues connecting to Snowflake

For a comprehensive guide on generating an access token, see https://docs.snowflake.com/en/user-guide/programmatic-access-tokens

The network policy setup for your user in Snowflake can block us from authenticating and connecting to your Snowflake account. The authentication policy setup for your user in Snowflake can prevent the network policy from blocking us, if it is an issue, by running

CREATE|ALTER AUTHENTICATION POLICY <my_policy_name>
  PAT_POLICY=(
    NETWORK_POLICY_EVALUATION = NOT_REQUIRED
  );

The authentication policy can also block us from connecting if the authentication methods have been configured, and do not include PROGRAMMATIC_ACCESS_TOKEN in the available authentication methods. To see if this is the case you can run

DESCRIBE AUTHENTICATION POLICY <authentication_policy_name>;

To add the authentication method to your authentication policy, you can run

ALTER AUTHENTICATION POLICY <authentication_policy_name>
  SET AUTHENTICATION_METHODS = ('PROGRAMMATIC_ACCESS_TOKEN');

To generate the programmatic access token

ALTER USER IF EXISTS <example_user> ADD PROGRAMMATIC ACCESS TOKEN example_token;